Data gives a snapshot of the overall health of an OST program and is beneficial, as it provides a wealth of information and insight. Data allows program leadership to see areas of strength, can help to display active progress and brings awareness to potential improvements. Data enables us to gauge if the programming being offered is relevant or how effectively it is being implemented.
Data collected should be relevant to stakeholders such as the youth and their families, school districts, community and business partners and funders, as it offers tangible evidence of results.
Before you begin to collect data, it’s critical to define what you want to find out. What aspects of your program are you trying to measure? How does this align with your mission and overall goals? Broadly, you may want to assess: Who attends our program? Does this match with who we want attending? How engaged are youth in the programming and what results have we seen in participants?
Knowing the importance of data and the questions you want to ask is one thing, but actually going about collecting the data can be challenging.
Common Data Sources
- Attendance (both OST program and school day)
- Student grades
- Test scores
- Behavioral referrals
- Pre/Post tests
- Interviews and group feedback
OST programs also have an advantage in their flexibility to collect data that is unique to a program and it’s goals. For example, data could be collected on specific improvements in social-emotional behaviors, specific gain of skill sets or improvement in specific goals.
Key to accurate and timely data is the develop systems that collect data on a regular and on-going basis. It should be incorporated into the program implementation. For example, attendance should be taken every time an activity or event meets, grades and test scores should be collected as they are released. Pre-test and post-test should be offered at the beginning and end of a program or event. Observations, surveys or feedback sessions should be scheduled at regular intervals. Attempting to go back and collect data only when various reports are due will likely result in inaccurate information and is a burden on staff. Additionally, programs should ensure that all staff collecting data are properly trained on how to collect information. Lastly, examine what resources you have available to collect this data. Assess the proper staff, space, equipment and time available to collect what you need.
Data is a major part of the continuous improvement cycle. Programs should be able to draw a line from it’s mission and visions to it’s goals and to outcomes, as evidenced in data collected. That data collected in turn helps to reveal new goals and objectives.
21st CCLC Site Monitoring
Frequent Data & Site Monitoring Challenges for 21st CCLC grantees
With the substantial amount of reporting required by the 21st CCLC grant, it is easy to feel overwhelmed. First, make sure you have read and reviewed your grant application. Then make sure you are familiar with the concepts introduced in both presentations Be Prepared for Monitoring
Collecting Different Types of Data: The evaluation requires grantees to report information in different ways. Some evaluations want you to calculate the average daily attendance, some evaluations want the average weekly attendance, some evaluations want the unduplicated number of students served, and some evaluations only want you to count the students served 30 or more days. Therefore, it is critical to carefully read what each line item is asking and calculate accordingly. For example, if average weekly attendance is asked, you would take the attendance from each week, add them all up and divide it by the number of weeks the program operated as of the last Friday in April. Additionally, it is important that before your program even begins, look at the data that will be required and make sure that you have mechanisms to collect that data. These Strategies for Managing Paperwork offers practical suggestions that may help.
Collecting Data on Impact: Ensure that the goals set are realistic, achievable and measurable. If it is determined that the original goals were too ambitious, what progress was made? Could this goal be made with additional resources or moving forward does a more realistic goal need to be set?
Using Data: All of your hard work in completing the program evaluations should be capitalized upon, as there is a lot of valuable data that comes from the reports. However, grantees often find it challenging to work with Principals and School Improvement Teams on how to use program evaluation to monitor and improve program outcomes prior to the grant continuation. This presentation on using data for improvement details this issue.
21st CCLC Site Monitoring
- Monitoring and the On-site Monitoring Protocol
- Annual Local Evaluation Report Template (ALERT) 2016-17-This is a PDF document. (PDF)
- Virginia Department of Education On-Site Monitoring Report-This is a Word document. (Word)
- Single Sign-on for Web Systems (SSWS) Student Data Collection Survey Instructions-This is a Word document. (Word)
- Pre-Monitoring Technical Assistance Directions-This is a Word document. (Word)
- Pre-Monitoring Technical Assistance New Grant Implementation Guide-This is a Word document. (Word)
- Program Quality Self Assessment Tool 2017 -This is a PDF document.(PDF)
- Program Quality Self-Assessment Tool Guidebook 2017 -This is a PDF document.(PDF)
- 21st CCLC Monitoring Webinar (PPT)
Field Trips: During monitoring visits, grantees will sometimes find it confusing as to what field trips are considered meaningful and allowable. As outlined in the presentation Field Trips: “You Want to Go Where?” , trips should be connected to the academic or enrichment program and provide an educational experience from which students can grow academically or culturally. Trips can’t be for entertainment. Remember to document how all field trips are linked to program goals and academic learning. Lastly, don’t forget that lunch is not covered by grant funds.
Examples of Field Trips
- T.V. Stations
- Colleges & Universities
- Aquarium or Zoo
- Natural Sites (State Parks, Botanical Gardens, Caverns)
- Police Department
- Live Theatre
- Historical Sites
- Farms (Vegetable, Orchards, Creamery)
Afterschool Alliance: Articles on using how to determine the best data collection for your site and how to use that data
Wallace Foundation: This interim report presents early findings from a study of how afterschool systems build their capacity to understand and improve their practices through their data systems. It examines afterschool data systems in nine cities that are part of The Wallace Foundation’s Next Generation Afterschool System-Building initiative, a multi-year effort to strengthen systems that support access to and participation in high-quality afterschool programs for low-income youth.