The Continual Improvement Process

Define

Define your program goals and desired outcomes. Create an actionable plan broken down into annual, quarterly, monthly, weekly or project-based goals. Define and plan how to use resources such as supplies, budget, time and staffing. Supporting documents and goals can include Logic Models, SMART Goals, and program budgets.

Implement

Set your plan in motion. Check in on your original goals and resources while implementing and adjust accordingly.

Collect Data
Program Surveys Assessments Monthly or quarterly reports Annual reports Observations Student Attendance (afterschool and school day) Student Grades and test scores Behavior referrals Activity pre/post tests Activity surveys Informal feedback/observations/assessments
Analzye

Not only can use data be used for reporting, but it should be used recognize trends. What story is your data telling you? Compare hard data such as attendance and grades against the feedback you are getting from surveys, observations and speaking to people. What is working well in your program and where are the areas you can improve? Directors should also have staff analyze data for similar purposes so this is happening on a larger program level and on a smaller scale level.

Improve

Now that you have analyzed the data, you can create a plan of action to improve the quality of your program.

Let’s take an example of using this process and apply it to a hypothetical OST program. In this scenario, we will say that we have an existing afterschool program, that runs during the school year and is gearing up to return in the fall.

Define

Provide a high-quality afterschool program, Monday through Friday, at Jones Middle School, which serves an entire student population of 1000 students, in areas of academic support, physical health and recreation, social-emotional learning and family and community engagement. Students are from low-income, high-risk backgrounds and therefore stand to benefit from extra out-of-school time support and enrichment. Students attending at least once weekly will see improvements in grades, school-day time attendance, and behavioral referrals.

Supporting documents and goals

  • Program Logic Model (overall program goals)

 SMART goals (smaller sub-goals)

Implement
  • Recruit quality staff at the end of previous school year – summer
  • Budget is approved
  • Build program schedule
  • Purchase supplies, conduct staff trainings
  • Announce program start through various channels of communication
  • Recruit participants
  • Program begins
  • Monitor program
Collect Data
  • Program Surveys
  • Assessments
  • Monthly or quarterly reports
  • Annual reports
  • Observations
  • Student Attendance (afterschool and school day)
  • Student Grades and test scores
  • Behavior referrals
  • Activity pre/post tests
  • Activity surveys
  • Informal feedback/observations/assessments
Analyze

Overall, the Jones Afterschool Program is a successful, high-quality program that is well attended by students, provides positive student outcomes, is well received by staff, parents and community partners.

  • Program Survey Results: Students: Students enjoy the afterschool program. Getting academic support and getting to spend time with peers are the top two reasons students report staying afterschool. Students report wanting to spend less time on academics and more time engaged in recreational and social activities. Students also report wanting more snacks. Afterschool Staff: Overall pleased with the program. Staff feels the structure of the program allows them to be creative and empowered in the material and activities they are implementing in their activity/club. Multiple club sponsors are reporting are frustration with students arriving tardy to the club or activity. Parents: Parents are overall pleased with the program reporting they are happy to know their children are in a safe environment, engaged in an activity that is helping their academic or social needs. A common theme in parent feedback is suggesting activities we already have available.
  • Observations: Site Director formally observes each activity at minimum once per quarter, providing staff with feedback afterwards, using the Observation Report. Staff can use this in their own self-assessments and SMART goals. Observations vary by each program but overall trends include 1. Activities that are the most highly attended and well received when students have a positive and respectful relationship with the staff member 2. Staff that are consistent with rules and approach to students see less behavioral issues. 3. Students seem to enjoy sports, recreation, and special event activities the most.
  • Staff Assessments: Staff assessments are held at the end of the school year or semester, depending on activity and held at the activity half-way point for all first-time staff or activity leaders. Results vary by staff but overall, most staff are highly effective, a small portion superiorly effective and one staff that will not have their contract renewed.
  • Student Attendance (afterschool and school day): School day attendance is up 3% from last school year. Afterschool attendance shows that 97% of students attended the program at least one time over the school year and 67% of students attend at least once weekly, 45% of students stay more than once weekly.
  • Student Grades and test scores: 89% of students that attend afterschool at least once or more weekly, compared against students that do not attend. 93% of students enrolled in an academic enrichment program showed individual grade and test score improvements between the start and end of the academic activity.
  • Behavior referrals: Students that attend at least once weekly are 25% less likely than non-attending peers to have school-day behavioral referrals, and 50% likely if they attend 3 or more times weekly. Over the school year, 27 students were temporarily restricted from attending the program, with 2 students being indefinitely being excused from the program. Primary reasons for students being temporarily restricted include student fights, skipping program activities to engage in inappropriate behavior with other students, confrontation with staff
  • Activity pre/post tests and surveys: Staff members of all activities had students complete either pre/post tests or surveys (appropriateness determined by activity type) at the end of the Fall and Spring semesters. Activity sponsors individually analyze data and share the data with the Site Director.
  • Informal feedback/observations/assessments: We should be offering more activities to Special Education and English Language Learning students.
Improve

Areas to improve include:

  • Multiple club sponsors are reporting are frustration with students arriving tardy to the club or activity. See corresponding SMART goal for the full plan for improvement. Plan of action includes implementing bells afterschool, emphasizing the importance of not being tardy afterschool with students, activity leaders implementing student tardy log to track students that are repeatedly tardy.
  • A common theme in parent feedback is suggesting activities we already have available. See the corresponding SMART goal for the full plan for improvement. Plan of action includes increasing communications to parents so they are better informed of what is happening in the program.
  • After analyzing the attendance of various populations against specific program types and also Site Director’s own general observation, participation among Special Education students and English Language Learners is down overall compared to the general population of students. See the corresponding SMART goal for the full plan for improvement. Plan of action includes offering activities designed and geared specifically towards these populations and recruitment of these students by working with counseling and department chair heads.

Templates & Samples

 

SMART Goal for AS Template_FCPSASP