Assessment of the Impact of Participation in Out-of-School Time Programs
on Detroit Students’ Academic Performance

Introduction

In May of 2006, Michigan State University conducted an analysis of patterns of student participation in after-school programs and examined the relationship between program participation and academic performance. From June 2004 through August 2005, data was collected from 5,481 students participating in 56 21st Century Community Learning Center (CCLC) school sites and 7,707 youth in 50 Mayor's Time program sites. In total, the database contains information on 12,559 Detroit youth. This information was then integrated with data from Detroit Public Schools (DPS) to see what impacts, if any, were related to after-school participation. Some of the site characteristics of these locations included:

• Site type: Mayor’s Time sites were distributed across community-based organizations, public schools, and faith-based organizations; in contrast, 21st CCLC were nearly all public schools.
  
• Grades served: The majority of Mayor’s Time sites served elementary school students, while the majority of 21st CCLC sites served middle school students.
  
• Gender: In general, sites tended to serve more girls than boys; 10 sites served only girls.
  
• Race: At most sites (about 85%), the majority of students were African-American.
  
• Dosage: Mayor's Time sites tended to have concentrations of students receiving a lower dosage than did 21st CCLC sites.

Some Key Findings

• Different out-of-school time organization types serve different groups of students and have different programmatic targets. Students participating in Mayor's Time programs tend to be somewhat different compared to students participating in 21st CCLC programs. Overall, Mayor's Time sites served a greater proportion of boys and elementary school students.
  
• Students tend to participate for less than 30 days per year. Almost half of students attended for 10-29 days, and only 24% attended for at least 30 days. This suggests that recruitment and retention strategies may need improvement and should be examined to increase the number of days that children attend programs.
  
• Students who attended out-of-school time programs for more days tended generally to be doing better academically. Students who attended 10-59 days were those with higher GPAs and higher TerraNova reading and math scores compared to students who attended 1-9 days. Students who attended 60 days or more tended to not differ significantly from the low-attending students except on TerraNova math.
  
• Students who attended out-of-school programs for more days showed greater improvement in TerraNova math scores over time compared to students who attended few days. In addition, the results for GPA and TerraNova reading scores showed a similar pattern, although these were not statistically significant.

Recommendations

1. Continue to build the capacity of partner sites to collect data. Mayor's Time has made significant progress in assisting primarily community-based out-of-school time programs to collect detailed service utilization data. These sites tend to have less experience with systematic evaluation and documentation, and participation in the data-tracking efforts can assist with evaluation that promotes program improvement and sustainability. The data sharing partnership between Mayor’s Time and Detroit Public Schools lays the foundation for the assessment of impact of program participation on academic achievement.
   
2. Improve data quality. Certain demographic characteristics, such as student gender and grade, as associated with outcome variables, such as GPA and TerraNova scores as well as with attendance. For this reason, it is important that data on demographic characteristics be collected on every student in order to take their effects into account during the analytic process.
   
3. Maintain the partnership with DPS. The partnership with DPS in which they provide school outcomes data is a vital part of the Mayor's Time evaluation process.