What we're asking in education
What are the benefits of after-school programming for children?
Effective after-school programming can bring a wide range of benefits to youth, families, and communities. Programs have the potential to develop pro-social skills, improve academic performance, reduce risky behaviors, promote health and wellness, and provide a safe, structured environment for the children of working parents. Understanding conditions that are necessary to improve outcomes for children through after-school programming is critical in effectively filling service gaps for children and families in need. For over fifteen years, PRG has conducted outcomes research and annual evaluations of an after-school program in Baton Rouge, Louisiana.
Results from a First-Year Evaluation of Academic Impact of an After-School Program for At-Risk Students
This paper, published in the Journal of Education for Students Placed at Risk in 2007, presents the research findings of an evaluation of the academic impacts of 21st Century Learning Centers (CCLC) in Louisiana. Using quasi-experimental design, the article operationalizes academic achievement as core and subject test performance on nationally standardized pre- and posttests (Iowa Test of Basic Skills; ITBS). Based on previous research and evaluation requirements, the article (a) employs outcomes of interest to policymakers (standardized test scores); (b) uses program attendance as a key independent variable; (c) uses efficient methods to control for extraneous impact on the dependent variable; and (d) focuses the evaluation on a specific group of students—at-risk children in Louisiana. Findings indicate that the 21st CCLC program is having a positive academic impact on participants who attend the program for 30 days or more. Further, impacts are shared across specific grantee programs, specific subjects, and subgroupings of students. Finally, the study finds that intensity of attendance is positively related to academic impact.
Jenner E., Jenner L., Results from a First-Year Evaluation of Academic Impact of an After-School Program for At-Risk Students, Journal of Education for Students Placed at Risk (2007), doi: 10.1080/10824660701261144
Student Perspectives on the Influences of Participation in an Afterschool Program
At the end of each academic year, PRG research analysts conduct focus groups with roughly thirty elementary and middle school children to explore how an afterschool program is influencing school performance, attitudes toward school, and social-emotional capacities. Focus groups consistently suggest that the afterschool program positively impacts academic skills, by providing dedicated time for students to complete homework in a structured environment and time to review class lessons outside of school, often with the support of peers and instructors. Each year, conversations touch on several themes concerning participants’ understanding of social-emotional skills such as cooperation, empathy, and managing conflict. Although participants are not always able to clearly define concepts related to these social skills, they are often able to provide examples of why these skills are important or instances in which they have used these skills during afterschool programming.
How can innovative, problem-based learning improve improve STEM engagement and achievement in Elementary and Middle School?
Workers in science, technology, engineering, and math (STEM) occupations have higher employment rates and higher median earnings than those non-STEM jobs. Elementary, middle, and secondary education lay the foundation for postsecondary STEM-related majors and occupations. Persistent academic engagement is a well-established predictor of academic achievement. While research has documented the decline in STEM interest among students from elementary to high school, little evidence exists to demonstrate why student perceptions of STEM curricula and careers change during this time. Opportunities to interact with problem-based, immersive, or simulation-based learning in elementary and middle school may motivate students to engage with and pursue opportunities in STEM subjects. Employing approaches that range from large-scale cluster-level RCTs to in-depth qualitative interviews with students and teachers, PRG’s research examines the potential impact of innovative, classroom-based science learning experiences on students’ engagement with science, math, engineering, and data science. PRG’s work has also explored the classroom conditions under which innovative learning experiences are most effective and why.
Data Adventures. PRG is conducting a five-year evaluation of Data Adventures, a classroom-delivered data science and digital storytelling learning initiative designed by WestEd and Santa Clara County Office of Education (SCCOE). PRG is conducting an iterative design-based research formative evaluation as well as a cluster randomized controlled trial (RCT). This evaluation will assess the program’s readiness for rigorous evaluation (Phase I) followed by its impact on data literacy outcomes and STEM engagement of students (Phase II) across 10-20 middle schools in Santa Clara, California.
ENpowered. PRG conducted a five-year evaluation of Project SYNCERE’s ENpowered engineering curriculum to determine its impact on STEM achievement and engagement in middle school students. ENpowered is a 10-week classroom-based engineering design curriculum that incorporates a cross-school design competition that aims to build a passion for STEM learning among underrepresented youth in Chicago. PRG’s impact evaluation of ENpowered was a QED that aimed to draw causal inferences about the effects (impact) of ENpowered on middle school students’ achievement in mathematics and science. PRG also conducted an implementation evaluation to understand how ENpowered works in practice, interpret its efficacy, provide feedback for program improvement, and identify features necessary for sustainability and replication. This program was implemented in 22 schools in Chicago, Illinois.
EngiLearn. PRG collected data from over 2,500 students across 30 schools for this Investing in Innovation (i3)-funded RCT. Findings from the study did not indicate that the simulation-based science learning experience improved engagement and self-efficacy in science, however, qualitative analysis from teacher interviews provided useful feedback for intervention improvements to better align with classroom instruction that would be enjoyable for a diverse population of students.
Aquatic Investigators 3.0. The findings from the i3-funded research provided a foundation for a subsequent NSF-funded pilot study that examined the potential effectiveness of a series of STEM simulation-based learning experiences on students’ engagement with science, self-efficacy in science, and interest in pursuing a STEM career. In addition to assessing the intervention’s effect on these outcomes, PRG’s evaluation examined the relationship between these social and emotional learning outcomes and effective methods for measuring them.
Evaluation of the ENpowered Program
This final evaluation report summarizes results from an impact study of a 10-week classroom-based engineering design program designed to STEM engagement and achievement in middle school. The study employed a quasi-experimental difference-in-difference approach and included of 1,016 students across 19 middle schools in Chicago. The impact evaluation examined the ENpowered program’s impact on science and math grades. The implementation evaluation aimed to explore the extent to which the ENpowered model was implemented as intended at each study site and gathered formative feedback from participating teachers and students to provide continuous feedback for improvement. The confirmatory analyses indicate that participation in ENpowered did not have a discernable effect on middle school science or math grades. The implementation study found that teachers and students reported a high level of satisfaction and enjoyment with the program and that the program provides valuable exposure to the engineering field.
Lass, K., Walsh, S., Burgess, K., Jenner, E. 2025. Evaluation of ENpowered: Final Evaluation Report. Report submitted for publication on the Education Resources Information Center.
Teachers’ Perspectives on Improving Simulation-Based Science Learning for the Classroom
EngiLearn, a computer-simulation based ocean science curriculum and professional development program, was piloted in fifth-grade classrooms in Virginia. Through feedback questionnaires administered after the pilot, science teachers provided insight into the feasibility of the program for classroom-based teaching and the conditions under which it could be improved. Findings indicate that students were highly motivated by the new technology and readily engaged with the more innovative and collaborative aspects of the curriculum. Reading and writing components of the intervention, however, were not dynamic enough to engage students with varied literacy levels.
Can mentoring programs aimed at developing social and emotional skills improve educational outcomes?
Literature suggests that there is a window of opportunity to prevent the consequences of disengagement for students who may be on the path towards school dropout, by developing targeted interventions that focus on the transitions into middle and high school. PRG is conducting several studies to examine the efficacy of school transition and mentoring programs that aim to improve academic outcomes, social and emotional skills, and school connectedness.
Achievement Mentoring (AM). PRG investigated the efficacy of AM, a one-to-one adult mentoring intervention for 10th and 11th grade students identified as high-risk for dropping out of school due to high absenteeism, course failure, or disciplinary infractions. The project served 393 students in 18 high schools in communities in Delaware, Maryland, New Jersey, Pennsylvania, and rural North Carolina. PRG conducted an experimental evaluation to measure impacts on academic and disciplinary outcomes, as well as social and emotional learning skills. The implementation evaluation provided annual formative data on the extent to which the mentoring program was implemented as intended at each study site and qualitative feedback from mentors and mentees regarding the perceived value of the program.
Peer Group Connection – High School (PGC-HS). PRG conducted a multi-site individual-level RCT with 9th-grade students in rural North Carolina schools. PGC is a high school transition and cross-age peer mentoring program that aims to improve academic outcomes, social and emotional skills, and school connectedness. The sample consisted of 1,351 9th-grade students from six high schools.
Peer Group Connection – Middle School (PGC-MS). PRG conducted a multi-site individual-level RCT with 6th-grade students in New Jersey, Maryland and rural North Carolina middle schools. PGC-MS is a middle school transition and cross-age peer mentoring program that aims to improve academic outcomes, social and emotional skills, and school connectedness. The sample consisted of 1,902 6th-grade students from nine middle schools.
Evaluation of Achievement Mentoring
This final evaluation report summarizes the results from an impact study of an individualized adult mentoring program designed to prevent school dropout for students identified as being at risk for dropout after ninth grade. The study employed a randomized controlled trial (RCT) of 393 tenth-grade students. The impact evaluation examined the AM program’s impact on educational outcomes, as well as social and emotional outcomes. The implementation evaluation aimed to explore the extent to which the AM model was implemented as intended at each study site and the amount of programming students assigned to the treatment condition received. The confirmatory intent-to-treat (ITT) analyses indicate that the offer to participate in AM in 10th grade had a statistically significant impact on students’ self-efficacy, perceived adult support at school, and decision-making skills, as well as likelihood of being suspended in 10th grade. Analyses on the subsample of participants that were enrolled through 11th grade indicate that the program had a significant effect on students’ perceptions of adult support, academic self-image, and decision-making skills at the end of 11th grade. The implementation study found that most students assigned to the mentoring program received about half the recommended dosage of mentoring.
Lass, K., Walsh, S., Burgess, K., Levin, J., Jenner, E. 2025. Evaluation of Achievement Mentoring: Final Evaluation Report. Report submitted for publication on the Education Resources Information Center.
Effects of Cross-Age Peer Mentoring Program Within A Randomized Controlled Trial
This paper summarizes results from an impact study of a cross-age peer mentor program designed to prevent school dropout during the transition from middle to high school. The study employed a randomized controlled trial (RCT) of 1,351 ninth-grade students. Although the intent-to-treat (ITT) analyses indicate modest, yet potentially meaningful program impact on ninth-grade outcomes of disciplinary action, school attachment, and expectations of degree attainment across varying dosage levels, complier average causal effect (CACE) estimates suggest that a threshold level of program participation broadens the program’s impact on additional academic achievement and social and emotional learning outcomes. Given the adverse effects of the transition to high school, this promising evidence indicates that the cross-age peer mentoring intervention could be a cost-effective and sustainable strategy for high schools to implement. In presenting the study findings, we outline two methods for estimating the CACE, or the effect of program participation, that allow researchers to leverage RCT data, beyond ITT analyses, to provide meaningful context for how and when interventions work.
Jenner, E., Lass, K., Walsh, S., Demby, H., Leger, R., Falk, G. Effects of Cross-Age Peer Mentoring Program Within A Randomized Controlled Trial. Manuscript submitted for publication.
Evaluation of Peer Group Connection – Middle School
This final evaluation report summarizes results from an impact study of a cross-age peer mentor program designed to prevent school dropout during the transition from elementary to middle school. The study employed a randomized controlled trial (RCT) of 1,902 sixth-grade students. The impact evaluation examined the PGC-MS program’s impact on educational outcomes, including attendance in school and number of courses passed in sixth grade. The implementation evaluation aimed to explore the extent to which the PGC-MS model was implemented as intended at each study site and the amount of programming students assigned to the treatment condition received. The confirmatory intent-to-treat (ITT) analyses indicate that the offer to participate in PGC-MS did not have a discernable effect on student attendance or the number of classes passed during sixth grade. The implementation study found that most schools experienced barriers to implementing the program to fidelity with their sixth graders, who in turn did not get the recommended minimum amount of programming. The schools that were unable to offer the program to fidelity were severely impacted by the COVID-19 pandemic.
Lass, K., Walsh, S., Demby, H., Leger, R., Burgess, K., Jenner, E. Evaluation of Peer Group Connection – Middle School: Final Evaluation Report. Report submitted for publication on the Education Resources Information Center.
Impact on Antecedents of Student Dropout in a Cross-Age Peer Mentoring Program
PRG presented at the SREE (Society for Research on Educational Effectiveness) spring 2020 conference as part of a symposium entitled Unpacking the Logic Model: A Discussion of Mediators and Antecedents of Educational Outcomes from the Investing in Innovation (i3) program.
PRG researchers presented findings that suggest that a cross-age peer mentoring program, Peer Group Connection – High School (PGC-HS), has the potential to mitigate negative outcomes associated with the transition into high school by increasing students’ school engagement and expectations for the future during their ninth-grade year and, when implemented with high fidelity, reducing the likelihood students receive disciplinary referrals and suspensions. Results also suggest the program may not being working as hypothesized with regards to other targeted behavioral and attitudinal antecedents. From a practical standpoint, research such as this is of import to both program developers and educators. For the PGC-HS program developers, these findings on the mechanisms of behavior or behavior change targeted by the program provide validation about how the program is working and whether it is achieving its goals. In areas where the program does not seem to be “moving the needle,” developers plan to utilize the above information in conjunction with results gleaned from the implementation study to revisit the curriculum to better understand program theory and make adaptations to the program that will improve overall performance. Likewise, educators can use information gathered from this type of exploratory research to target their finite resources toward programs that solve a particular area of concern for their school and their students’ needs. They can utilize results to justify implementing, sustaining, and/or expanding a particular program; they can also make decisions about whether they should implement complementary intervention(s) to address unmet needs.
Funded through an Investing in Innovation (i3) development grant, The Peer Connection Study investigated the impact of PGC-HS on student outcomes related to dropout prevention. The study was an RCT with a sample of 1,533 ninth-grade students in rural North Carolina.
