Out-of-school time includes “both traditional programs operating during afternoon hours and more comprehensive efforts that respond to the needs of children, youth and parents during evenings, weekends, summers and holidays by offering activities [and services] that help youth grow, learn, and develop” (American Youth Policy Forum, 2006).

The Youth Program Quality Intervention and Assessment (YPQ) framework describes high-quality out-of-school time programs as those that ensure students are physically and emotionally safe; build supportive environments through positive relationships; provide opportunities for youth to learn and engage with new content, build skills and develop; and empower youth to be leaders in their own development (Ramaswamy et al., 2013).

Adelante Mujeres—Chicas Program

A growing body of research demonstrates that high-quality out-of-school time programming benefits students both academically and in developing social and emotional skills such as creativity, self-identity and problem-solving, which are known to contribute to the likelihood of success in school and in life (e.g., Akiva et al., 2013; Durlak et al., 2010; Kidron & Lindsay, 2014; McCombs et al., 2017; Naftzger et al., 2014; Pierce et al., 2010; Vandell, 2013).

In alignment with this research, the Initiative has supported over 50 community organizations and schools since 2013, strengthening out-of-school time programming for over 15,000 middle school students across Oregon. Participating programs receive three or more years of funding, which typically ranges from $25,000 to $60,000 annually depending on program size and model.

Participating programs engage in a robust learning community facilitated by the Institute for Youth Success at Education Northwest. The learning community supports program quality improvement using an Oregon-specific adaptation of the nationally recognized, research-based Youth Program Quality Intervention and Assessment. It also provides a safe and supportive space for practitioners to share ideas, work through challenges and develop as professionals.

The Initiative centers the experiences and perspectives of program staff and students, and positions foundation staff as learners alongside program leaders and staff. As a result, insights from the learning community shape the Initiative to better meet program needs and to strengthen the out-of-school time field in Oregon.

Bend Science Station

Overview of evaluation findings

The Initiative evaluation was designed to assess the Initiative’s effectiveness, support internal learning and adaptation, identify and document the impacts of the Initiative and participating programs, and share what we learn with program leaders, funders and other stakeholders.

Through the Initiative, we have learned a lot about the impact of out-of-school time on students in Oregon, how to think about and measure the impacts of these programs, and how to support intensive efforts to bolster program quality.

Our evaluation demonstrates that:

The Initiative is increasing access to high-quality out-of-school time programs for middle school youth most likely to experience the opportunity gap: students of color, students in under-resourced rural communities, and students from low-income families.

Students benefit from out-of-school time program participation in interrelated ways that are fundamental to their success as students and beyond, including:

• Academically, through greater school attendance and engagement as well as by developing academic mindsets like future orientation.

• Through a sense of belonging, which is particularly important for middle schoolers who are forging their identities in response to learning opportunities and social interactions.

• By developing a wide range of social and emotional learning skills, particularly confidence.

The Initiative is boosting out-of-school time program quality. Participating programs from across Oregon are improving by using the YPQ within a broader, supportive learning community.

Many types of out-of-school time programs, approaches and contexts can be effective. While we saw positive patterns for certain types of programs in our analysis (those focused on social and emotional learning, for example), all programs can build quality. Further, a diverse set of programs is important for meeting student and family needs statewide.

The Initiative’s learning community provides valuable networking, peer learning and professional development opportunities not easily found elsewhere. High-quality out-of-school time programs rely on skilled, well-supported staff. Participating staff benefit from the safe and supportive environment provided within the learning community.

The Initiative is cultivating authentic learning and improvement through a combination of high expectations for engagement in program quality improvement, low-stakes use of assessment results, and intentional trust- and relationship-building.

The Initiative is contributing to the development of the out-of-school time field in Oregon by bringing together a large, diverse group of programs, providing leadership on program quality improvement and coordinating with other funders and networks.

Continued support for out-of-school time programs and for program engagement in quality improvement is needed. There are limited funding opportunities for out-of-school time programs in Oregon, especially funding that is stable and long-term, and/or coupled with supports like the Initiative’s learning community.

We need to reframe the impact and measurement of out-of-school time programs to focus on more proximal, useful measures of student progress and the conditions that support student success.

Camp Fire Central Oregon

Many of our findings align with existing research on the impacts of out-of-school time for students and their families. Our contributions to the research base include providing Oregon-specific findings, and findings about the value of culturally specific programs in particular. Our emphasis on program quality and social and emotional learning may also benefit others as the field moves increasingly toward a focus on more proximal measures of student progress and the conditions that support students.

We also hope that other funders and networks can use the information we share about the importance of program quality improvement for program staff, how we supported program quality improvement within the learning community, and the related lessons we learned. Finally, we hope that by elevating and highlighting the perspectives of students, families and staff, we are demonstrating the value and validity of this data.

Ultimately, what we have learned through the Initiative gives us confidence that out-of-school time programs are an effective means of addressing the opportunity gap. These lessons also continue to inform the Initiative’s ongoing development and investment in out-of-school time program quality improvement.

DOWNLOAD A BRIEF, PRINTER-FRIENDLY PDF OF THE EVALUATION HIGHLIGHTS.

The Initiative is ongoing

As of early 2021, more than $11 million has been invested in supporting out-of-school time programs through the Initiative. In the 2020–2021 school year, 41 programs are actively participating in the Initiative, many of which have at least one more year of eligibility.

The Initiative will continue to fund existing high-quality out-of-school time programs with a focus on supporting program quality improvement through a learning community. The Initiative team (Oregon Community Foundation, The Ford Family Foundation and Institute for Youth Success staff) will keep working to strengthen Oregon’s out-of-school time field by coordinating and collaborating with other funders and networks and by continuing to support the Institute for Youth Success as a valuable resource for programs regardless of their participation in the Initiative.  

The Initiative team will also continue to prioritize learning and refine the Initiative. The team is exploring a number of ways to further incorporate what we’ve learned thus far into Initiative plans, including a possible expansion of the Initiative to include more programs as well as deeper or more sustained program quality improvement work for those ready to do so.

Adapting to the COVID-19 pandemic

OSU Extension Service Garden Club

When the COVID-19 pandemic began to impact Oregon schools in March 2020, it also affected out-of-school time programs, many of which had to close at least temporarily or make major operational changes (e.g., by providing emergency services to families or moving to virtual programming). The Initiative adapted by releasing programs from improvement planning and reporting requirements for the year and by adjusting learning community activities to ensure relevance given the pandemic, while also making participation optional rather than mandatory.

The Initiative is further supporting the Institute for Youth Success in offering a series of virtual conversations and learning opportunities to help out-of-school time programs and other youth development organizations adapt and plan during the pandemic. These opportunities will continue through at least spring 2021 and are open to anyone, regardless of their participation in the Initiative.

As schools continue to reopen in 2021, we expect that out-of-school time programs will begin to shift back to providing in-person programming. As we learn more about what this will look like, the Initiative team will continue to adapt, seeking ways to reincorporate accountability and support for program quality improvement and learning community participation while also meeting evolving program needs.

NEXT: About the Out-of-School Time Initiative →