Participating Out-of-School Time Programs & Their Students

About participating programs

Out-of-school time programs are selected for participation in the Initiative through a competitive review process that assesses their readiness for and interest in improving program quality, as well as their existing strengths in 1) serving middle school students of color, students from under-resourced rural communities, and students from low-income families; 2) family engagement; 3) positive adult role models; and 4) purposeful academic support. These are all important research-based elements of effective out-of-school time programming. 

The Initiative supports programs regardless of their content focus, type, model or context, resulting in an intentionally diverse set of over 50 participating programs to date. Participating programs include those focused on general youth development; gender-specific programs; science, technology, engineering and math (STEM); college and career readiness; and those that incorporate subject-specific enrichment or assist students with homework. Some programs incorporate more than one of these components. Some also provide students and families with additional services such as housing and wrap-around supports.

Arts in Education of the Gorge

Participating program design also varies. Some programs were developed locally—like the science, technology, engineering, art and math (STEAM) programming facilitated by Arts in Education of the Gorge—while others are part of national programming networks like Boys & Girls Clubs. Some are culturally specific (i.e., developed and led by and for specific cultural communities, often centered on race/ethnicity), such as the Adelante Mujeres Chicas program which is focused on the youth development of Latina girls. Others are culturally responsive—designed to incorporate practices that are tailored to the strengths and needs of the communities being served, such as the Human Solutions LearnLinks program that supports youth from immigrant and refugee families within housing development community centers.

All participating programs have group-based designs rather than drop-in or one-on-one designs, though some also incorporate individualized supports like mentoring. Some run multiple types of programs or program offerings per year, some run the same programming in multiple locations, and some deliver one type of program in one place.

Programs also vary in their context. Some are in urban settings and focus on engaging students in specific schools or neighborhoods, such as Elevate Oregon, which serves students in the Parkrose neighborhood in Northeast Portland. Others are in rural, remote or frontier communities, like Kids Club of Harney County in Burns and RiverStars Performing Arts, sponsored by the Illinois Valley Community Development Organization in Cave Junction.

Building Healthy Families/Wallowa Resources

Some programs are implemented by community-based organizations like Human Solutions and Building Healthy Families, others by higher education organizations like Oregon State University’s Science & Math Investigative Learning Experiences (SMILE), and still others by K-12 schools or school districts like Port Orford-Langlois School District. A few programs have also been 21st Century Community Learning Centers (e.g., Eugene School District 4J) or Multnomah County SUN Community Schools partners (e.g., Impact NW), making them a part of other out-of-school time networks in Oregon. Two tribal organizations also participated: the Confederated Tribes of Warm Springs and the Confederated Tribes of the Umatilla Indian Reservation.

For the purposes of our educational outcomes analysis, we grouped the 21 programs that provided student data into one or more of the following categories: 

• Focused on social and emotional learning: 12 programs integrate social and emotional learning curricula or work intensively on one or more aspects of social and emotional learning (e.g., growth mindset or cultural identity).

• Academically intensive: Six programs feature more intensive subject-specific enrichment—including STEM/STEAM programs—or more intensive connections to in-school learning.

• Culturally specific: These six programs are developed and led by and for a specific population, often centered on race/ethnicity.

Some programs fit into two or even all three of these groups, underscoring the multifaceted nature of out-of-school time programming. We also explored outcomes for other program characteristics, but found that these groupings yielded the most promising and straightforward results.

See the results of our analysis in Initiative & PROGRAM impacts.

Review OUR methodological details in About the Evaluation.

About the students they serve

Middle school is a transitional time from childhood to adulthood when adolescents experience more independence, seek out new environments and social settings, and begin to define their own interests, goals and values. Adolescent brains are especially adaptive and ripe for social, emotional and academic learning.

Although adolescence is often thought of as a time of turmoil and risk for young people, it is more accurately viewed as a developmental period rich with opportunity for youth to learn and grow. Research reveals that throughout adolescence, the connections between brain regions become stronger and more efficient, and unused connections are pruned away. This means that adolescents’ brains can adapt, and they become more specialized in response to demands placed on them by their environments—by learning opportunities at school, for example, and by social interactions with their peers and families (National Academies of Sciences, Engineering, & Medicine, 2019).

The evaluation team collected three sets of data that help us describe the students served by participating programs:

• Data about students, their participation and educational outcomes.
• Student photovoice projects.
• Surveys on student social and emotional learning.

All demographic and survey data are about students who participated in out-of-school time programming during the 2014–2015, 2015–2016 and 2016–2017 school years.

Camp Fire Columbia

Program participation & educational outcomes data

The 21 programs funded in the first few years of the Initiative collected and shared data for about 3,600 students, most of whom (82%) participated in one year of programming. Students spent an average of 57 hours in programming in any given year, but this varied widely depending on the student and type of program (from less than 1 hour to over 400 hours).

Slightly more than half of these students were girls, 90% were eligible for free or reduced-price meals, and 19% had a special education designation. More than 43% of the students were Latino/x, approximately 25% were American Indian/Alaska Native, about 10% were Black/African American, and more than 41% had a language of origin other than English (based on race/ethnicity categories that are not mutually exclusive). More than 57% of students attended Initiative programming in a rural part of Oregon.

We had enough identifying information to pair 3,100 of these students (86%) with educational records from the Oregon Department of Education, allowing us to follow their progress on educational measures available statewide (e.g., attendance).

Boys & Girls Club of Portland

Finally, we acknowledge that students in Oregon persistently face educational disparities based on race, economic status and other demographic characteristics (Oregon Department of Education, 2020). When we disaggregated educational outcomes data for participating students, we found some patterns consistent with statewide data. For example, Initiative students learning English, students eligible for free or reduced-price meals, and Black/African American students were disadvantaged in terms of meeting grade-level standards in math and English language arts compared to their Initiative peers. We also found patterns that were not consistent with statewide data and/or not consistent across cohorts. We included analyses in the impact section to share findings related to meaningful patterns for particular groups of students as feasible.  

Review findings from our analysis of educational outcomes data in Initiative & PROGRAM impacts.  

Student photovoice

Photovoice is a youth participatory evaluation method that engages students in personal sharing through photographs and the written word. It is especially useful in situations where there is an uneven power dynamic, such as in adult-led programs for youth (Strack et al., 2004). In 2015–2016, students in eight participating programs completed photovoice projects describing how they see themselves and how others view them in school, in out-of-school time programming and in their communities. Photovoice allows students freedom of expression through words and images, while helping them describe their interests and identities from a strengths-based perspective. The finished projects from 2015–2016 highlighted youth commitment to school, peers as an important support network, the ways youth are perceived internally versus externally, and hope for the future. The evaluation team used these direct statements from students to complement quantitative data and analysis.

Student surveys on social & emotional learning

Students in participating programs were also surveyed in spring 2015, 2016 and 2017 about their social and emotional learning, using a tool adapted from the Engagement, Motivations and Beliefs Survey created by Youth Development Executives of King County. The survey asked students to reflect on themselves as individuals as well as on how much their programs supported them through academics, belonging and engagement. It included sections on academic identity, mindsets, interpersonal skills and cultural identity, and asked whether statements relating to social and emotional learning were not at all true, somewhat true, mostly true or completely true.

More than 1,300 students completed the survey at least once. Respondents were demographically similar to the students in the full Initiative dataset. The only group that was under-represented in terms of race/ethnicity was American Indian/Alaska Native (4% of survey respondents, compared to 25% in the full dataset). Most respondents were students of color (69%) and/or eligible for free or reduced-price meal programs (85%).

In our analysis, the mostly true and completely true categories were combined to show agreement. Selected survey results provide a glimpse of student perspectives and skills. 

Participating students report a strong sense of academic identity and social and emotional capabilities to aid them in their path to adulthood.

Source   Survey data collected through the Initiative; questions adapted from the Youth Development Executives of King County’s Youth Engagement, Motivation and Beliefs Survey.

Through survey responses, we also see that students have high academic aspirations and value learning. Just under 90% mostly or completely agreed that getting a college education is important to me and getting good grades is one of my main goals. At a developmental phase when adolescents are defining themselves and exploring new interests and skills, it’s encouraging to see that 90% also agreed that I can do a good job if I try hard enough.

NEXT: Improving program quality →