Measuring Soft Skills in OST Programs: An Interview with Nicole Yohalem, Part 2

By Sam Piha

The Learning in Afterschool project is promoting five learning principles that are well rooted in education and youth development research. Teachers and youth workers alike know that these principles are important to engaging young people in learning. Although these principles are vital to developing important skills, sometimes referred to as 21^st century skills, many refer to them as “soft”. Further, the lack of accessible tools that measure these skills has been a problem for the OST field. 

Recently the Forum for Youth Investment published From Soft Skills to Hard Data: Measuring Youth Program Outcomes, which offers a survey of measurement instruments that measure many of these skills. They include:

Nicole Yohalem

•      Communication
•      Relationships & Collaboration
•      Critical Thinking & Decision-making
•      Initiative & Self-direction

Below, we interview Nicole Yohalem (Forum for Youth Investment) one of the co-authors of this report.

Q: What criteria did you use in selecting instruments?

A: We considered several factors. First we looked for measures where a majority of the contents mapped directly onto one of our four areas of interest. We looked for measures that were appropriate for use in a range of settings, including OST programs, and focused on tools that can be used with upper elementary through high school age youth, since a lot of useful work has already been done by CASEL to review measures for use with younger children. We also prioritized measures that are accessible to practitioners and relatively easy to use. Because we are committed to ensuring practitioners have access to tools that yield reliable (consistent) and valid information, we also looked for instruments that at a minimum, had been investigated for scale reliability, factor structure and sensitivity to OST program impact. 

Q: How might the guide be helpful for OST programs?

A: In selecting measures there are some important things for program leaders to consider. First and foremost, outcome measures should reflect the goals and activities of the program. Programs should measure outcomes that they value and that they are actively trying to influence. Second, it is important to select measures that will yield reliable and valid information. Finally, there are all the practical issues to consider – cost, ease of administration and accessibility. 

The guide includes information about all of these considerations. For each instrument we summarize the origins and focus on the tool, include sample items, and discuss user and technical considerations. Where possible we also include information about length, cost, format, supplemental tools, and training. Our technical reviews focus on the extent to which reliability and validity have been established. 

Q: Finally, do you see this new resource helping to address any important risks or opportunities facing the OST/afterschool movement at this time? 

A: Unfortunately we haven’t done a good job of coming to consensus on what to call important skills like critical thinking and decision-making, relationships and collaboration, communication and initiative and self-direction. I hear these referred to as social-emotional skills, soft skills, 21^st century skills, new basic skills, higher-order thinking, non-academic outcomes…the list goes on. 

If we could get more consistent about naming these and measuring them, programs will be more likely to identify them as target outcomes and demonstrate their ability to move the dial on these skills. At the policy level, we have historically under-invested in programs that are good at developing these skills. With the education and business sectors increasingly recognizing their value to school and workplace success, we have a unique window of opportunity to demonstrate the important role that afterschool programs play in supporting learning and development. 

________________________________________________________________

Nicole Yohalem

Director of Special Projects, The Forum for Youth Investment

Nicole oversees Forum projects on out-of-school time, postsecondary success and bridging research, policy and practice; speaks on behalf of the Forum at national conferences and events; and serves as an advisor to several foundations, organizations and initiatives connected to the Forum. She has authored numerous reports, articles and commentaries, and oversees several regular Forum publications, such as the Ready by 21, Credentialed by 26 issue brief series.