Measuring Soft Skills in OST Programs: An Interview with Nicole Yohalem, Part 2

By Sam Piha

The Learning in Afterschool project is promoting five learning principles that are well rooted in education and youth development research. Teachers and youth workers alike know that these principles are important to engaging young people in learning. Although these principles are vital to developing important skills, sometimes referred to as 21^st century skills, many refer to them as “soft”. Further, the lack of accessible tools that measure these skills has been a problem for the OST field. 

Recently the Forum for Youth Investment published From Soft Skills to Hard Data: Measuring Youth Program Outcomes, which offers a survey of measurement instruments that measure many of these skills. They include:

Nicole Yohalem

•      Communication
•      Relationships & Collaboration
•      Critical Thinking & Decision-making
•      Initiative & Self-direction

Below, we interview Nicole Yohalem (Forum for Youth Investment) one of the co-authors of this report.

Q: What criteria did you use in selecting instruments?

A: We considered several factors. First we looked for measures where a majority of the contents mapped directly onto one of our four areas of interest. We looked for measures that were appropriate for use in a range of settings, including OST programs, and focused on tools that can be used with upper elementary through high school age youth, since a lot of useful work has already been done by CASEL to review measures for use with younger children. We also prioritized measures that are accessible to practitioners and relatively easy to use. Because we are committed to ensuring practitioners have access to tools that yield reliable (consistent) and valid information, we also looked for instruments that at a minimum, had been investigated for scale reliability, factor structure and sensitivity to OST program impact. 

Q: How might the guide be helpful for OST programs?

A: In selecting measures there are some important things for program leaders to consider. First and foremost, outcome measures should reflect the goals and activities of the program. Programs should measure outcomes that they value and that they are actively trying to influence. Second, it is important to select measures that will yield reliable and valid information. Finally, there are all the practical issues to consider – cost, ease of administration and accessibility. 

The guide includes information about all of these considerations. For each instrument we summarize the origins and focus on the tool, include sample items, and discuss user and technical considerations. Where possible we also include information about length, cost, format, supplemental tools, and training. Our technical reviews focus on the extent to which reliability and validity have been established. 

Q: Finally, do you see this new resource helping to address any important risks or opportunities facing the OST/afterschool movement at this time? 

A: Unfortunately we haven’t done a good job of coming to consensus on what to call important skills like critical thinking and decision-making, relationships and collaboration, communication and initiative and self-direction. I hear these referred to as social-emotional skills, soft skills, 21^st century skills, new basic skills, higher-order thinking, non-academic outcomes…the list goes on. 

If we could get more consistent about naming these and measuring them, programs will be more likely to identify them as target outcomes and demonstrate their ability to move the dial on these skills. At the policy level, we have historically under-invested in programs that are good at developing these skills. With the education and business sectors increasingly recognizing their value to school and workplace success, we have a unique window of opportunity to demonstrate the important role that afterschool programs play in supporting learning and development. 

________________________________________________________________

Nicole Yohalem

Director of Special Projects, The Forum for Youth Investment

Nicole oversees Forum projects on out-of-school time, postsecondary success and bridging research, policy and practice; speaks on behalf of the Forum at national conferences and events; and serves as an advisor to several foundations, organizations and initiatives connected to the Forum. She has authored numerous reports, articles and commentaries, and oversees several regular Forum publications, such as the Ready by 21, Credentialed by 26 issue brief series.

Measuring Soft Skills in OST Programs: An Interview with Nicole Yohalem, Part 1

By Sam Piha

The Learning in Afterschool project is promoting five learning principles that are well rooted in education and youth development research. Teachers and youth workers alike know that these principles are important to engaging young people in learning. Although these principles are vital to developing important skills, sometimes referred to as 21^st century skills, many refer to them as “soft”. Further, the lack of accessible tools that measure these skills has been a problem for the OST field. 

Recently the Forum for Youth Investment published From Soft Skills to Hard Data: Measuring Youth Program Outcomes, which offers a survey of measurement instruments that measure many of these skills. They include:

Nicole Yohalem

•      Communication
•      Relationships & Collaboration
•      Critical Thinking & Decision-making
•      Initiative & Self-direction

Below, we interview Nicole Yohalem (Forum for Youth Investment) one of the co-authors of this report.

Q: Of the many important skills, why did you focus on the four skill areas presented in your paper?

A: We didn’t want to create a new framework because there is so much good 

existing work out there.  So in identifying these four areas to focus on, we reviewed commonly used and cited frameworks from the Collaboration for Academic, Social and Emotional Learning (CASEL); the Partnership for 21^st Century Skills; and the U.S. Department of Labor. We identified the common constructs across those frameworks, focusing specifically on skill-oriented outcomes and those that are amenable to intervention by afterschool programs. We also focused on skills that are cross-cutting, which means we left some things out that relate to specific content knowledge (e.g., technology, global awareness). That’s how we came to communication, relationships & collaboration, critical thinking & decision-making and initiative & self-direction. We aren’t suggesting this is a comprehensive list of important skills, or that these are the only skills afterschool programs should focus on. We may tackle additional areas in an updated report next year.

Q: Why are these “soft skills” deemed important?

A: There is growing evidence and recognition that these skills and dispositions are critical – to academic success, workplace success, and to overall wellbeing.  Teachers, students, parents and Fortune 500 companies all think these kinds of skills are critical. In a 2006 national survey of employers, collaboration, work ethic and communication were among the most important skills necessary to succeed in the workplace. On the academic side, focusing on social skills is linked with developing a positive connection to school, improved behavior, and increased achievement.

Q: What do you see as the role of OST programs to build these skills? In other words, why are OST programs well positioned to build these skills?

A: We feel these kinds of outcome areas could really be a strategic niche – in economic terms – a “comparative advantage” for many youth programs. Afterschool programs operate with limited resources yet have significant flexibility compared to schools. These are skills that youth programs are good at building and supporting, and they matter for learning and development.

Q: Why did you think it was important to identify instruments to measure these skills?

A: We know that high quality afterschool programs can help young people develop these and other skills, but to live up to this potential, activities need to align with outcomes and programs need tools that are accessible and that do a good job of measuring them. When you are tracking things like attendance, grades or standardized test scores, which many afterschool programs do, data are typically obtained from school records, which means program leaders and evaluators rarely face decisions about what instrument to use.

To be continued in Part 2

________________________________________________________________

Nicole Yohalem

Director of Special Projects, The Forum for Youth Investment

Nicole oversees Forum projects on out-of-school time, postsecondary success and bridging research, policy and practice; speaks on behalf of the Forum at national conferences and events; and serves as an advisor to several foundations, organizations and initiatives connected to the Forum. She has authored numerous reports, articles and commentaries, and oversees several regular Forum publications, such as the Ready by 21, Credentialed by 26 issue brief series.

How Kids Learn: LIA's First Conference

By Sam Piha

On January 27th, over 180 people from across California and the country met at the David Brower Center in Berkeley to hear from experts on How Kids Learn - the first conference sponsored by the Learning in Afterschool project. Participants included representatives from school districts, youth organizations, private funders, the State Department of Education, and afterschool advocates.

Experts shared their best thinking and recent research in 20 minute presentations to the audience. Speakers included Paul Heckman, Associate Dean of Education at UC Davis; Michael Merzenich, a leading brain researcher at UC San Francisco; Alexis Menten from the Asia Society; Shawn Ginwright, a professor and community practitioner from San Francisco State University; youth from San Francisco's Youth Empowerment Fund; Carol Tang from the Coalition for Science After School; and many others. For a complete list of presenters, see the How Kids Learn website.


Thanks to the youth and young adults of Change Agents, we will feature a videotape of each presentation on our Learning in Afterschool YouTube channel in the very near future.

Clockwise: Paul Heckman, Michael Merzenich, Alexis Menten,
Shawn Ginwright, Youth Empowerment Fund, and Carol Tang.

Afterschool for Older Youth and More: An Interview with Karen Pittman, Part 2

By Sam Piha

Karen Pittman

Karen Pittman is President and CEO of Forum for Youth Investment and is known nationally as the leading advocate for youth development. Many credit her with launching the youth development movement and being an important thought leader promoting policies and systemic approaches to supporting young people's development, including the Ready By 21 initiative. See her full bio below. 

Q: In California, all the dollars supporting afterschool learning opportunities for high school youth come from federal 21st CCLC funds (most of the funding for elementary and middle school programs   come from protected state funds); how important do you think it is that we include high school age youth in the afterschool equation?

A: It’s critically important that teenagers, and especially our most vulnerable youth, have access to supports and opportunities geared toward helping them build skills, connect with positive adults, and navigate the transition to adulthood. This is especially important given how many young people are not on track to graduate and how many more are not on track to graduate college and career-ready. California’s funneling all of its 21st CCLC dollars to high school programming has been a boost for the field as you all have had a chance to innovate programmatically and experiment with things like credit recovery.


Q: What do you see as the major risks and opportunities facing the out-of-school time (OST) movement?

A: I think the conversation that is unfolding about expanded learning opportunities represents a real opportunity for the field. The education community is acknowledging schools alone cannot ensure all young people are ready for college, work and life. The business community is acknowledging that young people need more opportunities to develop 21st century skills. Communities are building infrastructure to support broad goals, meaningful partnerships, and shared data. Now is the time for OST programs to shore up their commitment to quality and to put a stake in the ground about the kinds of outcomes they want to be held accountable for. If test scores and grades are not necessarily the best things to measure success by – what is? The development of communication, collaboration, and critical thinking skills? Creativity? Hope? We need to do a better job of naming and measuring outcomes we know are critical to youth success and that programs can influence.   

_________________________________________________________________________

Karen Pittman is a co-founder, President and CEO of the Forum for Youth Investment. She started her career at the Urban Institute, conducting numerous studies on social services for children and families. Karen later moved to the Children’s Defense Fund, launching its adolescent pregnancy prevention initiatives and helping to create its adolescent policy agenda. In 1990 she became a vice president at the Academy for Educational Development, where she founded and directed the Center for Youth Development and Policy Research and its spin-off, the National Training Institute for Community Youth Work.

In 1995 Karen joined the Clinton administration as director of the President's Crime Prevention Council, where she worked with 13 cabinet secretaries to create a coordinated prevention agenda. From there she moved to the executive team of the International Youth Foundation (IYF), charged with helping the organization strengthen its program content and develop an evaluation strategy. In 1998 she and Rick Little, head of the foundation, took a leave of absence to work with ret. Gen. Colin Powell to create America’s Promise. Upon her return, she and Irby launched the Forum, which later became an entity separate from IYF.

Extended Learning Time and More: An Interview with Karen Pittman, Part 1

, 

Karen Pittman

By Sam Piha


Karen Pittman is President and CEO of Forum for Youth Investment and is known nationally as the leading advocate for youth development. Many credit her with launching the youth development movement and being an important thought leader promoting policies and systemic approaches to supporting young people's development, including the Ready By 21 initiative. See her full bio below. 

Q: You have recently cautioned us to think more clearly about diverting funds and efforts to expand learning time by expanding the school day.  Can you briefly summarize your concerns?

A: Ensuring that all young people are ready for college, work and life requires integrated communitywide commitments to learning. While the current push to expand learning time offers some great opportunities, it could actually have the opposite effect of decreasing overall learning time in communities. This is because if public funding currently available to community-based organizations through programs like the 21st Century Community Learning Centers is redirected toward schools, some CBOs will have to reduce their programming or even shut down.  


Also, a shift toward schools alone being held accountable for expanded learning time could destabilize effective collaborations that are finally in place, actively supporting effective community-school partnerships. But the most important thing to remember is simply that more time doesn’t necessarily equal more learning. Learning opportunities must be high quality if they are going to produce more learning – whether they happen in classrooms or CBOs. 

Q: The Learning in Afterschool Project is stressing the need to increase learning engagement in OST programs by promoting 5 learning principles that should characterize how we design activities in afterschool programs.  These principles are: learning that is active, collaborative, meaningful, supports mastery and expands horizons. Can you comment on which of these you think are most important?

A: They are all critical. Your principles reflect what our own field experience and research suggest about the characteristics of effective learning environments. They speak to both staff practices and program content, which is important. I think that working toward mastery, which goes right to the intersection of program content and staff practices – is something we need to be more intentional about in out-of-school time (OST) settings. Ensuring that young people have opportunities to engage in activities that build their skills and interests over time can be difficult in settings where participation and staffing patterns aren’t always as consistent as school.


Q: Do you have any advice of how best to get these principles better integrated in our expectations of afterschool programs?

A: In our experience generating data that give staff concrete feedback about the extent to which they are implementing these principles and the kinds of practices that support them is extremely  powerful. We know that professional development as usual – sending staff off to a training here and a training there – does not change organizational practice in sustained ways.  The Forum’s Center for Youth Program Quality is working with over 60 networks of OST programs around the country to build their capacity to do this kind of continuous quality improvement work. This kind of data-driven continuous quality improvement approach is the direction that education and other human services fields are moving as well.

___________________________________________________________________

Karen Pittman is a co-founder, President and CEO of the Forum for Youth Investment. She started her career at the Urban Institute, conducting numerous studies on social services for children and families. Karen later moved to the Children’s Defense Fund, launching its adolescent pregnancy prevention initiatives and helping to create its adolescent policy agenda. In 1990 she became a vice president at the Academy for Educational Development, where she founded and directed the Center for Youth Development and Policy Research and its spin-off, the National Training Institute for Community Youth Work.

In 1995 Karen joined the Clinton administration as director of the President's Crime Prevention Council, where she worked with 13 cabinet secretaries to create a coordinated prevention agenda. From there she moved to the executive team of the International Youth Foundation (IYF), charged with helping the organization strengthen its program content and develop an evaluation strategy. In 1998 she and Rick Little, head of the foundation, took a leave of absence to work with ret. Gen. Colin Powell to create America’s Promise. Upon her return, she and Irby launched the Forum, which later became an entity separate from IYF.

Ongoing Debate On Using 21st CCLC Funds for Extending the School Day

By Sam Piha


On this blog we have been following the discussions nationwide about whether or not to allow 21st CCLC funds for afterschool to be diverted to extending the school day. We have interviewed several advocates, including Jennifer Davis and Carla Sanger, about this topic. 


Recently, the Washington Post posted a blog that featured differing views authored by Jennifer Davis, co-founder and president of the National Center on Time & Learning, and Jodi Grant, executive director of the nonprofit Afterschool Alliance. To read their comments and the comments of many readers, click here. 

Adding Arts to STEM: Making STEAM

By Sam Piha

Thinking about how to promote STEM (Science, Technology, Engineering, and Mathematics) in afterschool is a topic that has everybody talking; this includes activities during the school year and in the summer. A recent article in Education Week focused on efforts to gauge girls in STEM activities, and featured Techbridge, a Bay Area program.

The Afterschool Alliance recently published a paper offering an analysis of the impact and outcomes of attending a high-quality afterschool program that emphasizes STEM activities. They found that “attending high-quality STEM afterschool programs yields STEM-specific benefits that can be organized under three broad categories: improved attitudes toward STEM fields and careers; increased STEM knowledge and skills; and higher likelihood of graduation and pursuing a STEM career.”

Now many STEM advocates are making the case that adding the disciplines of art and design to STEM activities (STEAM) greatly enhances the learning experience. A recent EdWeek article explored the idea of merging the arts with STEM to further unlock students’ creativity and innovation.

The National Science Foundation has underwritten a number of conferences on this subject, including hosting a forum at Rhode Island School of Design titled, “Bridging STEM to STEAM: Developing New Frameworks for Art-Science-Design Pedagogy.”

What do you think? Should the arts be integrated into afterschool STEM activities?

Addressing the Afterschool Needs of Youth in Rural Settings

By Sam Piha

There has been a growing interest in the needs of rural youth and how we can better serve their needs through afterschool programs. We want to call your attention to a number of new resources that can serve to support this work:

• Temescal recently published a paper, I Feel Like I’m Somebody: Older Youth and High School Afterschool Programs in a Rural California Town. This paper was authored by Logan Robertson, who has conducted research on such programs and managed an afterschool program for older, rural youth.  

• A new blog titled “Rural Education”offers new important information on the needs of rural youth. The latest blog informs us of recent research that reveals that rural youth have a greater incidence of substance abuse and a lower rate of college enrollment. 

• The California Afterschool Network website offers a number of articles and relevant events that promote the importance of quality afterschool programs in rural settings.

New California After School Division Chief

By Sam Piha

Michael Funk

On December 15, 2011, Tom Torlakson, State Superintendent of Schools, announced that Michael Funk will head the new After School Division within the California Department of Education (which until the recent CDE realignment had been a unit of another division).

Michael Funk served as the founder and Executive Director for the Sunset Neighborhood Beacon Center since 1996 and has served on the California Advisory Committee on Before and After School since its inception. He has also been the co-director for the Learning in Afterschool Project (LIA).

Michael has been a strong advocate for quality afterschool programs that reflect the learning principles of the LIA Project – learning that is active, collaborative, meaningful to the participants, promotes mastery, and expands horizons. We congratulate Michael on this appointment and look forward to his leadership of the California Afterschool movement. 

Expanding Horizons and Global Learning

By Sam Piha

On January 27, 2012, the Learning in Afterschool Project will sponsor a one-day conference entitled How Kids Learn.  One of our featured speakers will be Alexis Menten from the Asia Society.

Alexis Menten, Asia Society

Learning that expands young people's horizons is one of the five key learning principles of the Learning in Afterschool Project. This includes the promotion of global learning and understanding. You can read Alexis’ views on expanding horizons, lengthening the school day, and global competencies by viewing her post on the Global Learning blog. The Asia Society also offers a resource website to guide afterschool programs in promoting global learning entitled Expanding Horizons. 

To learn more about an award-winning youth program that is focused on developing young people’s global competencies and view some of their resources, check out Global Kids. 

A new publication entitled Virtual Vacation: A Leader's Guide offers an integrated approach to promoting global awareness. It will be available from Temescal Associates very shortly. This approach was developed by the NHP Foundation and implemented in afterschool programs near New Orleans.  

     ___________________________________________________________

Alexis Menten is a Director in Asia Society’s Education division, where she leads afterschool and youth leadership initiatives for the Partnership for Global Learning. The Partnership connects educators, business leaders, and policymakers to share best practices, build partnerships, and advance policies to ensure that all students are prepared for work and citizenship in the global 21st century. In support of this mission, Alexis directs Asia Society’s Expanding Horizons initiative, which provides technical assistance, professional development trainings, and resource publications that advance global learning as an essential approach for all high-quality afterschool and summer programs, and as a means to build collaboration between schools and community partners. She is a graduate of Bryn Mawr College.

Why Collaborative Learning?

 By Sam Piha

The Learning in Afterschool Project promotes collaborative learning as one of its five learning principles of quality afterschool programs.

Why do we emphasize collaboration? YouthLearn offers some important reasons why collaborative learning is important:

“The New Economy is distinguished by interlocking partnerships and networks of people and organizations. In the past a loner may have been at a disadvantage, but today, he or she may not be able to survive. Businesses work in teams; they outsource, form alliances and hire without ever running an advertisement. Put simply, the opportunities available to your kids will be as much dependent on their mastery of communication and collaboration skills as on writing or math skills.

People care about things they feel a part of and about which they feel at least some degree of ownership. If your kids are involved in planning and decision making, they'll show a level of enthusiasm and curiosity that schools can only dream of.

More knowledge, creativity and ideas can be found in two minds than in one, and even more can be found in four or in 10. When kids see what their friends have done with a project, they add to it and create something even more original. In out-of-school programs, you have the freedom to let kids work in groups almost all the time and to shift the groups around so that kids learn from lots of different people.

Face it, there's no way you can know about everything, especially once new technologies like the Internet, computers, software, scanners, cameras and all the other devices are added into the mix. Then again, why should you have to? Don't be afraid to let kids teach each other when one of them becomes an expert in PhotoShop and another an online audio wizard—that's the real power of collaboration. Rather than being mired in the details of ever-changing software programs, you can focus on the important jobs of coach, guide and educator.”

The Global Development Research Center offers a number of useful tools for those interested in exploring collaborative learning, with specific techniques and methods. We recommend that you visit this site and other sites that will appear when you do an internet search for "collaborative learning".