Gratitude in Evaluation: Asset-Based Program Evaluation During COVID-19

By Guest Blogger Jason Spector, Policy Studies Associates

Jason Spector 
Policy Studies Associates

We are entering a holiday season unlike any other, with limited gatherings and far too many empty seats at the table. Out-of-school-time (OST) staff, youth, and families face deep grief, challenge and uncertainty. It is not a time where program evaluation is at the forefront of our minds, and when evaluation does come to the fore in the OST field, it is often in the context of service gaps, disparities, and young people’s learning losses. These are all too real and point to the deep challenges of this era, from the disparate educational and health impacts of COVID-19 on communities of color to the mounting academic learning loss that is inextricably connected to race and class across the country. 2020 is a year of intersecting health, economic, education, and racial justice crises; it is also the year where resilience has shone through and there is much to express gratitude for and celebrate in the OST field and beyond. 

Asset-based evaluation can be a helpful, and hopeful, approach during this time and into the future.

What do we mean when we say asset-based evaluation? Consider the definition from the Glasgow Centre for Population Health: “Asset-based approaches emphasize the need to redress the balance between meeting needs and nurturing the strengths and resources of people and communities. They are ways of valuing and building on the skills, successes and strengths of individuals and communities, which focus on the positive capacity of individuals and communities rather than solely on their needs, deficits and problems. These assets can act as the foundation from which to build a positive future.” In short, assets do not diminish problems and needs, but they do serve as a broadened foundation upon which to build an evaluation strategy.

Source: California Afterschool Network (CAN)

How can an asset-based evaluation approach help address your organizational needs during COVID-19? Here are some helpful tips to get started:

• Reexamine how you frame continuous improvement. Continuous improvement often involves developing and adapting strategies to meet problems or challenges. This is necessary, but incomplete. An asset-based approach can reframe continuous improvement as both addressing challenges and intentionally identifying program, community, and individual strengths—and then developing strategies to amplify those strengths. For example, a strength of the virtual program structure is it can attract a broader range of staff and volunteers across geographic boundaries. An asset-based continuous improvement question asks when virtual programming makes sense to continue in the future to broaden the staff and volunteer talent pool.
• Broaden your measures of success. OST youth, families, and staff are benefitting from new services (e.g. food support and IT connectedness) and developing new socio-emotional strengths as they collectively face down adversities. At the organization level, providers are learning to rapidly adapt and develop new systems for virtual and blended learning that will increase organizational capacity for years to come. These are worthy indicators of success.
• Use qualitative data collection to bring in more voices. Inclusivity is critical to an asset-based evaluation. Voices from students, families, and staff can rise to the fore through interviews and focus groups, as well as through open-ended survey questions. Who better to speak to the assets of the community than community members? 
• Use an analytic lens that seeks out the positives. Both qualitatively and quantitatively, it is critical to analyze where are things working well and for whom. Whether these are individual case studies or examples of successful mass mobilization efforts (e.g., increasing student access to devices), there is much to learn from individual, program, and community strengths that can be both celebrated and scaled.
• Celebrate the wins through positive feedback loops. When a staff member is called out by a student or family member for excelling in their role, how do you ensure that that information not only makes it into a grant report, but directly back to that staff member in a timely manner? Building positive feedback loops where information flows to all stakeholders to celebrate wins—not just address challenges—builds buy-in for evaluation and bolsters staff and community morale. Let evaluation be a source of inspiration!

As you consider taking one or more steps toward an asset-based evaluation approach, be mindful of evaluation overload. It may be preferable to do fewer things and do them well during this time. Accordingly, be cognizant of matching your evaluation strategy to your organizational needs and capacity, so that asset-based evaluation doesn’t become ‘just one more thing.’ And as you dive in, please share your examples of how you practice gratitude and/or asset-based evaluation—I am grateful for, and look forward to, learning alongside you.

Jason Spector is a Senior Research Associate at Policy Studies Associates. He formerly served as the Senior Director of Strategy and Evaluation at After-School All-Stars. Jason recently authored a chapter in the new book Measure, Use, Improve! Data Use in OST! titled “What’s Your Why? Matching Evaluation Approach to Organizational Need.” He can be reached at jspector@policystudies.com or on LinkedIn.    

 

A Look Ahead to Afterschool in the Next Decade

As we head into a new decade, we thought it would be appropriate to hear from afterschool leaders regarding how they viewed the opportunities and the challenges to afterschool moving forward.

COREY NEWHOUSE, PUBLIC PROFIT
EMERGING TRENDS/ OPPORTUNITIES: There are more and more options for expanded learning programs to specialize in certain kinds of practices, whether STEM, arts, CTE, culturally sustaining practice, and so on. How they navigate among these choices is an open question for me, and possibly an opportunity for additional support and guidance.

CHALLENGES: Challenges include a strong economy that creates more competition for talent, minimum wage laws in many cities and states, plus AB 5 in California. The cost pressures on programs continue to increase, and philanthropies are lessening their support for expanded learning rather than increasing it. Also, the distinction between social and emotional learning and positive youth development practices can use more clarification, especially from a practice perspective.

JEFF DAVIS, CALIFORNIA AFTERSCHOOL NETWORK 
EMERGING TRENDS/ OPPORTUNITIES: The vision described here is THE FUTURE. In 2020 and beyond, the Expanded Learning field will leverage the Science of Learning and Development to strengthen the case and strengthen our practice that we create the conditions necessary for youth to thrive. Expanded Learning programs are also serving communities that have a lot of needs. In 2020 and beyond Expanded Learning programs will partner better with health, mental health, and social service sectors to offer relevant interventions when kids and families are in need of support beyond what we can provide in our programs.

CHALLENGES: The rates that support Expanded Learning programs are still woefully insufficient and decision-makers such as legislators are becoming fatigued with the message that Expanded Learning programs need more investment to meet baseline operational costs, let alone live up to their true potential. 2020 will be another critical year to rally our staff, families, and communities to communicate clearly and with large numbers that Expanded Learning programs are worthy of investment.

STU SEMIGRAN, EDUCARE FOUNDATION
EMERGING TRENDS/ OPPORTUNITIES: I see the emerging trends include the issues of mental health and well-being (both for our students and our staff), school safety (anti-bullying / positive school climate), equity & inclusion, and student voice.  Some of the focus areas that address these emerging issues and trends include:  SEL, Trauma-informed practices Restorative Justice, Mindfulness, and Growth Heartset.

CHALLENGES: Adequate funding and developing and retaining quality staff.



DEBORAH MORONEY, AMERICAN INSTITUTES FOR RESEARCH

EMERGING TRENDS/ OPPORTUNITIES: I see and hope for a rising priority in the shape and value of programs for older youth, and youth action and voice in society.

CHALLENGES: We need to rise to the challenge to systemically support the adults in the field with meaningful career pathways and preparation to support their critical role in youth learning and development.

BILL FENNESSY, THINK TOGETHER 
EMERGING TRENDS/ OPPORTUNITIES: The emerging trend in Expanded Learning that continues to grow, and I believe is on the verge of doing so exponentially, is "Workforce Readiness", or the "Career" part of the new "College and Career" focus.  There is now a proliferation of Industry Pathways and Academies in high schools, which to produce great outcomes really requires important foundational work at both the middle and elementary school level.  The opportunities to create, develop, and innovate effective programming models in Expanded Learning Programs to support this Workforce Readiness movement are currently boundless.  This is truly an incredible opportunity for Expanded Learning Programs to provide more than multiple modalities of learning that directly support the Instructional Day.

CHALLENGES: The biggest challenge I see facing the field is the ability to find and retain program level staff.  With the dramatic increases in minimum wage, it is almost impossible for programs to financially compete for quality program level staff.  There is work now beginning which hopes to make work experience/internships in Expanded Learning Programs as a part of a course of study for post-secondary students on a Teaching Pathway.  Teachers that have Youth Development experience have better student engagement skills that directly result in excellent classroom management.

REBECCA FABIANO, FAB YOUTH PHILLY 

I’m thinking about:

• The Census and its impact on Youth Programs/OST; also thinking about voter registration (of staff and participants/families). 
• Equity and professional development (PD)- who gets to go to conferences for example, where do we invest our PD dollars in our organizations and in the field? 
• The role of extended learning programs in addressing poverty (e.g, living wages, skills for 21st century employment, entrepreneurship).