Date: Wednesday, April 2, 2025
Hi everyone! My name is Elyse McCall-Thomas, Evaluation Manager with the Natural Sciences and Engineering Research Council (NSERC). At the beginning of my career, I had the opportunity to evaluate a program that funds organizations offering hands-on STEM promotion and education activities for children and youth (K-12), as well as training and resources for teachers to help them further engage and encourage their students to develop an interest in STEM. I loved this evaluation! Not only because it was extremely interesting, but I also learned about the importance such activities play in exposing, engaging and encouraging children and youth to consider further STEM education and careers.
As I prepare to evaluate this program again, I am looking back on lessons learned and opportunities to improve our scope, approach and methods.
The organizations funded by the program vary widely as do the STEM education activities they provide (e.g., science fair, after-school robotics program, summer camps). Also, different activities focus on different groups including children, youth and teachers. This highlights the need for a flexible evaluation framework that respects the diversity of programs and beneficiaries. By tailoring evaluation methods to distinct goals and audiences, we can more accurately measure the impact of STEM education.
Examine separately the activities focused on teachers from the activities focused on children and youth. For the latter group; disaggregate the activities by duration and outcomes they are likely to address (e.g., short-term programs create awareness and interest, longer term programs help build foundational skill and increase engagement). This will facilitate the matching of indicators and data collection instruments to the actual purpose of each STEM education activity.
Enriched STEM training and resources for teachers not only builds on their existing knowledge and curricula but also creates a multiplier effect that can result in greater sustained exposure to and engagement in STEM activities for students. Consequently, connecting with teachers can provide valuable insights regarding the quality of the training and resources they received, and how STEM training translates into classroom practice including what works and what does not work in different contexts. Reaching teachers, however, can be a challenge if you don’t have access to names and contact information.
Multiple pathways can be leveraged to help increase the participation of teachers in evaluation including school boards, unions and associations, social media, advertising on STEM hubs, large STEM-focused organizations and snowball sampling such as asking teachers to forward a survey to colleagues. Regardless of which pathways are used, it is important to consider other best practices to increase response rates including brief instruments, clear purpose and language, assurances of anonymity and confidentiality, strategic timing and providing incentives where feasible.
The American Evaluation Association is hosting Extension Education Evaluation TIG Week with our colleagues in the STEM Education & Training Topical Interest Group. The contributions all this week to AEA365 come from our STEM Education & Training TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.