STEM Education & Training TIG Week: “Are we changing with the times or stuck in a rut?” A Resource Hub of Surveys for Evaluating STEM Education Programs by Sara Bolduc, Melissa Page, and Marcia Nation

Date: Sunday, March 30, 2025

Hi! We are Sara Bolduc, Melissa Page, and Marcia Nation, three past and present STEM TIG leaders. Sara Bolduc, PhD, President of Sara Bolduc Planning and Evaluation LLC and Melissa Page, PhD, Evaluator at The Evaluation Group, were Co-Chairs during 2024 for the STEM Discussion and Networking Series. Marcia Nation, PhD, Principal and Owner of Nation Evaluation Consulting, LLC, is the current Program Chair.

The STEM Education and Training TIG held a Discussion and Networking event in 2024 that asked: “Are we changing with the times or stuck in a rut?” The 90-minute virtual discussion included covered how surveys provide an easy mechanism to evaluate our work. While we don’t want to reinvent the wheel, are we missing the mark on an ever-changing landscape, especially in the STEM field? During the lively discussion, people shared experiences with existing surveys, de, and shared resources useful for evaluators.

Hot Tip

Before using any survey, follow these steps.

1. Know the outcome you seek. Look for surveys that answer the right questions for your outcome in as few questions as possible. When you cannot find a survey that aligns well to your outcome or your program, consider developing questions that will benefit your evaluation of the program, but also the broader community. Consider publishing your survey to update the field on newer topics. A helpful resource for this is Designing Quality Survey Questions (2019) by Sheila B. Robinson and Kimberly Firth Leonard.
2. Know your audience (age, demographics, biases, context). Think about edits that may enhance the audience’s understanding of the question being asked and for the context you wish to evaluate. If significant edits are made (e.g., regarding readability, change of words, etc.), ensure you pilot test the items with potential survey respondents and  plan to rerun the validity and reliability statistics for the new survey.
3. Decide if you need demographic data. In today’s politicized environment, many survey respondents are hesitant to answer questions about their age, gender, sexual identity, or other sensitive information. Do not ask these questions if you do not intend to analyze the data using them. If you need thus information, consider placing these items at the end of the survey.

Rad Resources

The following survey titles and links are a sample of cool resources generated during the networking event. Please carefully review any surveys and any copyright or citation requirement before using the survey.

• Attitudes toward Mathematics Inventory
• Science Teaching Efficacy Belief Instrument (STEBI)
• Mathematics Teaching Efficacy Beliefs Instrument (MTEBI)
• Sense of Social Fit Scale
• Panorama Social-Emotional Learning Survey (USER GUIDE): Topics and questions for Students, Teachers and Staff
• Panorama Family-School Relationships Survey
• S-STEM and T-STEM (and many, many more) from The Friday Institute (2012)
• Hour of Code: Elementary Student Coding Attitudes Survey (ESCAS-Mason & Rich 2020)
• Undergraduate Research Student Self-Assessment (URSSA)
• Mentorship Effectiveness Scale
• Critical Thinking Assessment Test (CAT)
• Mentor Behavior Scale
• Science Competence Measure
• GRIT -Consider The Problem With Grit before using this measure  
• Critical Iceberg Model –  Send  and email to hosava.kretzmann@nau.edu for a copy
• Measuring Community Cultural Wealth
• Surveys of Enacted Curriculum
• National Assessment of Educational Progress (NAEP)
• Retrospective survey for pre/post designs

Lesson Learned

Our discussion and networking session was a great example of crowd sourcing by asking participants for the names of surveys they have used in the past or were familiar with for use in the STEM fields. AEA 365 Blogs and TIG discussion boards are a great source of information or a platform to ask and receive feedback.

Asking other evaluators to review or field test your survey can also provide valuable information about how a participant may be interpreting the survey item or to point out structural issues with survey items.

Developing survey items requires careful consideration of word choice, simplicity, conciseness, and use of language relevant to the program but accessible by the audience. Reach out to experienced evaluators, program staff, and potential survey respondents to ensure your questions will be received well and will provide the answers to evaluation outcomes.

The American Evaluation Association is hosting Extension Education Evaluation TIG Week with our colleagues in the STEM Education & Training Topical Interest Group. The contributions all this week to AEA365 come from our STEM Education & Training TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.