Date: Monday, March 10, 2025
Hello, AEA365 community! Liz DiLuzio here, Lead Curator of the blog. This week is Individuals Week, which means we take a break from our themed weeks and spotlight the Hot Tips, Cool Tricks, Rad Resources and Lessons Learned from any evaluator interested in sharing. Would you like to contribute to future individuals weeks? Email me at AEA365@eval.org with an idea or a draft and we will make it happen.
Please note: This blog represents the author’s views and not the views of any professional affiliation.
We are Dr. Liz Johnson, Principal and Owner of Liz Johnson Education Consulting, and Dr. Michelle Burd, consulting as Principal and Owner of Burd’s Eye View. We evaluate education programs in a state where legislators are passing bills that are inhibiting inclusive practices on college and university campuses. As culturally responsive practitioners, we have noticed repressive legalism shaping program changes in some of our projects. Repressive legalism is “the interpretation and application of legal norms and other facets of the legal environment in a manner that shuts down a focus on other approaches (e.g., institutional responses promoting inclusion for students of color)” (Garces et al., 2021, p. 1059).
In doing evaluations, we were asked to scrub questions about race, ethnicity and gender from surveys.
We were told programs intending to implement inclusive practices were discontinuing them.
We watched teams drop plans to recruit 50% minoritized students.
Witnessing these decisions highlighted the politics of evaluation. Are we judge àla Scriven? Do we support leaders’ organizational decision-making? Is our role collaborator, facilitator, educator, coach? Evaluators take on different roles at different times. What are evaluators’ roles when repressive legalism challenges inclusive higher education practices?
In one project, we decided to advocate for student-centered practices that ensure all students receive high quality mentoring that builds on their life experiences, interests and knowledge bases. To illustrate, faculty mentoring workshops in cultural responsiveness, which strengthens student outcomes, were dropped. We followed Boyce’s (2017) strategy, gently questioning clients’ thinking, nudging, meeting them where they were. The University’s compliance document stipulated no training regarding race, color, gender, sexual orientation could be mandatory. We asked, “Why not voluntary?” The document included exceptions for research, grant activities, and data collection – exceptions we could highlight.
We have been educating ourselves about the law with Rad Resources like Best College’s Anti-DEI Legislation Tracker. This index summarizes the latest legislation in each state. Knowing the law can promote dialogue between evaluators, grantors, grantees, and lawyers developing guidelines.
Understanding specific laws and policies governing education where we work, we can pose questions that open up other approaches like:a) What does the law actually prohibit? What’s still allowable?b) How are people at near peer institutions navigating this?c) What language allows the project to implement the strategies and reach the same outcomes? Could projects shift to concepts like providing access, increasing social and economic mobility, fostering inclusion, centering students, supporting low income, underserved, or first generation students?
We continue to study as the landscape changes.
For example, from 1740-1834, southern states banned teaching enslaved people to read. See Ed Trust’s up-to-date heat map of attempted, pending, enacted Anti-DEI legislation by state.
Demographic data help colleges and universities implement their missions of providing robust education for all – documenting evidence about communities. How and when demographic data are used matters, not that it exists. And laws make exceptions for research and grant-funded programming including support for veterans and low income, underserved, or first generation students. See the University of Houston’s explainer outlining practices supporting students, faculty and staff with exemptions for research and data collection.
It takes energy to practice social competencies of evaluation with clients under pressure to appear neutral while laws challenge equity-oriented movements. Socially competent evaluators are “allaying fear and anxiety, establishing rapport, building and maintaining professional credibility, recognising tacit social dynamics, preventing and managing coercion attempts, and preventing and managing hostility” (Moretti, 2021). Culturally responsive evaluators must preserve social emotional resources needed to listen calmly, ask with curiosity, read carefully, collaborate regularly, and evaluate responsively as institutional actors navigate legislation challenging implementation of inclusive practices and policies.
Boyce, A. S. (2017) Lessons learned using a values-engaged approach to attend to culture, diversity, and equity in a STEM program evaluation. Evaluation Program Planning. Vol. 64, 33-43. doi: 10.1016/j.evalprogplan.2017.05.018
Garces, L. M., Johnson, B. D., Ambriz, E., & Bradley, D. (2021). Repressive Legalism: How Postsecondary Administrators’ Responses to On-Campus Hate Speech Undermine a Focus on Inclusion. American Educational Research Journal, 58(5), 1032-1069. https://doi.org/10.3102/00028312211027586
Moretti, E. (2021). Navigating the awkward, challenging social context of evaluation. Evaluation Journal of Australasia, 21(3), 163-176. https://doi.org/10.1177/1035719X211030713
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.