Date: Tuesday, November 25, 2014
Message from the President - It's Time to Put Our Vision Into Action
It is hard to believe that Evaluation 2014 has already come and gone. Even though the conference is over, those who are interested still have a lot of work to do to carry forth the conference theme: Visionary Evaluation for a Sustainable, Equitable Future.
We’ve had a lot of excellent conversations around this topic. The next step is to move from conversation to action, individually and collectively. Here are four areas where I think evaluators can apply systems thinking and build relationships to help develop a sustainable, equitable future. What are yours?
1. We need a new paradigm of how humans and nature intersect. Simply put, we need to make significant changes to how we live to live within the constraints of our natural resources and our expanding global population.
During a conference session on Wednesday afternoon, Matt Keene showed the three-minute movie “Welcome to the Anthropocene,” which outlined the trends we have seen over the past 250 years: exponential population growth; more people moving to cities; globalization; global tourism; and massive movement of rock, soil, and other natural resources, all leading to the rise in greenhouse gases, a changing climate, the loss of biodiversity, the acidification of the oceans, and rising sea levels.
Evaluators can bring attention to these trends that provide the context for our work. We analyze data, look at trends, and work in support of the public good. How do we position our field and practices so we help society move in a direction in keeping with our global situation?
2. We need to pay more attention to the significance of the segmentation of our social systems. We have not adequately recognized the consequences of the segmentation of our social systems and how it undermines our ability to move toward new goals. For example, education, health, financial markets, water, and agriculture are separately organized. So many of the programs we evaluate are within these systems and we are not seeing the interconnections among them. Yet, if we are to move to a sustainable, equitable future, these systems may need to be redesigned with different relationships so they better support this goal.
3. We need to focus more heavily on sustainability, the intersection of human and natural systems, and the redesign of our social systems at the level of geographic communities.
A geographically based local community seems to be an important unit for considering the combination of economic, social, and environmental boundaries. What if we encouraged stakeholders of an intended evaluation to build their goals and revise their paradigms at the level of a geographically based local community?
From a natural environment perspective, a geographically defined community would fit the geographic boundaries of the area. Experts in the natural environment could help inform how to define the unit of analysis, governance, and change that serves both the human and natural systems. From an economic perspective it includes business, energy, technology, education, health, and other vital services that people need. From a social justice perspective, its boundaries would consider the cultures and perspectives that are connected to that economic and environmental setting. These boundaries may be porous, dynamic, and flexible, but by thinking this way, you are likely to find an orientation within which you can position your evaluand.
4. We need to pay more attention to polycentric/multiple decision making and structures. We must look at the interplay among formal hierarchical social systems, private business, networks, partnerships, social movements, families, and neighborhoods. What I mean by this is that all of these types of structures – each with its range of decision making from authoritative and cohesive to collaborative and informational – are leverage points for moving in a desired direction. They are interacting with and influencing one another. We need to attend more to the interplay of these different structures and decision making, as they help break down social systems built on old paradigms and create new ones in support of new paradigms.
During the conference, I heard from many people who wanted AEA to help them act in accord with a sustainable, equitable future. There are many possible ways to do this, such as encouraging networking, establishing a group to develop a position statement, connecting with other organizations, working within TIGs, setting up a task force to develop a plan of action around the issue, and providing webinars and sponsoring communities of practice about the topic. What are your thoughts on what, if any, action AEA should take? If you would like to get involved in thinking this through, or if you have other ideas for how we can support a sustainable, equitable future for all, please email me at bparsons@insites.org.
I look forward to hearing from you, and I wish you all a happy Thanksgiving!
Warm Regards,
Beverly Parsons
AEA 2014 President
Face of AEA - Meet Chari Smith
Name: Chari Smith
Degrees: B.S., Psychology; M.S., Marketing
Years in Evaluation Field: 13
Joined AEA: 2010
Affiliation: CRSmith Consulting, Evaluation into Action
Why do you belong to AEA?
I believe evaluation should be accessible, practical, and, above all else, usable. As a sole proprietor, I don’t have anyone down the hall to run a question by, or discuss methods. AEA, and the local chapter OPEN, is my connection to the evaluation community. I appreciate learning about evaluation trends, new tools, and approaches so I can continually improve my own practice.
Why do you choose to work in the field of evaluation?
I like helping organizations understand the value and use of program evaluation. It’s an empowering journey to facilitate the organizational cultural shift. I like the a-ha! moments, when someone realizes how a piece of data can inform program change. It’s a privilege to do this through teaching, coaching, and doing program evaluation with a range of nonprofit organizations.
What's the most memorable or meaningful evaluation you have been a part of?
A two-year project that included a cohort of nine organizations that worked in economic development with Native communities. My client was the coordinator for this cohort, providing quarterly meetings and onsite coaching. The goals were to build organizational capacity among these organizations as well as define social entrepreneurship.
Through interviews, observations, and surveys, we used data to make program design changes. We also did annual reports and discussed the findings with the funder as well as what program changes were made based on the data. This was meaningful because the client modeled data-based decision making. The cohort built strong relationships over the two years, and continues to work together today. Through an evaluative lens, I witnessed how a complex project evolved and ultimately thrived.
What advice would you give to those new in the field?
1. Build authentic relationships. This is a critical ingredient in establishing trust. This trust enables people to believe in the evaluation process.
2. Validate their thoughts. When you come across those who disagree, listen. Validate what they’re saying. Offer a different opinion.
3. Be clear on what you offer. If you’re offered a project that is of no interest, and you do not have the skill set for it, be up front about that if you choose to pursue the project. Or, if you can, pass on it all together. Write down what your ideal project and/or client would look like, and go after that.
Diversity - Meet the New MSI Fellows
From Zachary Grays, AEA Headquarters
Among the many exciting highlights of Evaluation 2014 was the graduation of the 13th Minority Serving Institution (MSI) fellowship cohort. This cohort had an incredible year, introducing a brand new FAQ based on their rigorous experience in the program to orient future fellows. Congratulations to Tamara Bertrand, Edilberto Raynes, Andrea Guajardo, Denise Gaither-Hardy, and Ana Pinilla! AEA commends your tremendous work and contributions to the MSI program.
AEA is proud to announce the newly selected fellows for the 14th MSI fellowship. With over 40 exceptional applicants, it goes without saying that narrowing down the selected five fellows was incredibly difficult. Read on to meet the 2014-2015 MSI fellowship fellows!
Tiffeny Jimenez is an assistant professor at National Louis University (NLU) and director of the Ph.D. program in community psychology. She has mostly worked and written on various community-based research projects spanning social justice issues, including coordinating a collaborative statewide cross-disability leadership training initiative, consulting on initiatives to change the culture of a university to be more supportive of women faculty in the STEM fields.
Julia Lechuga is currently an assistant professor at The University of Texas at El Paso. Dr. Lechuga's research focuses on the development, cultural adaptation, and dissemination and implementation of behavioral interventions to reduce the risk of infectious disease in ethnic minorities and underserved populations.
Tamarah Moss, Ph.D., MPH, MSW, is an assistant professor with Howard University School of Social Work. As a social work educator and public health social work researcher and practitioner, Dr. Moss has expertise in the areas of reproductive and sexual health, with topics including family planning, HIV, health and social service delivery, and social work teaching and learning.
José A. Muñoz is an assistant professor in the Department of Sociology at California State University, San Bernardino. He teaches courses in sociological theory, qualitative research, and Latino health. José’s research areas include social movements and immigrant politics.
Elizabeth Williams, Ph.D., is an assistant professor in the Master of Public Health Program at Tennessee State University in the Department of Public Health, Health Administration, and Health Sciences. An applied medical anthropologist by training, she has previously served as the associate director of the Office of Minority Affairs at the Vanderbilt-Ingram Cancer Center and as the director of the Office of Disparity Elimination with the Tennessee Department of Health.
AEA welcomes the newest MSI cohort! We look forward to working with them over the next year. AEA would also like to thank Art Hernandez, MSI program director, for his continued leadership and exemplary guidance of the MSI program. Visit the AEA website to learn more about the MSI Program and this year’s fellows.
Have a story to share for the AEA diversity column? Contact me at zgrays@eval.org to learn how to share your diversity in evaluation stories. I would love to share your story!
Potent Presentations Initiative - What to Do Now that the Conference is Over
From Stephanie Evergreen, Potent Presentations Initiative Coordinator
I’m sure that at Evaluation 2014 you learned heaps of information to improve your practice and your operating theories. No doubt your evaluation expertise has improved. I’m betting you also learned a lot about how to give a presentation. Whether it was through the Potent Presentations Initiative (p2i) training materials or just by watching other evaluators present, you likely picked up tips and tricks that can make a presentation really shine. Don’t wait until Evaluation 2015 to try those ideas!
The cool thing about presentation skills is that they transfer really easily between a conference setting and the other places you need to present as part of your job. In fact, our conference survey from Evaluation 2013 showed that 56.5 percent of you who have used p2i have applied things you learned from the Potent Presentations Initiative in other circumstances, such as giving stellar talks on the job. Fifty-seven percent also reported using p2i training for presentations at other conferences outside of AEA (and we are super happy about that – keep it up and tell them we sent you!).
For more inspiration on how p2i and strong presentation skills in general can be applied to improve presentations in other contexts, read our posts from AEA365. Here are some highlights from those posts:
Try a new idea per month and by Evaluation 2015, you’ll be set up with a rockstar skill set!
Policy Watch - Evaluation Policy and Capacity Development
From Cheryl Oros, Consultant to the Evaluation Policy Task Force (EPTF)
Thank you to all who attended sessions on evaluation policy at the recent AEA conference in Denver and to those who provided input on policy options and future directions. The EPTF has been collecting and studying the evaluation policies and plans of agencies and other countries while also interviewing evaluation office directors to better understand how to influence the development of evaluation policy. We also are very interested in the consequences of such policies. If you have or know of evaluation policies and plans that you would like to share, please let me know. Your input will be quite valuable. The EPTF plans to make a set of policies and plans available on its website. These can serve as examples for other agencies and countries to use in developing or updating their own statements.
On another matter, AEA’s International Working Group and the EPTF recently worked together to encourage the U.S. delegation to the United Nations (U.N.) to support a U.N. resolution during the International Year of Evaluation in 2015 (EvalYear) to build national capacity for the evaluation of development activities. AEA recently sent a letter to the U.S. ambassador to the U.N. encouraging a vote in favor of the forthcoming resolution, entitled Building Capacity for the Evaluation of Development Activities at the Country Level. Consistent with EvalYear, international cooperation is encouraged in building capacity for evaluation at the country level. The resolution further requests that an update on the progress made in building evaluation capacity be presented and considered during the quadrennial comprehensive policy review of operational activities for development of the U.N. system.
Turning to a recent report related to evaluation policy, the U.S. Government Accountability Office (GAO) has released Program Evaluation: Some Agencies Reported that Networking, Hiring, and Involving Program Staff Help Build Capacity, GAO-15-25. By way of background, the GPRA Modernization Act of 2010 aims to ensure that agencies use performance information in decision making, and the administration has encouraged agencies to strengthen their program evaluations. In a survey of agencies, GAO found uneven levels of evaluation expertise, organizational support, and evaluation use. Half the agencies reported increased efforts to improve their evaluation capacity since 2011. About half reported increased use of evaluations to support management and policy decisions, with over a third of those reporting doing so to a moderate or greater extent. The report describes which resources and activities the agencies found useful for improving their capacity to conduct and use evaluations.
The AEA EPTF stands ready to engage in an exchange of information and materials with U.S. agencies and other countries to advance evaluation policy and planning and to learn from their execution. If you have any comments, suggestions, or questions regarding the development, revision, or implementation of evaluation policy, contact me at EvaluationPolicy@eval.org. MY EPTF colleagues and I look forward to hearing from you.