Date: Tuesday, September 30, 2014
Message from the President - Exceptional Education Awaits at Evaluation 2014
Dear AEA Colleagues,
Denver, here we come!
In just two weeks many of us will be gathering for Evaluation 2014, AEA’s annual conference. In the August newsletter, I talked about the conference theme, Visionary Evaluation for a Sustainable, Equitable Future, and the presidential strand sessions tied to the theme.
This month, I’d like to highlight several other important aspects of the conference. First, in regard to the conference theme, I’d like to call your attention to the plenary sessions. In the opening plenary on Wednesday, October 15, the conference thematic program leaders, Matt Keene and Lovely Dhillon, will join me in delving into the conference theme and its significance for your time at the conference, and beyond. We have a very special 14-year-old guest from Boulder, Colorado – Xiuhtezcatl Martinez – to help reinforce the importance of a sustainable, equitable future for generations to come.
The plenary sessions on Thursday and Friday will look at the private sector’s role in determining whether we will have a sustainable, equitable future and what it will look like.
On Thursday, October 16, Bob Willard will focus on how businesses are increasing their attention to their social and environmental responsibilities to society. He will explain why this is happening now, the shifts in thinking within corporations, the new form of capitalism, and the opportunities and challenges this provides for evaluators.
On Friday, October 17, Jane Reisman will facilitate a panel on social impact measurement. This session will pick up on Bob’s call the previous day for evaluators who have been heavily involved in the public, nonprofit, governmental, and philanthropic sectors to bring their knowledge to bear as social investment funds, corporations, and others wrestle with social impact measurement. This is an intriguing trend, and I’m confident that you’ll see new ways of envisioning your evaluation work.
Another important session is the closing plenary, held on Saturday, October 18, which focuses on AEA’s role in creating a sustainable, equitable future. This session will allow us to have a conversation about how AEA can support your plans for visionary evaluation and contribute to the global evaluation community. This is an important topic, and I hope to see many of you join in on the discussion.
Second, I’d like to make you aware of the opportunity our 50 Topical Interest Groups (TIGs) present for you to take a leadership role within AEA and the evaluation field in general. Each year, TIG chair and program chair positions need to be filled. These are great opportunities to build new relationships within your areas of interest, and I hope you will get involved. I encourage you to review the times of the business meetings for the TIGs in which you are involved (or want to get involved), and step up when the call for nominations for these leadership positions are made. If you can’t be at the conference, get in touch with the current TIG leaders and let them know your interest.
On a related note, the Leadership Development Task Force established by the AEA board had its first meeting this month. Its charge is to look at ways to help support leadership development within AEA. Keep an eye out for members of the task force – Kathy Newcomer, Nicole Vicinanza, Katye Perry, Leah Neubauer, and George Julnes – and offer your suggestions about the type of leadership development support you think would be helpful.
Third, I invite you to come to the Evaluator Competencies Listening Post on Friday afternoon, which will be led by board member Donna Podems. You’ll find it listed under the Presidential Strand. The board is eager to hear your thoughts on this important topic.
And finally, don’t forget to have some fun while you’re in the wonderful city of Denver! Many of the TIGs have planned social events, especially on Thursday evening. And the Wednesday and Friday receptions are great opportunities to make new connections and renew existing friendships.
I look forward to seeing many of you in a couple of weeks!
Warm Regards,
Beverly Parsons
AEA 2014 President
Face of AEA - Meet Hans-Martin Boehmer
Name: Hans-Martin Boehmer
Degrees: Ph.D. in economics, Georgetown University; BA in economics, University of Bonn
Years in the Evaluation Field: 5
Joined AEA: 2009
Affiliation: Former senior manager, Independent Evaluation Group, The World Bank Group; as of spring 2015, visiting professor, Sanford School of Public Policy, Duke University; and adjunct professor, School of International and Public Affairs, Columbia University.
Why do you belong to AEA?
I worked in the field of international development for 15 years prior to joining The World Bank Group’s Independent Evaluation Group. Unfortunately, or perhaps this is inevitable in a large organization, being in an evaluation department does not always make one the most popular person around. It was good to join the American Evaluation Association, both to find more peers, but also to strengthen the professional foundations of my work. At the end of the day, when being persuasive with counterparts, particularly in independent evaluation functions as opposed to the self-evaluation area that also exists within The World Bank Group, one’s professional credibility is perhaps the most important asset.
Why do you choose to work in the field of evaluation?
Looking back, I ask myself why I didn’t join the field earlier. When working with developing countries from the perspective of an organization like The World Bank Group, everyone has enormous responsibility to bring the best experience to clients. At the same time, it is inevitable that new challenges and solutions need to be found constantly.
My first assignment, over 20 years ago, was to conduct a self-evaluation of The World Bank’s structural adjustment support for Burkina Faso. Since this field was new to me and self-evaluation guidelines are readily available within The World Bank, I did my best to complete the task. However, I cannot say that I really understood how to ask the right questions, let alone apply approaches that would make this self-evaluation more than just another “checked box.”
Having seen how development projects are put together in both high-capacity and low-capacity countries, and how vast the responsibility is that comes with providing development advice that can have an impact on millions of people, I decided that I needed to learn more about the evaluation function without the pressure to comply with the need to show that everything is rosy. So when the opportunity arose to join the Independent Evaluation Group (IEG), I took it.
While I was at IEG, I had the privilege to be part of the launch of the CLEAR Initiative – which stands for Centers for Learning on Evaluation and Results – and work directly with many of our partner agencies to support the development of regional centers that have the task of strengthening the overall capacity, standing, and role of evaluation across virtually all continents. Being the chair of the board of the CLEAR Initiative for the last two and a half years was perhaps one of the highlights of my time at IEG.
What's the most memorable or meaningful evaluation that you have been a part of?
That’s a good question and the answer is perhaps surprising. I would point to an evaluation where I was the one being evaluated, rather than the evaluator. The U.S. Government Accountability Office (GAO) had been asked by Congress to look at the impact of the assistance by multilateral institutions on the development in Russia during the first decade of independence.
It was an eye-opener to see how systematic and in-depth the GAO carried out its mandate. At the same time, because The World Bank is not a U.S. entity and a lot of the normal powers of the GAO do not apply to The World Bank, it was always a balancing act between providing the requested information, working with the evaluators to help them understand the overall context in which we were working, and the particular constraints that apply to the work of The World Bank Group as an international organization that is governed by a board representing nearly all countries around the world.
I felt very fortunate to have as an institutional support group the auditor generals of the U.K., France, and Ghana who could always provide advice on what was an appropriate question for the GAO to ask, what material could be shared, and when to protect the proprietary information that The World Bank Group obtains from countries based on its institutional nature.
Nevertheless, I felt that we had a very good rapport with the GAO team and we were looking forward to the final report. As is common practice, we discussed the draft with the GAO, and while we did not agree with every conclusion, it was generally a fair and balanced assessment of The World Bank Group’s support to Russia during the first 10 years of its post-Soviet existence.
We knew, of course, that not everything went as planned and that some blunders shaped the early years. (And then there was the financial and economic crisis in 1998!) Still, when we read the news the day after it was released, it was dominated by the headline from the Congressional press release, which read, “Aid to Russia worse than wasted.”
What advice would you give to those new to the field?
In a nutshell, always seek to ask the right question. The power of the evaluator is in asking questions others are afraid to ask, or perhaps are not asking because they are too close to their work to even question basic, established thinking. If I can paraphrase Abhijit Banerjee when he spoke at The World Bank about the book “Poor Economics” (written together with Esther Duflo), as an evaluator, always be on the guard against “ideology, ignorance, and inertia.”
Diversity - Diversity in Action: Making the Most of Your Evaluation 2014 Diversity Experience
Given that there is just under a month left until Evaluation 2014, and over 700 educational sessions to choose from, I am certain many of you have been carefully building your agenda (using the nifty agenda builder tool, of course) to ensure you attend all of the must-see presenters and presentations. I implore you all to take the time to check out the wonderful sessions that highlight AEA’s diversity initiatives. The following are just a few exciting items you should consider adding to your agenda for Evaluation 2014.
1. GEDI
The Graduate Education Diversity Internship (GEDI) recently commenced its 11th cohort with new scholars Natalia Woolley, Iliana Perez, Kevin Lee, Justina Grayman, Kristin Mendoza, Erica Roberts, Kisha Woods, Danielle Cummings, and Amber Williams joining the program under the direction of program directors Dr. Ashaki Jackson, Dr. John LaVelle, and Dr. Stewart Donaldson. Already fully immersed in the program, the nine scholars have successfully completed two of their program meetings, the CGU Profession Development Workshop Series, and, most recently, the second annual CREA conference in Chicago. The cohort will soon be on their way to Evaluation 2014 in Denver, Colorado, where they will begin their first major program evaluation of the academic year.
“Sustaining a Culturally Competent Workforce through Early Mentorship: Reflections on the AEA GEDI Program” will bring program directors Dr. Ashaki Jackson and Dr. John Lavelle together once again with “anniversary” cohort (2013-2014) scholars Anael Ngando and Bailey Murph to discuss lessons from their yearlong program, including applying, creating values about, and examining strategies that incorporate cultural competence in evaluation. Join them Thursday, October 16, 2014 at 2:00 p.m. in Mineral D for this rousing panel from these future leaders in evaluation.
2. MSI
This is a big year for the Minority Serving Institution (MSI) fellowship program. This year’s cohort has been hard at work elevating the MSI experience for future participants, unveiling a brand new MSI FAQ and eager to serve as advisory members charged with mentoring prospective fellows and reviewing incoming fellowship applications. AEA is proud to announce that we have collected over 40 impressive applications from faculty members across the U.S. eager to become MSI fellows. We are looking forward to introducing the 2014-2015 MSI cohort. Stay tuned!
Join Tamara Bertrand, Edilberto Raynes, Denise Gaither-Hardy, Ana Pinilla, and Andrea Guajardo on Friday, October 17, 2014 at 3:30 p.m. in Agate A for “Reflections on Program Evaluation Experiences of a New MSI Cohort.” Led by program director Art Hernandez, the cohort will share and reflect on their experiences during the fellowship year and submit their individual and group work that they have been preparing throughout the fellowship year.
3. AEA Student Travel Awards
Every year, the AEA hosts a student travel award competition to full-time students to offset the cost of attending the annual conference, offering three types of travel awards with varying award amounts. The Type III travel award applicants are challenged to write an essay identifying ways to increase the diversity of people entering the field of evaluation and/or ways to increase the cultural competencies of evaluators more generally. This year’s winners, Jameson Lopez and Natalie Cook, will participate in a dialogue at Evaluation 2014 focusing on increasing the diversity and cultural competence of the field.
Leading “From Experience to Solution: AEA Student Travel Support Winners’ Reflections on Cultivating Cultural Competence in the Field of Evaluation” on Thursday, October 16, 2014 in Marble is 2013 AEA Marcia Guttentag Promising New Evaluator Award winner and Chair of the Multiethnic Issues Topical Interest Group, Dominica McBride. Join Dominica, Jameson, and Natalie for a roundtable discussion in which they will share their unique experiences and potential solutions in a synergistic dialogue designed for action!
4. Cultural Competence Photo Booth
While dashing about the conference, be on the lookout for the Cultural Competence Working Group’s photo booth table. Join Kari Greene and Derrick Gervin of the Cultural Competence Working Group, and tell them how you will be responsive to culture in your evaluation work this year in a photo starring yourself and your response on a dry-erase board. Kari and Derrick will be positioned at the conference ready to take your photos for this media project designed to stimulate the evaluator’s capacity to be culturally responsive. Take a look at this fantastic video from Evaluation 2012 and the AEA Statement on Cultural Competence in Evaluation for inspiration.
These are just a few of the fantastic sessions celebrating and addressing diversity here at AEA and in the evaluation profession. Be sure to add these wonderful sessions to your conference agenda. Want to learn how to build your agenda? Simply log in and click here. Next, click “My Agenda” and start building away at your conference agenda to create the ultimate Evaluation 2014 experience. The presentations mentioned above may be found under the “General” track. For assistance with building your agenda, contact info@eval.org. We can’t wait to see you all in Denver, Colorado, in October!
Have a story to share for the AEA diversity column? Contact me at zgrays@eval.org to learn more on how to share your diversity in evaluation stories. I would love to share your story!
Policy Watch - Evaluation Policy Sessions to Attend at Denver Conference
The annual AEA conference, Evaluation 2014, will be held in Denver, Colorado, on October 15-18, 2014. There are several interesting sessions on evaluation policy, with opportunities for members to interact with presenters and provide input. Here are just a few of these sessions:
Evaluation Policy Task Force (EPTF) Update. Session #1205; October 16, 2014; 8:00 - 9:30 a.m.; Granite C. The AEA Board of Directors established the EPTF to enhance AEA's ability to identify and influence policies that have a broad effect on evaluation practice and to establish a framework and procedures toward this objective. Hear an update on the work and accomplishments of the EPTF, learn of recent broad events affecting its work, and provide your input on the EPTF’s plans.
Debating How to Change the Terms of the RCT Debate: Options for AEA and the Field of Evaluation. Session #356; October 18, 2014; 9:45 - 11:15 a.m.; Mineral E. Bill Trochim, Mel Mark, Michael Crowley, Stephanie Shipman, and I will address attempts to institutionalize “rigorous” evaluation, its benefits and risks, and the political dynamics shaping adoption – all as federal evaluation policy continues to evolve. The session will address what AEA can do to influence policy by assisting decision makers and stakeholders in better understanding evaluation, including technical approaches, study designs, determining questions, planning, budgeting, and utilizing evaluation results. I will cover EPTF work to improve linkages across methodological divides within the evaluation community, advance methodological pluralism, and engage federal decision makers.
Building New Partnerships to Promote the Use of Evaluation among Public Policymakers. Session #10011; October 17, 2014; 1:45 - 3:15 p.m.; Capitol 4. Rakesh Mohan, of the Idaho State Legislature Evaluation Office, notes that we will need to build new partnerships with policymakers and understand the nuances of the public policy process. These partnerships will help us better understand policymakers’ information needs and allow us to educate them about the value of evaluation in making informed policy and program decisions.
Evaluation Capacity Building and the Canadian Federal Evaluation Policy. Session #1635; October 16, 2014; 2:00 - 2:45 p.m.; Room 701. Explore the history and nature of evaluation policies regarding evaluation capacity in the Canadian federal context. The senior director of the Treasury Board of Canada's Centre of Excellence for Evaluation will describe how the most recent evaluation policy has had an influence on evaluation capacity building, and the range of monitoring and reporting activities undertaken.
Introduction to Evaluation and Public Policy. Session #7019, October 16, 2014; 4:45 - 6:15 p.m., Quartz. This session focuses on how evaluation can affect public policy, rather than on evaluation policy. In the session, George Grob, the EPTF cochair, will explain how the public policy process works, guiding evaluators through the maze of policy processes, such as legislation, regulations, administrative procedures, budgets, reorganizations, and goal setting. George will provide practical advice on how evaluators can identify the policymakers they need to influence, how to contact them, and how to get their evaluations noticed and used in the public arena.
If you have any comments, suggestions, or questions regarding developing evaluation policy, contact me at EvaluationPolicy@eval.org. We look forward to hearing from you.
Potent Presentations Initiative - Audience Engagement Strategies
AEA rock star Sheila Robinson has pulled together more than 20 strategies for engaging audiences during your conference presentation. From the easy (eye contact!) to the more involved (get up and move!), you can browse her collection and select the right engagement strategies for your presentation. Yes, yours! Even if you are delivering a 15-minute paper session, you should plan on including some audience engagement.
The following are a few strategies that are easy to implement and are especially appropriate for paper presentations, think tanks, and workshops.
Rhetorical questions
Asking participants to think about the answer to a question even if they do not have the opportunity to answer aloud can keep them engaged. Ask questions that are relevant to them and make them think. Ask them to consider the “what ifs” (e.g., What would you have done? How would you feel if …? Have you ever …? How would you go about …? Have you ever considered …?).
Quick tip: Sprinkle questions in where appropriate throughout the presentation, but don’t overdo it!
Turn and talk
Posing a question and asking audience members to turn and talk with a nearby participant allows for all to participate simultaneously and invites every audience member to become part of the presentation. Use this strategy as a warm-up activity as you introduce your topic, after a long segment of a presentation to help audience members process what they have just heard, as a way for them to discuss ideas and learn from each other, or as a closing activity to review key learnings.
Quick tip: Ask audience members to turn and talk for one to three minutes. Plan your strategy to regain their attention. Without a microphone, this can be challenging. Use a timer that is loud enough to be heard over conversations, or ask your audience to look for your raised hand as a signal to bring their conversations to a close.
Variation: Table talk: Ask audience members to talk in groups of three to six if they are seated at tables. If there are more people who need to share ideas, give this a bit more time. A general rule of thumb is about one minute per participant.
The full 20-plus strategies will be released in a workbook and posted to the p2i site soon. You can also catch Sheila’s Coffee Break on this topic on Thursday, October 9.
Book Profile – “Community Psychology: Foundations for Practice”
Victoria C. Scott and Susan M. Wolfe are co-editors of “Community Psychology: Foundations for Practice,” a new book to be published by SAGE Publications in December 2014.
From the Publisher’s Site:
Drawing upon the wisdom of experts in the field, this reader-friendly volume explores both foundational competencies and the technical how-to skills needed for engaging in community psychology practice. Each chapter explores a core competency and its application in preventing or amending community problems and issues. With case examples throughout, this text offers a practical introduction to community outreach and intervention in community psychology.
From the Editors:
“Community Psychology: Foundations for Practice” is relevant to evaluators working across a variety of settings from any discipline. This book includes chapters on foundational competencies relevant to evaluation, such as understanding ecological systems, cross-cultural competency, and professional judgment and ethics. Many of the skills-based competencies are also frequently used by evaluators, such as empowerment evaluation, community needs and resource assessments, organizational and community capacity building, working with evidence-based interventions, collaborative community partnerships, advocacy, and dissemination and sustainability. Each chapter is written with clear definitions and descriptions of the competency, examples of how the competency looks in action, information describing how to develop the competency, resources, and discussion questions. It is useful as a reference book, professional development resource, or as a textbook.
About the Editors:
Victoria C. Scott, Ph.D., MBA, is a community psychologist who has devoted her professional career to working with nonprofit organizations to optimize their performance through consultation, training, research, and evaluation. Dr. Scott is especially passionate about improving the quality of healthcare and healthcare outcomes. She holds an academic appointment at the University of South Carolina, where she is both a clinical assistant professor in the Department of Neuropsychiatry and Behavior Science and director of research and evaluation at the Office of Continuous Professional Development and Strategic Affairs.
Susan M. Wolfe, Ph.D, is a community and developmental psychologist with over 28 years of professional experience. She has worked across a variety of settings, including public hospitals, a community college district, a public school system, universities, research institutes, and the federal government. She has worked across topic areas such as domestic violence, homelessness, education, adolescent development, maternal child health, technological innovation, children’s mental health, nursing homes, and policy. She is an active member of the American Evaluation Association, most recently as program co-chair for the Community Psychology TIG. She is CEO of Susan Wolfe and Associates, LLC, where she provides research, evaluation, capacity building, and coalition development services to nonprofit organizations, government, foundations, school districts, and public health organizations. She is also an adjunct assistant professor at the University of North Texas Health Science Center.