Date: Friday, February 27, 2015
Message from the President - AEA Seeks Exemplary Conference Proposals
Dear friends and colleagues,
I am so pleased to report that proposals are flowing in for our next AEA conference, Evaluation 2015, which will be held in Chicago on November 9-14, 2015. We are sending the call for proposals to evaluation colleagues across the U.S. and the world in hopes of receiving a diverse and extraordinary collection of proposals to select from this year. We would love your assistance with getting the word out in the next couple of weeks. Please consider sharing the call for proposals with colleagues engaged in evaluation work who might not typically attend AEA conferences, know much about AEA, or might not be receiving our communications. We think this is the perfect year to reach out and broaden our community as we host what promises to be an inspiring, multicultural, and extraordinary evaluation conference. The deadline for submitting proposals is March 16, 2015, so please act fast so our guests have plenty of time to prepare their proposals. Learn more about the AEA call for proposals here.
During Evaluation 2015, we will explore and learn from what has gone well in evaluation. We hope to uncover evaluation’s major achievements, assets, unexplored potentials, innovations, strengths, opportunities, high point moments, and visions of valued and possible futures. We are inviting evaluators, evaluation scholars, students, evaluation sponsors, and evaluation users from around the world to teach us about these successes. This invigorating discussion will take place engaging the theme of Exemplary Evaluations in a Multicultural World: Learning from Evaluation’s Successes Around the Globe.
As we listen to each other’s exceptional experiences, we will listen for themes of success – how were these evaluation exemplars managed? How did evaluators engage with stakeholders? How were evaluation conclusions developed, what lent them credibility, and what led to evaluation use? We think there is much to learn from examples of high-quality, ethically defensible, culturally responsive evaluation practices that have clearly contributed to decision-making processes, program improvement, policy formulation, effective and humane organizations, and ideally to the enhancement of the public good. We believe AEA faces an extraordinary opportunity to have these discussions with the Global Evaluation Community during 2015 – The International Year of Evaluation.
Click here to watch a video about preparing your proposal(s) for AEA 2015.
Warm wishes,
Stewart Donaldson
AEA President
Message from the Executive Director - Our Journey: A Look Back and Ahead
“The journey of a thousand miles begins with one step.” – Lao Tzu
When we look to measure our progress on a project, we often look ahead to all of the work that still needs to be done. But there is value in considering all that has been accomplished, too.
It has been my pleasure to serve as your executive director and as the leader of the AEA administrative management team over the 18 months of our journey together. During this time our team has learned much about AEA in general. More importantly, we have begun strengthening relationships with many individual AEA members. In order to keep you posted on our progress and plans for the future, I will be writing quarterly articles for the AEA Newsletter this year. I welcome your feedback and input on topics you would like me to explore in more detail in these articles.
The Evaluation 2015 conference is an important cornerstone to AEA’s value, and we are working very hard to make sure it is a valuable experience for everyone. The attendee survey results for the Evaluation 2014 conference yielded approximately 700 responses (23%). They provided good feedback on areas that went well and areas that need improvement. Here are a few quotes gathered from the post-conference survey:
Here are a few comments that include suggestions:
With the 2014 conference being the first AEA conference entirely under our management, we agree that improvements can be made.
To help us make improvements, we worked with TIG leaders, collecting comments from more than 50 individuals who gave us several dozen suggestions on improvements to the online session submission platform. Many of these suggestions were incorporated into the release of the platform in January. (Don’t forget: The deadline for submission is March 16, 2015!). A team of staff and members of the Conference Advisory Group will review the suggested changes for the mobile app and enhancements to the onsite guide for better alignment and improved features for navigating the conference.
Opportunities Ahead for Involvement in AEA Activities
Recently, you should have received an email from the AEA staff regarding opportunities to volunteer with one of the many Working Groups that the staff team here at the AEA headquarters have formed to assist in the development of specific programs. I encourage you to think about volunteering. Working Groups are a great way to engage in meaningful work with your colleagues. They also help the staff team to get better acquainted with evaluation, evaluators, and the values that drive this organization. With more than 20 active Working Groups, chances are good that you will find a project and group that interests you. I have deep appreciation for those members willing to give their time to help the organization, and to them I say thank you!
Another way to get involved (as many of you already know) is through the Topical Interest Groups. However, we recently learned while analyzing membership data that more than one-third of AEA members are not engaged in TIGs at all. TIGs offer a terrific opportunity to learn more about areas of evaluation that interest you and expand your professional network. There are now over 50 TIGs, with four new ones added just in 2014. If you did not join a TIG when joining AEA, you can simply log into the AEA website, go to the “Members Only” tab and click on “Update TIG Selections.” The AEA staff team is available to answer any questions you have about TIGs.
After 18 months, we have made significant progress and, admittedly, have had a few missteps. We are charging forward by doing the work that needs to be done to continue strengthening this wonderful association and the field of evaluation. In the next issue, I will explore the link between the AEA Strategic Plan (currently being refined) and the management plans that are informed by the Strategic Plan.
The AEA management team is eager to assist you with any questions you might have. You can find a list of all the team members here. It has been an honor and a pleasure serving you over the past 18 months. I look forward to a great 2015 with you!
Best wishes,
Denise Roosendaal
AEA Executive Director
Diversity - AEA Continues Its Work on Cultural Competence
In this month’s Diversity column, Osman Ozturgut will update you on some of things the Cultural Competency Working Group has been working on.
The statement on cultural competence in evaluation was drafted by the Cultural Competence in Evaluation Task Force of the American Evaluation Association’s Diversity Committee, reviewed by the AEA Board of Directors, approved by a vote of the full AEA membership, and disseminated to the public in 2011. Since then, a group of evaluation folks has gathered from around the country with the charge to disseminate the American Evaluation Association (AEA) Public Statement on Cultural Competence in Evaluation. The overall mission of the group is to (1) disseminate the Statement so that evaluators and people who commission evaluations understand the importance of cultural competence in evaluation and the essential concepts and practices that characterize culturally competent evaluations; and (2) translate the Statement into real change, both for evaluation practice and for those communities that we serve. The group’s goals are to (1) increase awareness about the Statement and the resources associated with it (i.e., increase the number of people who know that the statement exists, know how to access it, and who have read it, and increase the number of people who know that there are resources associated with the statement on the AEA website); and (2) increase the use and application of the concepts and essential practices described in the Statement, regardless of the type of evaluation (process, outcome, impact, cost, personnel, product), the setting in which the evaluation is engaged (government, academia, business, community, education, etc.), or the form of evaluation (policy, practice, teaching, etc.) conducted.
Our Working Group members span academic, evaluation practice, government, and nonprofit management settings. Our members have been tasked with various responsibilities that focus on policy, teaching, resources, and the evaluation of the Working Group activities. Our subgroups and members have actively contributed to the Annual Conference and the Summer Institute through sessions and workshops and to the broader evaluation community through sponsoring aea365 posts.
Our Policy Subgroup was formed to influence funders and sponsors and their grantees in the awareness of the importance of cultural competency in their work. We are working with national evaluation leaders to organize a virtual meeting focused on essential practices of cultural competence in evaluation policy. The small-group discussion will highlight how concepts presented in the Statement can be applied to evaluation at the federal level. Using two federal agencies as case studies, the meeting participants will identify opportunities to integrate cultural competence in evaluation policies.
Our Teaching Subgroup was formed to disseminate the Statement for teaching and training purposes. We are developing a video to highlight and increase awareness of the Statement and to communicate the key and essential points of the Statement and of cultural competence in evaluation. The video will be used in a variety of settings. We intend to show this video at Evaluation 2015, to be held November 9-14, 2015, as a way to highlight the importance of the AEA’s Public Statement on Cultural Competence in Evaluation. This video will be the first piece of a longer course on cultural competence in evaluation.
Our Resource Subgroup is working to update the resources available on the AEA website about cultural competency in evaluation. These resources will include multiple modalities, such as written, audio, and visual. Soon, you will have access to our updated site for the resources.
And, our Evaluation Subgroup developed an evaluation plan to evaluate our dissemination activities and outcomes. Evaluating our work seemed natural!
AEA is incredibly excited about the ongoing projects of the Cultural Competence Working Group. To learn more about their work and how to get involved, contact Cindy Crusto at cindy.crusto@yale.edu.
Osman Ozturgut is an assistant professor of doctoral studies at the University of the Incarnate Word in San Antonio, Texas, and an active member of the AEA Public Statement on Cultural Competence in Evaluation Dissemination Working Group.
Policy Watch - The President's 2016 Budget Evaluation Guidance
From Cheryl Oros, Consultant to the Evaluation Policy Task Force (EPTF)
Every year at about this time the president sends Congress a proposed budget for the next fiscal year. The president’s 2016 budget contains ongoing efforts to better integrate evidence and evaluation into federal budget, management, operational, and policy decisions and to increase access to federal data for such efforts. These include:
1) Building agency evaluation capacity and developing tools to better communicate what works: The Office of Government-Wide Policy will be standing up its Office of Evidence and Analysis this year to help agencies make data-driven decisions. Under the president's proposal, that office would expand to 10 employees, with a budget of $4.4 million. The new Evidence and Evaluation webpage can be found at www.whitehouse.gov/omb/evidence. For highlights of the Evidence and Evaluation Agenda, see the budget fact sheet, Building and Using Evidence to Improve Results, and the “A Government of the Future” chapter in the main volume of the FY 2016 budget.
2) Promoting the use of high-quality, low-cost evaluations and rapid, iterative experimentation: The budget supports the expansion of the White House Social and Behavioral Sciences Team (SBST). SBST is already helping over a dozen federal agencies test the impact of behaviorally informed interventions on program impact and efficiency using rapid, rigorous, and low-cost randomized control trials.
3) Adopting more evidence-based structures for grant programs: For highlights, see the fact sheet “Improving Outcomes through Pay for Success.”
4) Making better use of already-collected data within government agencies: The “Building Evidence with Administrative Data” chapter of Budget Analytical Perspectives places a special focus on a wide range of efforts to make administrative data legally and practically available for policy development, program evaluation, performance measurement, and accountability and transparency efforts.
Along with specific agenda items, the budget embraces the creation of a legislative commission proposed by Sen. Patty Murray and Rep. Paul Ryan to examine ways to use government data to improve federal operations and support private research and industry. AEA has endorsed the main features of this proposed legislation while recommending revisions that would encourage the selection of evaluation methods based on the evaluation questions posed and circumstances, rather than an emphasis on any one method in all situations.
5) Conducting Strategic Reviews: Agencies are publishing results of strategic reviews for the first time on www.performance.gov and also in FY 2014 Annual Performance Reports. There is a government-wide summary of these results in the Analytical Perspectives Chapter of the President’s budget, called “Building a High-Performance Government,” at http://www.whitehouse.gov/omb/budget/Analytical_Perspectives.
6) Funding Cross-Agency Priority Goals: Agency leaders have set cross-agency goals for such issues as improving access to capital to enhance job creation, reducing foodborne illness through targeted inspections, coordinating multiple agency services to reduce veteran homelessness, and reducing hospital-acquired infections. Data-driven reviews are held by high-level leaders at least quarterly to remove barriers and accelerate results. Progress on goals is updated quarterly on www.performance.gov. The FY 2016 budget proposes a transfer of up to $15 million from agency budgets to support such progress in order to institutionalize a capability to fund cross-agency efforts, rather than handling these challenges on a case-by-case basis. Part of this funding would support the White House Leadership Development Program, through which emerging leaders will participate in full-time rotational assignments for one year, with responsibility for driving progress on the goals.
For those of you working in or with a federal agency, we are interested in hearing from you about related evaluation and data efforts, including those stimulated by earlier OMB guidance. To send me your thoughts or set up a discussion, please contact me at EvaluationPolicy@eval.org.
Potent Presentations Initiative - Potent Presentations Evaluation
From Stephanie Evergreen, Potent Presentations Initiative Coordinator
Few things are more fitting for AEA than evaluators evaluating. We asked you to evaluate the Potent Presentations Initiative, as part of the post-conference survey. While the survey still suffers a bit from low response rates (ahem), we can see some happy growth for p2i.
This year, a higher percentage of conference attendees said they had heard about the Potent Presentations Initiative and a higher percentage reported using the p2i materials in some way.
In what ways? Good question! The most popular uses were to prepare materials for the AEA annual conference (whew!) but nearly as many said they were using p2i to prepare presentations outside of AEA. That’s great! We hope you can rely on AEA for professional development that benefits your work as an evaluator. Even more interesting: Nearly half of those who said they use p2i materials reported using them to train other people on strong presentation development. Perfect!
It’s easy to use Potent Presentations materials to train others. I suggest you sit your team down for our three-part webinar series on message, design, and delivery. The videos are available on our homepage.
You can deliver the training yourself by downloading the slide decks and customizing the materials to better fit your group. The slides even contain our speaking notes. Access the PowerPoint files on our Tools page. All p2i materials are licensed under the Creative Commons Attribution, Noncommercial, Share Alike license, which means you are free to use our materials to train others as long as you give us credit, you don’t make money off of the training, and you also post your materials online for others to use.
Thank you! Thank you for helping to spread the word about Potent Presentations and for making the evaluation conference an ever-more-enjoyable place to teach and learn.
International Policy Update - AEA Completing Its First Collaboration with Central Asian Evaluators
From Mike Hendricks, AEA Representative to the International Organization for Cooperation in Evaluation (IOCE), with contributions from Jim Rugh, EvalPartners Co-Coordinator
Next month sees the completion of a unique, first-time collaboration between AEA, EvalPartners, and evaluation associations in three Central Asian countries: Kyrgyzstan, Kazakhstan, and Tajikistan. Also unique is that the lead entity for AEA is not our central organization but our local affiliate in Washington, DC: Washington Evaluators (WE).
This collaboration has two phases. In phase one, funded for $10,000 by EvalPartners through its Peer-to-Peer (P2P) support program, Dr. Donna Mertens, former president of AEA, visited Bishkek, Kyrgyzstan, in December 2014 to deliver two workshops. The first was a three-day workshop to help evaluators from four countries (Russia also) develop their personal capacities in transformative mixed methods addressing gender- and equity-focused evaluations, an area in which Dr. Mertens is an expert. Almost 40 persons from civil society, government, and the private sector attended this workshop, which was developed only after conducting a needs assessment of participants.
Dr. Mertens also offered a second, one-day workshop in Bishkek to help build networks among the evaluation associations in those four countries and beyond. For each workshop, her remarks were simultaneously translated into Russian for the benefit of participants. After each workshop, participants lighted the “evaluation torch” being used during EvalYear 2015 and carried the torch back to their own countries. You can find the final report and photos of these workshops here (click on the "WE Final Narrative" Report document).
Phase two of this collaboration between WE and colleagues in those CIS countries is being funded for $5,000 by AEA through its International Partnership Protocol Program (IPPP). In fact, this is the very first effort being funded through this new AEA program. Four evaluation leaders from Tajikistan and Kyrgyzstan will visit Washington, DC, in March 2015 to make presentations to WE members on the state of evaluation in Central Asia. While in Washington, these leaders will also meet with WE officials and representatives of other evaluation-related institutions in the Washington, DC, area to share their experiences, seek advice, and discuss possible future cooperation.
This collaborative project is important for several reasons:
If this collaboration interests you, please read the "AEA Launches International Webpage" article in this newsletter, which discusses AEA’s call for proposals for the next round of the IPPP program. Note especially that the deadline for applying is March 31.
In other international news, India became the first country to celebrate EvalYear 2015, hosting a week-long series of events from January 19-23 titled “Evaluations for Good Governance.” Mr. Rao Inderjit Singh, the Honorable Minister of State for Planning, presided over the week, which had been organized by several national and international organizations, including UN Women, International Initiative for Impact Evaluation (3ie), Oxfam, UNICEF, CARE, ISST, and CMS. Photos of the India events can be seen here.