Date: Friday, October 26, 2018
What a great time to be a member of AEA! This week, evaluators from around the world are assembled in Cleveland, Ohio, at AEA’s 32nd Annual conference. As we convene to learn and share, members are utilizing this event as an opportunity to establish a community with colleagues that will promote continual development and networking.
The annual conference is also a time to celebrate our accomplishments as an association, some of which include:
The conference also gives members the opportunity to participate in the Annual Business Meeting, where they can receive updates from AEA leaders about strategic initiatives and most important, share their voice to inform planning of the association.
Unfortunately, not everyone is able to attend the conference due to budgetary and timing constraints. However, we are continually working to provide additional opportunities for members to have a voice as we plan strategic initiatives that will affect the future of AEA.
If you are not able to attend the Annual Business Meeting, please join us in November for our Virtual Town Hall Meeting (date and time TBA), which will focus on strategic initiatives for the upcoming year.
Below are some key initiatives for 2019:
Overall, AEA is doing very well! We have a healthy investment reserve, and our membership numbers are steady. As we continue to grow and increase programs and initiatives, we must ensure that our revenue streams support our growth and the work of the association. As we plan on ways to accommodate rising costs, we want to do so informed by ideas for increasing member value. Please be sure to attend our November Town Hall Meeting so that you can weigh in on these important issues.
Many thanks to you for choosing to be a member of AEA. It is through your dedicated support that we continue to grow and champion the profession of evaluation!
The Face of AEA spotlights our members and their backstories - why they joined the profession, what drives them and memorable lessons they've learned along the way. Know someone who should be featured? Email the AEA editor, Kristin Fields, at kfields@eval.org.
Name: Chantel DePaepe Affiliation: American Association of Nurse Practitioners (AANP) Degrees: MPH Years in the Evaluation Field: 2 Joined AEA: 2018
I belong to AEA to improve my evaluation skills and to share my knowledge with others.
The most meaningful evaluation that I have been a part of was for the Transforming Clinical Practices Initiative (TCPI). AANP worked with the National Nurse-Led Care Consortium under a grant from the Centers for Medicare and Medicaid Services (CMS) as part of the TCPI. I helped to evaluate a small part of the program but I learned a tremendous amount about planning, implementing and evaluating a nationwide program, as well as the importance of connecting everything we did back to our goals.
It is easy to go off track from your goals. We had stakeholders present wonderful ideas, but their ideas did not align with our goals. Another issue was stakeholders did not have an evaluation plan in mind for their ideas. Therefore, I recommend that throughout every step of the program you keep reminding yourself and your team about what you are trying to accomplish and how you will evaluate what you implement.
From Anne Vo, Jessica Harlan, Patrick VBarlow and Asma Ali, HPEER TIG leads
Approved by the AEA Board in January 2018, the mission of the Health Professions Education Evaluation and Research (HPEER) Topical Interest Group (TIG) is to develop a network of program evaluators working across different areas of health professions education (HPE) to promote greater evaluation quality, relevance and use. HPE is concerned with the training of healthcare professionals across fields such as allied health, dentistry, medicine, nursing, pharmacy and veterinary medicine. These professions share regulatory, cultural and educational objectives that influence the education delivery life cycle, from pre-health professional training through continuing health professional education. Furthermore, HPE programs occur in a variety of learning environments, including traditional classrooms, labs, clinics, hospitals and other field-based settings. Regulatory and attainment of educational objectives also influence the ways in which HPE is delivered and evaluated. These increasingly complex and changing contexts and environments, in turn, contribute to the diversity of the evaluator community within HPE, as well as shape HPE practice, outcomes and evaluation.
The TIG aims to provide ongoing support of HPEER evaluators by supporting developing partnerships, scholarly opportunities and discourse among practitioners in the HPE field. The TIG also strives to promote inter-professional approaches, identification of best practices in HPE and provide opportunities for partnerships. Toward this end, the HPEER TIG will focus on the following priorities:
Interested in getting involved? The HPEER TIG will have its first TIG Business Meeting at Evaluation 2018 in Cleveland. The meeting will take place on Thursday, November 1 from 7:30 PM – 9:00 PM in Room CC-25A. If you are not able to make this meeting, please reach out to a member of the HPEER TIG leadership team to learn more.
Chair: Jessica M. Harlan, PhD The Johns Hopkins University School of Medicine E: jessica.harlan@jhmi.edu
Co-Chair: Asma M. Ali, PhD AA & Associates, LLC Educational Development Center [EDC] E: asma@aa-associates.org
Co-Program Chair: Anne Vo, PhD Keck School of Medicine of USC University of Southern California E: annevo@usc.edu
Co-Program Chair: Patrick Barlow, PhD Carver College of Medicine The University of Iowa E: patrick-barlow@uiowa.edu
Do you lead or participate in one of AEA's Topical Interest Groups (TIGs)? We want to hear from you and spotlight your work. Send an email to the AEA editor, Kristin Fields (kfields@eval.org) to share news, updates and articles for consideration in an upcoming AEA newsletter.
From Nick Hart, chair of the Evaluation Policy Task Force (EPTF)
The last year has been a very active one for evaluation policy in the United States. During the fall conference, Evaluation 2018, AEA’s Evaluation Policy Task Force (EPTF) is sponsoring several opportunities for members to learn about major activities and opportunities to provide feedback. Here are a few of the highlights for the conference:
Future opportunities will also be available to learn more about the EPTF and opportunities for feedback. AEA members can also always submit suggestions through AEA’s Issues and Ideas Portal.
From Sheila B. Robinson, Potent Presentations Initiative Coordinator
Evaluation 2018 is in full swing! As you attend the conference in Cleveland, or perhaps attend virtually from anywhere else in the world, consider making it a priority to study the presenters whose sessions you choose. While it can be challenging to focus on the presenter’s message, design, and delivery all at the same time, you can employ simple strategies to capture your key takeaways.
Good Old-Fashioned Note-Taking
Consider how you track your learning from a typical conference session – Do you take notes? If so, do you use your phone, a tablet, laptop, or paper and pen?
Depending on the location, the furniture setup and my mood, I use a laptop or a paper notebook. I use one of three configurations during any given session to capture my notes on the session content, and on the presenter characteristics. These configurations are:
I’ve learned to toggle between my two pages quite easily and it generally doesn’t detract from my experience. If going back and forth is problematic, format just one page with two columns.
In my “presenter characteristics” notes, I capture how the presenter successfully:
Of course, I do have some notes on what goes wrong in presentations – my “what not to do” list – but I tend to focus more on what presenters do that I want to emulate and incorporate into my own practice.
Organize, Review, Reflect
At a later time, organize your “presenter characteristics” notes in one place. If you decide to use paper and pen, you can scan or retype these handwritten notes (this works as an effective review/reflection strategy for me) and store in an electronic folder.
Every now and then, or when you’re preparing for a presentation, review your notes to look for patterns – what do you tend to pay attention to? How can you incorporate those strategies into your own presentations?
Pro Tip
If you can manage your attention and not lose out on the content of the presentation, take note of how others in the audience are reacting to the presenter. What makes them laugh? When are they more attentive? When do they tune out? Are their reactions to the presentation and presenter aligned with yours?
What Do the Rock Stars Do?
In 3 Traits of the World's Most Powerful Speakers author Danny Iny identifies three commonalities from three rock star presenters – Tony Robbins, Mitch Joel, and Nancy Duarte – and offers brief insights from each.
The article concludes with video clips of each presenter in action, a great opportunity to practice taking notes on presenter characteristics. And if not these videos, try note-taking on some of your favorite TED talks.
Have you successfully used p2i tools or p2i principles in your presentations? Do you have “before” and “after” slide examples you would be willing to share? Do you have ideas for, or are you interested in writing a blog article on Potent Presentations? Do you have an interest in sharing your tips for Potent Presentations through a brief video or webinar?
Please contact me at p2i@eval.org and let’s talk! I’m happy to help, offer guidance, or collaborate on any of these.
In early October, the European Evaluation Society (EES) held its 13th biennial conference in Thessaloniki, Greece. The theme of the conference was “evaluation for more resilient societies” – which the organizers noted was particularly apt given the magnitude of political, economic, social and cultural turbulence experienced in Europe, and globally, over the last decade. In the face of climate change, migrant crises, and the rise of nationalism, the European evaluation community is considering its role in advancing a more just society.
Of particular interest to those working on international evaluation were several sessions – including a plenary – on the Organization for Economic Co-operation and Development’s (OECD) Criteria for Evaluating Development Assistance. As discussed in the August 2018 newsletter, a movement is underway to revisit and evolve the criteria – and this movement was palpable at the EES conference.
The push for this is fairly straightforward: There’s recognition that the existing Criteria, though helpful, were created by a small number of powerful stakeholders and designed primarily to guide evaluation of projects or programs. Today, evaluation of complex interventions that take place within complex systems must be inclusive, culturally and contextually-sensitive, and complexity-aware. The conversations at EES were an important reminder that the next iteration of the DAC Criteria will need to be developed with input from a broad cross-section of actors, and will need to be responsive to new development paradigms if they are to remain relevant and influential in the future.
For more information on the conference or EES, visit https://www.europeanevaluation.org/.
Each year, AEA provides a number of awards to evaluators living and working in developing countries to subsidize their participation in the association’s annual conference. This year, the International and Cross Cultural Evaluation (ICCE) Topical Interest Group (TIG) received over 70 applications for just eight awards. We’re proud to share that the awards were distributed equitably – to four female and four male awardees, representing seven countries: Benin, India, Kenya, Nepal, Peru, the Philippines and Tanzania.
Each application was reviewed by a minimum of three individuals. This year we were very fortunate to have 30 volunteer reviewers, and we’re starting to collect names for next year. If you are interested in supporting the 2019 travel award application review process, please reach out to Shawna Hoffman at SAHoffman@rockfound.org.
Finally, the money raised at the annual silent auction goes directly to support the international travel awards. If you will be in Cleveland at Evaluation 2018 this year, be sure to come out on Friday night to bid on some great books and other items. For more information about the silent auction, please contact Hubert Paulmer at hubertpaulmer@gmail.com.
From the AEA Board of Directors
Did you miss the AEA Town Hall meetings? You can now access them on AEA’s website here. Be sure to watch the most recent conversation featuring Jean King and AEA Board member Susan Tucker discussing AEA’s Evaluator Competencies.
From the AEA Education Team
The Digital Knowledge Hub is an online platform featuring professional development opportunities for evaluators, by evaluators. View past eStudies, like this one:
eStudy 091: Designing Useful Surveys
Surveys for program evaluation, performance measurement, or needs assessment can provide excellent information for evaluators. However, designing effective surveys requires an eye for both unbiased question design and the best methods for data administration. Michelle Kobayashi shares guidelines and methods for survey development that will increase response rates and create reliable and valid questionnaires.
Presented by Kanti Gopal Kovvali, Author, Organizational Unlearning Specialist, Visiting Faculty TISS and NMIMS
The eStudy will expose participants to a radically different evaluation process that is rapid, transformational and sustainable. The eStudy will help participants to change their paradigms about evaluation, introduce new methodologies and tools and transfrom their orientation from doing good research to facilitating transformational research.
eStudy 095: Using the Cultural Consensus Method to Evaluate Program Impacts | November 27, 29, 12-1:30 p.m. EDT
Presented by Peggy Ochandarena, Chief of Party- Enhancing Palestinian Justice Program, Chemonics International Inc; Co-Director, Global Impact Collaboratory and Roseanne Schuster, Assistant Research Scientist, School of Human Evolution and Social Change, Arizona State University; Director of MEL, Global Impact Collaboratory.
Culture is the shared beliefs of a particular group of people, and strongly shapes what is socially acceptable and thus shapes action. Understanding the culture of an intervention's beneficiaries is critically important in designing interventions for effectiveness. In this webinar, leaders of the Global Impact Collaboratory, a partnership between Arizona State University and Chemonics International, give learners hands-on interaction with the theory, instrument design, and analysis for the CCM, with demonstration of its use in international development projects and application to a case study.