Friday, March 31, 2017
Message from the President - Fostering Member Engagement
From Kathy Newcomer, AEA President
My warmest wishes from downtown Washington, where every day there are yet more surprises and twists that one could never dream up! On the positive side, the rewarding rise in activism is heartwarming to me! I continue to be an optimist, as I believe that program evaluation presents a tool to help society learn about itself and about how to act in the public interest.
Recent political actions and discourse around immigration to the United States have posed challenges to the global nature of our profession and of our AEA community. Events have only made me more proud that our organization has established core values of inclusiveness, multiculturalism, and diversity. Evaluation scholars, practitioners, and students hail from all over the world, united in their objective of using evaluation to make the world better for all. Current events do not change the nature of who we are as an association of global evaluation professionals.
My focus this year is on promoting diversity, inclusion, and evidence. In line with these three mantras, I am working to enhance engagement with our members, the broader evaluation community, and current and potential consumers of evaluation! I am happy to report progress!
Diversity
In collaboration with Washington Evaluators, on January 30, we hosted our first Dialogue about the Role of Race and Class in evaluation work and impact. We had more than 80 evaluators in the audience and more than that online. Thanks to the leadership of Melvin Hall and the instructive dialogue among the panelists, it was a very rewarding experience! Our second Dialogue will be held in collaboration with the American Educational Research Association (AERA) in San Antonio at the end of April, and the third with the Center for Culturally Responsive Evaluation and Assessment (CREA) in Chicago in September. We are live streaming all three 90-minute dialogues, and have a filmmaker creating a short montage to be shared during a plenary on the topic at our Annual Conference in November in DC.
We would like to offer our sincere gratitude to the W.K. Kellogg Foundation for their generous support of this project with a grant in the amount of $55,300.
Dialogue and activism regarding social justice is occurring across our profession! On March 2, I was delighted to attend the Milwaukee affiliate’s annual conference that focused on social justice and it was a resounding success — we are living our values across the country! And next month I will be joining another one of our affiliate's annual conferences, the Eastern Evaluation Research Society, as participants discuss "evaluation looking forward" including the role of inclusive approaches to evaluation.
Inclusion
I asked Melvin Hall and Robin Miller to lead a new task force to promote membership engagement for our association, and more specifically to help us to: intentionally develop and sustain communications and strategies to ensure evaluators and evaluation stakeholders from all backgrounds feel welcome and engaged. They recruited a fantastic team, and already held a two-day meeting in DC on March 3-4. They are developing strategies and tools to advance AEA’ s in making good on our commitment to inclusiveness, multiculturalism, and diversity.
Evidence
AEA’s Evaluation Policy Task Force (EPTF), of which I am a member, is working proactively to influence federal legislation, regulations, and the Commission on Evidence-Based Policymaking to ensure that data continue to be available to the public and to encourage that timely and relevant analyses are used to inform public policies. Our EPTF is working collaboratively with more than 20 other organizations to ensure that our voices are heard. We also are creating opportunities for AEA to be at the table to advocate for the wise use of evidence at key venues, including through discussions with the Bipartisan Policy Center.
And lastly, "From Learning to Action" is the theme of our Annual Conference, and in line with this theme, I challenge our members to come to share what we have learned about what works in advancing diversity, inclusion, and evidence to improve our public policies, programs, and our society!
And don't just bring your ideas to discuss — I strongly encourage all of our members to find ways to take action to promote evaluation and the value of evidence in our society. To help facilitate this, I am excited to announce the EvalAction initiative this fall during the conference, which encourages AEA members to speak with Members of Congress and their staff about the role evaluation can play in shaping and informing policy. Learn more about the EvalAction initiative co-sponsored by Washington Evaluators and EPTF, and sign-up to participate here.
My door and ears are always open and I welcome engaging with our members to learn how AEA can better serve us and promote evaluation practice and evidence-informed policy and practice! We have the capacity and skills to help society learn from evaluation! Please help me foster membership engagement — contact me at newcomer@gwu.edu or AEA2017president@aea.org.
Diversity — Intentional Inclusivity at its Best: ¡Milwaukee Evaluation! Inc. 2017 Social Justice & Evaluation Conference
From Nicole Robinson, MPH/MSW
On March 2, 2017: We nailed it! We held our 3rd Social Justice & Evaluation Conference as the Wisconsin statewide affiliate of the American Evaluation Association. More than 130 evaluators, students, funders, and evaluation consumers joined us for the daylong event. And, we really did rock it!
We’re so enthusiastic because we did it with little to no money, volunteered time, and an abundance of good will and dedication. The political landscape in our country and in Wisconsin over the past decade (or more) have made social justice and inclusion an awkward experience (i.e., evaluation settings where good intentions are mixed with explicit and implicit bias, microaggressions, or blind spots and inequitable distribution of power).
Why are we celebrating?
We aimed for 50 percent racial/ethnic diversity, we got close to that: 48 percent with 62 people of color in attendance (our state is comprised of 12 percent people of color).
We had many firsts: first break-out session entirely in Spanish; Spanish-to-English translators (two!); six youth evaluators who volunteered and provided informal advice on how to nurture youth evaluators of color; and when we didn’t have any presenters that identified as Asian at the inaugural conference, we did this time — and the number of Asian attendees was its highest. We also had our first artist lead an art installation project to define social justice. See UC Berkeley’s great definition of social justice!
The keynote session by Dr. Debra Joy Perez focused on How to Stay Woke in the Resistance. We were joined by State Representative David Bowen, alongside AEA President Kathy Newcomer.
Our breakout sessions were bold. We asked several presenters to talk on a topic that would build the capacity of evaluators in multicultural settings. We had a session on how to handle –isms throughout the evaluation process and facilitate authentic meetings with this skill, another on how to mentor evaluators, including evaluators of color throughout their careers, and a session on culturally responsive evaluation from the evaluation consumer, a Latino-led nonprofit organization. See the full breakdown.
The ambiance embodied love. The lighting, the community center’s Black History artwork and architecture, the local food and the DJ’s tunes created an inspiring vibe. An impromptu moment: The keynote came on stage to her song! We purposely chose a space with gender neutral bathrooms and lactation rooms and encouraged parents with infants to come.
How did we achieve this?
Intentionality around our values. A few practices we can lift up:
The board (Nicole Robinson, Emily Connors, Kate Westaby, and Regina Lowery) is explicitly diverse as was our first-ever conference committee — Monique Liston (Chair, former GEDI), Lynne Morgan, Baily Murph (former GEDI), and Maria Beyer. We also approached it with a grassroots-style and hope that is never lost.
This committee was willing to ask a friend or family member to help create a meaningful conference, communicating that we’re all bringing something special together. This team did grunt work, the work that’s done in the shadows and they did it for a greater good.
There were opportunities for leadership, too. We love telling the story of how Monique Liston came to the first conference in 2013 as a volunteer and, after being exposed to evaluation for the first time, went on to add measurement studies to her doctoral program, became a GEDI, then presented at the second conference in 2015 on dignity evaluation while serving as the conference volunteer coordinator. This time she was the Conference Chair and is part of an evaluation firm, Derute Consulting Cooperative, which sponsored breakfast.
This is the kind of leadership growth we thrive on — it’s why as an affiliate we are working on a pipeline for evaluators of color. This is the pace of change and transformation that should be the norm in our state and the evaluation field.
Flavor, or sabor, was a “success factor” and social justice success indicators were created. We asked ourselves, how can we make sure this action, purchase or decision is reflective of our flavor and deep commitment to authentic social justice?
We tracked the diversity of our attendees and did intentional outreach to more than 1,000 individuals to tap into different networks. It’s so important to not just invite who you know or who’s in the know — also known as “the right people.” Our legitimacy as an organization putting on this event was directly tied to creating a communal space filled with meaningful moments.
We created a values-based budget that would honor and support local businesses, translators from a local cooperative, volunteer stipends and a DJ whose music transforms. We also valued our worth and charged more for the conference this year to keep us out of the red and our personal cookie jars, but didn’t want the price to stop anyone from attending so we bartered and traded when necessary.
We also balanced responsibility with the attendees stating the day was a gift for all of us, not from the affiliate to them. We asked attendees to take risks and experience sweet discomfort meeting new people or with being challenged. We did this at the first meeting of evaluators back in 2012 when we asked them to complete a survey comparing and contrasting the demographic makeup of the evaluation team with the population being evaluated. We don’t hold back.
Intentionality wasn’t easy.
Creating spaces that are unapologetically inclusive and intentional about values-based decision making takes more time (i.e., emails at 3:00 a.m.) and requires champions that are consciously adept or “woke,” to borrow a phrase from our keynote Dr. Debra Joy Perez, Chief Evaluation and Learning Officer, Gordon & Betty Moore Foundation. Many of us are still learning how to do that. Some decisions had to be reversed after we thought we’d figured everything out. The board provided more oversight than planned. Everyone involved behind the scenes was responsible for making it easy for someone on the team to challenge a decision or viewpoint based on values and inclusion.
We didn’t get everything right.
Although this article emphasizes race and ethnicity, we do define inclusion in other ways. While we had great diversity in race, gender, and age and met many new people, we wanted more representation from rural areas and the smaller cities and suburbs. Among the lineup of presenters, the nonprofit and government sectors were underrepresented and we could have benefited from the real talk of lay evaluators, youth, and evaluators who don’t have terminal degrees.
It was a good day.
As a result, two evaluators of color are interested in the board, one in particular is interested in becoming our first Inclusion Officer. Another attendee owns an evaluation firm and wants to discuss how she can pass on her business to an evaluator of color. This is exciting social change and not bad for a one-day event! As the affiliate moves into balancing budgets and other administrative tasks, it was nice to reflect on the positive and how we’ve serving as a catalyst for a movement toward intentional inclusivity in evaluation.
Photos courtesy of ¡Milwaukee Evaluation! Inc.'s Regina Lowery
AEA Dialogues on Race and Class in America
The full length recording of the January Race and Class Dialogue is now available on the AEA website. This complementary event was the first of three dialogues within a series called the AEA Dialogues on Race and Class in America. AEA partnered with the Trachtenberg School of Public Policy and Public Administration at George Washington University and the Washington Evaluators (a local AEA affiliate) for the event in Washington, DC. Melvin Hall, Northern Arizona University Professor, Former Member AEA Board of Directors, moderated discussion between panelists, including: Kristen Harper, Senior Policy Specialist, Child Trends; Nick Hart, President, Washington Evaluators; Kien Lee, Principal Associate/Vice President, Community Science; Guadalupe Pacheco, President/CEO, Pacheco Consulting Group, LLC; Debra Rog, Vice President, Westat, Former AEA President; Veronica Thomas, Howard University Professor.
AEA will host two more dialogues this year in San Antonio (April 2017) and Chicago (September 2017). The Dialogues on Race and Class will also be featured in a plenary session at Evaluation 2017. For more information on the series, please visit our website.
Potent Presentations Initiative - Check Out Our New Site!
From Sheila B. Robinson, Potent Presentations Initiative Coordinator
Hooray! It’s ready to go! Let’s get right to it and take a little tour around our new digs, shall we?
First, you’ll notice that the site looks quite familiar! We were able to integrate the great banner and p2i logo from graphic designer Christ Metzner with the familiar AEA banner and logo you’re used to seeing on all of our community pages.
Next, you’ll notice that we’ve updated the navigation of the site and simplified the organization by providing just four menu buttons found under the banner:
On the p2i HOME page, you’ll find a brief introduction to p2i, and the introduction of p2i’s three key components – Message, Design, and Delivery. Webinars for each provide in-depth learning and reference some of the resources found on the PRESENTATION TOOLS & GUIDELINES page.
All of our downloadable resources now live on the PRESENTATION TOOLS & GUIDELINES page. The page is organized with Checklists & Worksheets on top, then resources aligned to the p2i components – Message, Design, and Delivery – followed by resources for Audience Engagement.
As you browse this page, you’ll find links to additional content and pages along with the tools. Just look for tool titles that are links, as in this example:
Notice that “Slide Design Guidelines” is a link. This link will take you to another page of content on Slide Design. Another key addition to the site is that the authors who contributed the content are now recognized and their names linked to their websites or LinkedIn profiles.
Given the amount of content we had on poster design, along with the fact that posters are the largest category of presentations at our annual conference, we knew that POSTER PRESENTATIONS warranted its own page.
Here, you’ll find a page with specific guidelines for designing a conference poster, along with two additional navigation buttons. One takes you to even more content on Research Poster Design, while the other points to a selection of Award Winning Posters, not only from recent AEA conferences, but also from other organizations. Each poster image is accompanied by a brief explanation of what makes it a winner.
Don’t forget to visit the ABOUT US page to learn about the folks who have contributed to making p2i what it is!
Finally, please note that we’ve updated the p2i Twitter hashtag, as the original one selected is being used by others. We now have a hashtag that is all ours: #aeap2i. Please feel free to begin tweeting the p2i website and resources using this tag. Follow the hashtag #aeap2i by clicking on the top button found on the p2i HOME page, and while you’re at it, why not follow the association itself (@aeaweb) as well!
Behind the scenes…
Over the last year, we have worked behind the scenes to migrate and reorganize all content from the original p2i website to the main AEA site at eval.org (kudos to Zachary Grays, who did the heavy lifting!). We also updated the tools, and added new content and introductory language where needed. One reason for the move was to protect our content and tools from hackers. Our original site, built on a different platform, was a constant target and over the years we received countless notices from members that the site URL had been maliciously redirected (meaning it took people to a different website), or that downloads were not working. We’re confident now that the new site and all of our great content will be reliable and safe.
Be sure to visit eval.org/p2i and let us know what you think!
Sneak Preview! We have exciting new content for our p2i resource collection on its way to publication. Stay tuned to this newsletter to learn more!
International Update - What's on the International Evaluation Scene?
From Cindy Clapp-Wincek
When I was introduced to program evaluation in the late 1970s, I was taught that evaluation got its start with the evaluation of the Head Start Program in the United States. But evaluation has certainly become a worldwide phenomenon. The year 2015 was the International Year of Evaluation. In addition to many events raising the profile of the evaluation profession, the Global Evaluation Agenda 2020 was developed as a framework for how to continue to promote evaluations and their use in decision making (see http://www.evalpartners.org/global-evaluation-agenda; AEA Newsletter May 2016). The declaration of the Sustainable Development Goals by the UN in 2015 led to the development of the hundreds of indicators to be tracked by all the countries in the world, including the USA.
Associations and societies to promote evaluation and evaluators grow as well. The American Evaluation Association is the oldest professional association of evaluators. Shortly after AEA was created in 1986, the International and Cross Cultural TIG met at the annual conference in Boston and only had a couple dozen participants in the room. Now roughly 17 percent of AEA members — or approximately 1,200 individuals — live or work overseas. Voluntary organizations for professional evaluation (VOPEs) such as AEA exist in over 100 countries to support the capacity building of their evaluators, evaluation institutions, and systems.
Last summer, AEA’s International Working Group drafted the “International Engagement Strategy for the American Evaluation Association” to advise Denise Roosendaal, our executive director, in where and how the AEA should support AEA’s growing global work. Mike Hendricks, our previous International representative, summarized the planned work into 13 ways (AEA Newsletter, July 2016). A few of these ways include:
I greatly appreciate the opportunity to be the AEA representative to IOCE & EvalPartners, stretching to fill the very big shoes of our friend and colleague Mike Hendricks. If you want to get involved in any of the activities above, or have new international ideas, let me know and I will connect you as best I can.
Face of AEA - Meet Belinda-Rose Young
Name: Belinda-Rose Young
Affiliation: Evaluation and Translation Fellow with the Centers for Disease Control and Prevention
Degrees: MSPH, BS
Years in the Evaluation Field: 6
Joined AEA: 2015
Why do you belong to AEA? I joined AEA when it became my full-time job to conduct evaluations. Prior to this, I balanced evaluation contracts with conducting original research. However, with this new position, I wanted to surround myself with individuals who also wanted to grow professionally, and found a great home in the Translational Research Evaluation (TRE) TIG. Although I am a reserved person, I took it upon myself to connect with a couple of people, who then introduced me to more individuals who have even greater evaluation experience. Now, I am the communications chair of the TRE TIG! Taking on some level of responsibility, and subsequently being involved in more conversations and events, has kept me well informed about the future of AEA and the evaluation field, and has greatly improved my overall skills. I owe a lot to the TRE TIG and AEA.
What’s the most memorable or meaningful evaluation you have been a part of? A few years ago, I conducted a process evaluation of a translational research institute, whose “scholars” consisted of both graduate students and members of the community who were interested in making sure that the research being produced at the university was making it into the community. It was a rewarding experience to see the growth in all scholars at the end of a year — for the graduate students, their mindset on how they conducted research; for the community scholars, they felt more empowered because they were better equipped to serve their community. The experience also challenged me as to how I can better involve community members in my research and evaluation.
What advice would you give to those new to the field?
Don’t go at this alone. It’s very easy to become a team of “one” and try to navigate evaluation yourself. Get involved in topical interest groups, and attend training institutes and the annual AEA conference. Why repeat someone else’s mistakes when you have the opportunity to learn from them?
Announcing the 2016 AEA Marcia Guttentag Promising New Evaluator Award Winner
AEA honored four individuals and one organization at its 2016 Awards Luncheon in Atlanta. Honored this year were recipients in five categories who are involved in cutting-edge evaluation/research initiatives that have impacted citizens around the world. We will spotlight each award winner in upcoming issues. In this issue, we extend our congratulations to Lisa M. Dillman.
Lisa Dillman started her career as an early childhood educator working in a childcare center and in homeless shelters developing play spaces for children in crisis. Through these experiences, she learned that improving education and other social services requires thoughtful and systematic investigation and evaluation to bridge the gap between research and practice. This led her to the Social Research Methodology Program at UCLA's Graduate School of Education and Information Studies where she earned her Ph.D. in education, studying program evaluation.
Dr. Dillman's current evaluation practice is stakeholder-responsive and use-focused with the goal of helping programs reach their maximum potential in delivering services for social betterment. Through her work as a Senior Advisor at Education Northwest, she has the opportunity to lead implementation and outcome evaluations for a variety of programs, including teacher professional development, curriculum implementation, and collective impact. She also contributes to the scholarship about evaluation, having published articles about evaluation policy, evaluator competencies and evaluation theory.
“I feel very fortunate to work in this complex and dynamic field where I'm challenged every day and am constantly learning. Evaluation plays a vital role in shaping efforts to address urgent social justice issues, and I feel lucky to be able to contribute to this work.”