Date: Thursday, April 30, 2020

Weathering the Storm

From Anisha Lewis, AEA Executive Director

Seven weeks ago, our staff was focused on AEA business as usual, preparing for our annual Summer Institute and launching annual programs. As the COVID-19 unfolded, it presented numerous challenges that sent our world as we have known it to a screeching halt. I want to assure you that the American Evaluation Association stands ready to weather this storm, and we are turning this challenge into an incredible opportunity to generate new strategy, collaboration, and ways forward for our evaluation community.

The Harvard Business Journal quotes Winston Churchill as having once said, “Never let a good crisis go to waste.” I am a firm believer that change often comes out of discomfort and fear, among other factors. We recognize this crisis as an opportunity to revisit who we are, whom we serve and develop new strategies to enhance member value.

We have canceled our Summer Institute Conference scheduled for June, and we are working on a few scenarios for our Annual Conference scheduled for Portland in October. I want to reassure you that AEA is making decisions around future programming and events with your best interests, safety and health in mind. As we enter into a new reality, know that AEA is committed to supporting your health and safety based on guidance from the World Health Organization (WHO), the Centers for Disease Control (CDC), and local health communities.

At this time, we are exploring cancellation options for our annual conference, but doing so is not as simple as one may think. If we cancel the conference now, we will have to pay penalties of approximately $450,000. The venues with whom we have these contracts are closed, so we must wait to negotiate. In the meantime, we are planning for alternate programming, including a virtual conference in the event that we are not able to move forward with a live event. Please know that we hear and understand your concerns, and that our staff and leadership are collaborating to develop several contingency plans. We do understand that our members invest much into planning and preparation to attend annual meetings, and we are making efforts to provide timely updates to our members.

 In the meantime, we are assessing opportunities to determine which of our programs, events, and activities can be stabilized, and which could flourish. We are also committed to allocating resources for the long term and learning how our members will reassess priorities and redefine value.

We ask that you trust the process, and try to stay as happy, healthy, and as calm as you can. And when even that gets difficult, remember the proverb, “this too shall pass.”

Fighting the Common Enemy Together, Apart

From Billi Shaner, Senior Monitoring, Evaluation, & Learning Advisor

 “Yap, yap!” my 2-year-old insists as she taps my knee for what feels like the hundredth time this morning.

“Okay, one more, but then Mommy has to work,” I reply, as I pick her up to sit on my ‘yap’ and read another story.

As an evaluator in the field of international development, I’m used to working virtually, but doing so with kids at home, during a global pandemic no less, has been extremely challenging, both logistically and emotionally. My older daughter is also constantly asking to be told a story, though understandably in a very different way.

“When can I go back to school? Can we go to the playground?... Oh right, I forgot. Mom, how do viruses start, and how do we stop them?”

In this unprecedented time we’re living through, I’ve never been more aware how critical it is to “tell the story” that is not only meaningful for stakeholders, but enables them to use the evaluation’s results for continued decision-making. This responsibility we have as evaluators to work with leaders to arrive at a fuller understanding of what’s happening today to prepare them to make better decisions tomorrow – decisions that will ultimately lead to the “enhancement of the public good” – strikes me as exactly the thing the world needs right now.

Much of my experience in this field has been working with international Non-Governmental Organizations (NGOs) to evaluate their community education projects. Through these evaluations, my ultimate goal is to interpret participant data through a context-specific lens in order to make useful suggestions for program improvement. It wasn’t until this month, however, that I came to understand on a deeply personal level the complexities of continuing “typical” education programming in times of societal disruption. It is now clearer than ever how critical it is to include the voices of all who are impacted by programming changes to arrive at a workable solution moving forward. I now intimately understand how no two experiences are the same and all are equally valuable contributions to the story.

As we move forward in this new world, I find myself thinking more about AEA’s values of an inclusive, diverse, and global community of evaluators. We are fighting a common enemy in this virus, but no two experiences are the same, and all our stories are needed. I’m so looking forward to October when we’ll be together again to share and learn from one another. I’ll never again take that for granted. 

April's Ask AEA: Job Opportunities

Are you new to evaluation? Do you have questions, curiosities, or concerns about the industry? Are you debating career opportunities, upcoming goals, solutions to current issues, or are just seeking some friendly advice? The only way to grow in your profession is by asking questions. In our new series, Ask AEA, we want to provide our members with that opportunity by providing as many resources and guidance as possible. Submit your questions here. 

Read below for advice on job searching, titles, and opportunities within the evaluation field, and where to start. And thank you  for your question! 

“When looking for job opportunities, I'm always stumped on potential titles to search. What sort of job titles are out there for people in the evaluation field, especially to differentiate from the true data analysts and statisticians?”

Hi There! Zachary Grays from AEA Staff here! While I can’t recommend specific titles to search (there are so many!), I can point you in the right starting positions to help refine your search. A great place to uncover job titles that align with your skill set is to use the AEA Career Center. Our newly enhanced career center is a treasure trove of career and RFP opportunities and is historically one of the most trafficked areas of the AEA website (by both employers and job seekers). As a member, you can post your resume’ to be searched or you can search the posted positions to get a good sense of what the market is demanding in terms of available jobs. You can also setup email alerts to be notified when your ‘dream job’ is posted.

If you’re still looking for some guidance, I would recommend tapping your colleagues on EvalTalk, AEA’s listserv. This is a great medium within which to solicit the advice of your peers, benefit from their experience, and participate in an ongoing conversation. While job postings are prohibited, EvalTalk has an extraordinary list of recipients and archives that might help point you in the right direction.

Lastly, as we all know, networking is a critical skill to keep in your toolkit when job hunting. Consider plugging into local affiliate networks in your region as they frequently host events that might help connect you with a future opportunity. I hope all of this is helpful! I’m sure with all three of those resources you’ll find what you’re looking for in no time!

Submit your questions for the chance to be featured in AEA's monthly newsletter. Make sure to stay up-to-date on the latest issues to receive answers to your questions from professionals in the field.

Connecting in Crisis: The Cross-TIG Town Hall

Supporting Communication and Collaboration Among Evaluators in the Context of COVID-19

From Sue Ann Sarpy, Disaster and Emergency Management Evaluation TIG 

AEA Board Update

From Tessie Tzavaras Catsambas, AEA Board Secretary and Immediate Past President

Dear AEA Members,

Because of an administrative glitch, we have not been in touch with AEA Board updates this year. Throughout this time, however, the board has been hard at work.

Prior to COVID-19, we met in person in Washington, D.C. in January, and have had several board meetings since then. In January, we developed our annual agenda setting forth the work of the board for 2020. We discussed how best to position the board to listen to members and increase member engagement on the strategic issues the board considers. Topics on our annual agenda include:

• Strengthening member engagement
• Trends in the field and business of evaluation that we need to consider as we look at our evolving membership
• AEA’s positioning in the evaluation space—in the United States and internationally
• Building equity and inclusion in the work of the board and the association
• Living and promoting our board policies
• Monitoring how AEA is doing in key areas of our organizational life
• Planning of the budget and addressing expenses
• Fundraising to enable us to serve members in the best way we can

The board has also embarked on discussions regarding AEA brand today and in the future, and hopes to engage members in conversations about this important issue. We are being intentional about our dialogue with members in this topic. In addition to working groups and task forces, board members are considering including questions in the member survey coming out later this year, Townhalls, and possibly adding a few sessions at our annual conference (or online).

Post-quarantine, in our April online board meeting, we focused on how to manage the impact of the pandemic. As you know, the AEA Summer Institute had to be cancelled because it was not safe to hold it. Our staff is monitoring the situation for our Annual Conference in November. We're working closely with the board to make the best decisions for AEA, and to adjust our annual budget accordingly. There are legal and financial issues related to when the decision to cancel should be made and all factors are being monitored closely by AEA. The staff is actively looking at all options and planning to keep the membership fully informed.

The board is working to reframe the current crisis for AEA. We reviewed the current challenges in light of AEA’s mission and ends goals. We are exploring ideas for how we will achieve them, even if we are unable to hold our annual conference.

We appreciate the outpouring of warm messages, webinars, and ideas on how to support each other during the pandemic. We are blessed with a strong and dedicated community.

Stay safe everyone, and we will weather this together.

AEA Diversity and Local Affiliate Highlight

Working Inside Liminal Spaces with African Leaders and Evaluators

From Nicole Robinson and Miloney Thakrar

These are historic times, even before COVID-19. In a period of enormous economic inequality, climate change, racialized mass incarceration, and direct assaults on the pillars of democracy, many evaluators are sheltered in place, experiencing the first global pandemic in a modern era. At ¡Milwaukee Evaluation! Inc., an essential part of our work as the Wisconsin statewide AEA local affiliate is to seize liminal spaces, sites for liberation and solidarity, and engage in emancipatory capacity building for evaluators. For the past two summers, we have held historic meetings to bring 50 Wisconsin/Midwest evaluators in conversation with 50 African leaders, many of whom are responsible for evaluative work in medicine, journalism, law, and environmental conservation.

What did we discuss? Reading primarily works from African and African American authors, we examined:

• Wakanda evaluation: How can African leaders and people of color living in the U.S. connect on evaluation? We were encouraged by AfrEA’s Made In Africa initiative, which prioritizes indigenous knowledge systems.

• Decolonizing evaluation: How does imperialism and postcolonialism influence our evaluation practice? Where can we center an African-rooted evaluation in our work?

• Unpacking capitalism: More than ever, U.S.-based evaluators need to understand the mechanisms of free market ideologies in both the programs we evaluate and our evaluation methodologies. With the view that knowledge production, capitalism, and white supremacy are interlinked, we are extending that dialogue in an upcoming May webinar that will critique capitalism, the elite class, and corporate-run governments based on the readings from these sessions, and in recognition that the coronavirus pandemic is a triggering event, an event that will have direct effects on the knowledge agendas we set as a field.

• Corruption as a barrier to evaluation: How is corruption defined by evaluators and documented, if at all? How can we keep our integrity while working in a corrupt regime or system? How can we evaluate with authenticity? Our evaluation textbooks and professional development opportunities rarely cover these topics, but for those living in the margins, it has always been essential that evaluators understand (and name) political and economic corruption. Corruption has direct consequences on the distribution of resources, services, and assets. Or put simply, whether there is enough food to eat, clean air to breathe, life-saving treatment, and dignified work for all.

• Paternalism: Programs and financial investments (both globally and domestic) that are designed “to help” and “create innovative change” have ties to white savior complex that undermine genuine self-determination and balances of power. In small groups, we discussed how does paternalism, a form of white supremacy, show up in program activities and results?

These historic meetings created space for an exchange of ideas, professional development, and networking as an act of resistance. Pushing for change by directly confronting corruption, capitalism, and white supremacy in an evaluation project requires healing methodologies. As such, leadership development activities aimed to enhance an evaluator’s healing skills, placing such skills on an axis equal to our capacity to adhere to the scientific method. During our time together, one breakout group signed an anti-corruption pact and others committed to fighting anti-Black racism. In the last meeting, attendees practiced journalistic-style interviews to prepare them as the new face of evaluation, ready to leverage interactive mediums and fill the gap in available media of local evaluators of color speaking on social justice (the video will be out soon). Through dance, food, and laughter, we celebrated the generosity, courage, and compassion that inspires our evaluation work.

In our recent report on culturally responsive evaluation, the findings reveal we have more to do to increase the number of  evaluators of color and other underrepresented groups (LGBT+, ESL, youth, rural), but inclusion is not enough. Harnessing liminal spaces primes evaluation leadership to challenge the strands of society—like capitalism and corruption—that are traditionally considered too big, abstract, and immaterial for evaluation. 

Member Spotlight: A Q&A with Nancy Kingsbury

With over 50 years of dedicated service and vast leadership contributions during her distinguished GAO career and role within AEA, Nancy Kingsbury is an AEA member that has provided outstanding service to the public. Join us in celebrating her retirement and read more on her contributions, advice, and journey through the evaluation field.

How did you get started in this field and what motivated you to get where you are today? 

These [first two] questions are unavoidably interconnected. It all clearly started with my education, in particular my Ph.D. from Johns Hopkins University in experimental psychology and analytic methods. I worked for a time with a federal contractor whom I had worked for during school. But while I enjoyed the work, I discovered I hated selling the work.

My first government job was with a small technical research group at what was then the National Bureau of Standards (now NIST). I did studies on the human factors of keyboards for sorting mail in the Postal Service and the use of video recording in the courts at a time when the only court system using videos was in Alaska. I moved to the office of the Undersecretary of Commerce for Science and Technology where I was selected for a commerce development program called the Commerce Science and Technology Fellowship. That program provided some educational experiences, but mostly a work experience in another agency, intended to give commerceM mid-management scientists broader government policy experience. I ended up at the then Civil Service Commission, initially in the Office of Policy, and later in the Intergovernmental Personnel Act (IPA) grant program. This was a $20 million [dollar] grant program, largely aimed at states and local governments, to improve the merit-system basis of their personnel programs.

As with many such small programs, there were congressional pressures to reduce the funding, and the director of the program asked me if I could do an evaluation of the program that might provide some support for not reducing the funding. So that was my first actual “program evaluation”—a survey of grant recipients to provide evidence about the achievements and value of the program. In the end, the program kept its funding for several more years.

Around the time I was doing the IPA evaluation, I learned of a meeting at Harvard University of academics and practitioners who were interested in program evaluation in 1974 to discuss the possibility for forming a professional organization for people interested in evaluation. There was another evaluation organization in the education field, but this discussion focused on broader program evaluation interests.

In what ways have you been involved with AEA throughout your membership?

The group decided we should try to do that and, when volunteers were sought to make it happen, I put my hand up and ended up taking on the responsibility to get the Evaluation Research Society (ERS) incorporated and helping to kick it off. I ended up on the organization’s national council and, after a couple of years, I became the organization’s secretary treasurer—essentially, the business manager for the organization for a period of more than 15 years.

In 1978, I was recruited to the team at the Civil Service Commission that developed and then implemented the Civil Service Reform Act (CSRA) of 1978. I was well into that, including planning for how we would evaluate the creation of the Office of Personnel Management (OPM) and the new Senior Executive Service, when I was recruited to become the Director of Resource Management at the Peace Corps. That position included supervision of Peace Corps’ evaluation office, but had broader administrative responsibilities, including personnel and budget responsibilities at a time when President Carter was trying to make the Peace Corps more independent. It had been incorporated into the action agency some years earlier.

The Peace Corps was one of the first agencies to have an independent evaluation office. After the election of President Reagan, I returned to OPM where I managed through a RIF, in part by throwing myself into the Evaluation Research Society management. Two former presidents of ERS were senior executives at GAO’s Program Evaluation Methodology Division and they supported my application in 1984 to GAO’s executive candidate development program. I’ve held many positions since at GAO under three comptroller generals, culminating in the last 19 years, establishing and managing GAO’s central specialists organization, the Applied Research and Methods team, which includes a small center for Evaluation Methods and Issues, and many others with social science research skills applicable to evaluation.

To finish the AEA part of this story, in the mid-1980s when Joe Wholey was president of ERS, he and I and others worked to merge ERS with The Evaluation Network, the education evaluation professional organization. The two organizations had very different cultures and approaches to evaluation, so the discussions took some time. But that effort ended with the creation of AEA, and since I was still Secretary Treasurer of ERS, I also managed the business side of that merger. In the early 1990s, as my responsibilities at GAO grew, I passed the Secretary Treasurer role to Rita O’Sullivan and backed away from having a leadership role in AEA. My involvement since then has largely been just as a member, sending GAO staff to the annual meetings and occasionally attending myself.

Are there any specific moments, accomplishments, or people that stand out or particularly resonate with you?  Why do you feel these moments or people were so influential to you during your career?

In 50 years of public service, there have been many memorable events, as you might imagine. In more or less chronological order:

At CSC (Computer Sciences Corporation), my immediate supervisor, Joe Robertson, taught me the importance of gentle candor in dealing with staff (blunt, but supportive), and the opportunity to be tasked with supporting the passage of the Civil Service Reform Act by providing 190+ presentations all over the country about what it was about. This opportunity gave me great experience with speaking skills.

At the Peace Corps, the director at the time, Dick Celeste, a former governor or Ohio, whose leadership and support made the daunting task of making the Peace Corps “an independent agency within ACTION” an important challenge (but we succeeded in less than two years.).

At ERS/AEA, [I had] the opportunity to work with some amazing evaluators over a substantial period of time. I learned a lot. I also had an important adventure the year that AEA’s annual meeting in San Francisco was interrupted by the San Francisco earthquake. The organization’s finances were relatively fragile at the time, but we managed to get through that period and be financially solvent going forward.

At GAO, during the 35 years I was there, I worked with many excellent leaders and enjoyed the amazing variety of GAO’s work. I also valued the focus there on the quality of the work. One of my real satisfactions was having led the effort to articulate GAO’s Dimensions of Quality, which has been supported [and] essentially unchanged by three different Comptroller Generals. And, beginning in the summer of 2000, I was asked to figure out how to centralize GAO’s technical specialists across a wide variety of expertise (e.g., economics, social sciences, statistics, data science, science and engineering, computer security, actuarial science, demographics), figure out how to organize those resources, and ensure that their expertise contributed to the wide range of GAO’s work. Every day has been a new adventure.

How has AEA evolved over the years?  Where do you see the organization heading in the future?

I was privileged to participate personally in the two biggest initial changes [within AEA]: the creation of ERS and the merger to create AEA. The next major change was the transition to a professional (as opposed to volunteer) management of the association, which has made a huge improvement and been an important investment. I think the next challenge is going to be to take advantage of recent developments, particularly at the federal level, for evidence-based policy-making and management. Going forward, brining evaluator skills and focus on evidence is going to be hugely important for our country.

What advice would you give to someone just getting started in the field of evaluation?

Keep your analytic skills current and growing, be transparent in the methodologies you use, be open to new opportunities, and share your knowledge with others.

Call for Nominations: AEA Board of Directors Deadline Extended!

AEA Offers COVID-19 Support and Resources

AEA's top priority at this time is the health and well-being of its members and the evaluation community as a whole. We understand this is a strenuous and difficult time, and are dedicated to providing you with support and resources to help you navigate the evolving effects of the COVID-19 outbreak.

We want to remind you of a few of our resources to help you through this time.

• AEA's daily blog, AEA365, is rolling out a series of Saturday posts offering reflection and information for evaluation practice during this global pandemic.

Topics covered include Reflecting on the Role of Evaluator During this Global Pandemic, Tips + Resources for Virtual Gatherings, and Self-Care in the Age of Coronavirus.

Click here to subscribe to AEA365. We will continue to share resources and experiences of our community. 

• AEA encourages you to turn to your community of peer evaluators through EvalTalk. This discussion forum provides a safe space to trade insights and concerns, talk through specific challenges that are arising, and overall, to lean on each other for support. Now is the time to maintain strong connections with colleagues worldwide as we navigate today's climate together.

• You can also stay connected to your community through TIGs. Topical Interest Groups are defined around a certain topic or interest and create a forum enabling its members to become a resource that the entire community can leverage. This organized community and combined experience is needed now more than ever. As an AEA member, you are allowed to join up to five of the 60 Topical Interest Groups.

• AEA continually monitors its social media channels, looking to bring forward engaging and relevant content to our community. We continue to share resources circulated by the broad evaluation community. Follow us on Twitter @aeaweb or "like" our Facebook page for timely updates and content relating to the pandemic and beyond.

• While you are looking to stay connected to your teams, we recommend browsing the AEA Coffee Break library on the Digital Knowledge Hub. These 20 minute webinars are free to all members.

If you have resources you think would be valuable to the evaluation community, share them with us by contacting AEA at info@eval.org.

Oxford's Most Recent Evaluation, Research, and Nonprofit Publications

Available with AEA's Discounted Price 

Measures for Clinical Practice and Research edited by Joel Fischer, Kevin Corcornam and David W. Springer is the definitive reference volume on assessment measures for both practice and research in clinical mental health. This new edition includes hundreds of standardized measures, including new instruments for measuring children’s clinical conditions, new measures for couples and families and target searches for instruments in health care  conditions, personality disorders, and addictions.  

Volume 1: Couples, Families, and Children (9780190655792): $89.95

Volume 2: Adults (9780190655808): $99.95

Two volume set (9780190655815) $160.00

Just as a reminder—and this is of course something you should feel free to promote with your members—all AEA members can receive a 20% discount when they order through the website www.oup.com/academic using the discount code AEA20.

If you are a publisher and would like to participate as an AEA publishing partner, or if you are an author of an evaluation-related text from an alternate publisher that you would like to see participate, please contact the AEA office at info@eval.org.

AEA Professional Development Corner 

Advance Your Learning with AEA 

The Digital Knowledge Hub is an online platform featuring professional development opportunities for evaluators, by evaluators.

Register Now for AEA Upcoming Town Halls

Using the Revised AEA Guiding Principles in Your Practice 

When: 2 p.m. ET on Friday, May 8, 2020

The Guiding Principles, which support our ethical decision making as evaluators, were revised and approved by the membership in 2018. The Guiding Principles Working Group is tasked with assisting members to use these Principles in their practice. Toward that end, submissions for the AEA conference will need to reference the Guiding Principles. This town hall will review the major changes to the Guiding Principles made in 2018, and engage town hall participants in discussing ways the Principles can be incorporated in the evaluation practice from planning an evaluation, carrying it out, and supporting the use of its findings. Register now.

AEA eStudies

We know it is a difficult time for educators and students alike as you transition to virtual learning. AEA wants to remind you of some resources for both you and your students.

In-depth eStudy courses offer a deep dive into top-of-mind evaluation themes and topics. Open to both members and nonmembers alike, eStudies provide a diverse learning experience where collaboration is encouraged. eStudies are a great way to be immersed in top-of-mind evaluation themes and topics via an online course. Here are a few eStudies, archived and upcoming, to consider exploring:

Upcoming eStudies:

• Tuesday, May 5: eStudy 105: Social Network Analysis for Beginners
• Monday, May 4: eStudy 106: Introduction to Independent Consulting
• Thursday, June 4: eStudy 107: Introduction for R

Popular Recorded eStudies:

• eStudy 081: Dashboard Design
• eStudy 085: Using Correlation and Regression: Mediation, Moderation and More
• eStudy 97: More than two options- How to collect LGBTQ inclusive data
• eStudy 100: Principles – Focused Evaluation

Coffee Breaks

AEA Events of Interest

In this section, we spotlight events that may be of interest to the AEA community, as suggested by fellow members. Please note these events are not sponsored by AEA. If you would like to suggest an upcoming event, email Cady Stokes, AEA newsletter editor, at cstokes@eval.org.

Member Suggested Events 

In this section, we spotlight events of interest to the AEA community, suggested by fellow members. Please note these events are not sponsored by AEA. If you would like to suggest an upcoming event or highlight actions members are taking during the COVID-19 crisis, email Cady Stokes, AEA newsletter editor, at cstokes@eval.org.

Thank You to Our Longstanding Members

AEA would like to recognize and thank some of its most longstanding members. Click here to view individuals who are celebrating 5+, 10+, 20+ and 30+ years with the association this month!

Welcome to Our New Members

AEA would like to welcome those who have recently joined the association. Click here to view a list of AEA's newest members.