Date: Friday, February 21, 2025
Brian Yates PhD here, in Washington DC. I’m CoChair of the Costs, Effectiveness, Benefits, and Economics (CEBE) TIG. I’ve been doing, teaching, learning, writing about, and presenting on cost-inclusive evaluation for over 50 years, starting as a doctoral student at Stanford University (in the Department of Psychology) and continuing today at American University in Washington, DC.
What our TIG is talking (OK, AEA365-ing) about this week is Cost-Inclusive Evaluation (CIE). We’ve borrowed that term from Dr. Patricia Herman (thanks, Pat!). But, to make super-clear that this really is not (just) about money, maybe we should call this Resource-Inclusive Program Evaluation (RIPE) instead. That’s more accurate, and comprehensive.
And good way to do RIPE is with the Recipe model :-}
At its core, the Recipe model of cost-inclusive evaluation tries to answer three questions present in some form in most evaluations:
Cost- (Resource-) inclusive evaluations often measure and report only the total value of resources expended by a program, and global outcomes (effectiveness or monetary outcomes). This obscures the program itself, treating it as a “black box”—an unknown.
But, building on the Ingredients approach (what it’s called) of Dr. Henry Levin, my colleagues and I have developed the Recipe model.
The Recipe model promises, we feel, a more complete, more formative, more disseminable approach to including costs in program evaluations. It goes way beyond listing program resources and their costs. That won’t, by itself, describe a program for continuation or replication, any more than a list of ingredients (or just the total ingredients’ costs) tells us how to bake bread!
By detailing which resources are used and combined in each program Activity, and in what order, the Recipe approach makes program continuation, replication, and dissemination possible. Add information on what the interim results (Processes in the logic model below) should look like, and what the final Outcomes should be, and you’ve got a pretty complete “recipe” for the program … plus you’ve got the foundation of a truly comprehensive cost- (resource-) inclusive evaluation!
Like a good cooking recipe, this approach uses but goes beyond lists of types and amounts of resources, activities, processes, and outcomes observed for a program. It also explains what goes on between each of these four program elements: the all-important causal relationships (arrows).
To understand why programs fail, or even backfire, the Recipe approach evaluates potential causal connections between resources, activities, and processes that cause program outcomes. This actually helped us understand what flopped in a well-funded drug use prevention program which unfortunately increased adolescents’ willingness to use, and actual use, of drugs.
More generally, the Recipe approach encourages an understanding of programs that goes beyond solely focusing on possible cause -> effect relationships of program activities-to-outcomes to ask what often are the most pressing questions programs need to answer to optimize their use of limited resources:
a. how much of which resources does the program need to produce the outcomes it seeks?
b. how much money do we need to access and maintain those resources in the context in which the program hopes to (continue to) operate?
c. how does one actually do what the program does?
d. how can we convince the community, the funders, the participants that the program is a good investment of their space and tolerance, their funds, and their time and energy?
For several decades now, variants of the Recipe approach have produced insights into program successes and failures that would have been impossible with other approaches.
The American Evaluation Association is hosting CEBE TIG Week with our colleagues in the Costs, Effectiveness, Benefits, and Economics Topical Interest Group. The contributions all this week to AEA365 come from our CEBE TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.