Date: Wednesday, February 19, 2025
This post includes updated and improved material from a December 27, 2010 AEA365 post.
Hi! I’m Brian Yates, PhD. I’ve included cost as well as outcome measures in my quantitative and qualitative cost-inclusive evaluations since the mid-1970s.
Lesson Learned #1: Costs are not money. Money’s “just” one way to get access to the resources that make programs work. What matters for many programs is people’s time–clients’ as well as program staff’s, clients’ as well as staff’s space used, transportation (of clients to and from programs)…and not just total time spent working in the program, but the amount of time spent in the different specific activities that, together, are the program.
Hot Tip #1: Before asking stakeholders about program costs, we make a table listing the major activities of the program in columns and the major resources used by the program in rows. Different stakeholders estimate the amount of each resource that they use in each activity, and then compare others’ entries with their own. Insights into program operations often ensue!
Lesson Learned #2: The most valuable resources used in a program may not have a price, and may not be purchasable. Many programs rely on volunteered time and donated space and materials. A cost-inclusive evaluation can assign a monetary value to these resources according to what the same time from the same person would be paid in a job, but the most important thing to measure is the amount of time, the capabilities of the person, and the ways they spent their time.
Lesson Learned #3: When measured only as money, cost findings are instantly obsolete and do not aid replication. Inflation can quickly make specific monetary values for program costs out of date and, all too soon, laughably low. Translating 1980 dollars into 2025 dollars is possible, for instance, but does not inform planners what specific resources are needed to replicate a program in another context.
Lesson Learned #4: When presenting costs, keep resources in their original units. Yes, time is money…but it comes in units of hours to begin with. Report both, and your audience will learn not just price, but what it takes to make the program happen.
Hot Tip #2: How to squeeze benefits (monetary outcome) data out of effectiveness data? Multiplication, basically. Sample clients before, during, and after program participation to assess impacts of the program on clients’ use of health and other services. Validate with spot checks with records of those services. That’s effectiveness!
Next, transform effectiveness into monetary units: multiply a client’s frequency of service use by the cost of each service (average of health service provider’s fees for that service, for example). Then, compare costs of services used before versus after a program for similar periods. You’ve measured a potential program benefit that speaks louder than most outcome measures: cost savings produced. Have a range of fees? See whether the lowest, and highest, fees would change your findings.
Lesson Learned #5: – Just ’cause a number has a dollar sign in front of it doesn’t make the number any “better.” Not really. Benefits (and costs) are no more valid than the data from whence they came. The “GIGO” (Garbage In –> Garbage Out) principle applies: invalid costs and benefits data can lead to big mistakes about program funding.
The CEBE (Costs, Effectiveness, Benefits, and Economics) TIG website has loads of additional, free info to get you started on cost-inclusive evaluation (CIE)
Here’s a free online, downloadable manual I wrote on formative evaluation of not only cost, but also cost-effectiveness and cost-benefit…and not just for substance abuse treatment!
The American Evaluation Association is hosting CEBE TIG Week with our colleagues in the Costs, Effectiveness, Benefits, and Economics Topical Interest Group. The contributions all this week to AEA365 come from our CEBE TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.