Next up in my evaluator competency series is the second Reflective Practice standard:
1.2 Integrates the Canadian/US Joint Committee Program Evaluation Standards in professional practice.
The Program Evaluation Standards are a set of statements that evaluators – or evaluation users – can use to plan and execute high quality evaluations and to judge the quality of evaluation proposals and evaluations. The standards were developed by the Joint Committee on Standards for Educational Evaluation (JCSEE) and are used by both the Canadian Evaluation Society (CES) and the American Evaluation Association (AEA). The standards were most recently published in 2011 (the 3rd edition); they were reviewed more recently than that to see if they needed updating but it was decided that they were sufficient the way they are.
The standards are categorized under five categories:
- Utility
- Feasibility
- Propriety
- Accuracy
- Evaluation Accountability
In each of these categories, there are several standard statements that describe what high quality evaluations should do. For example, under the category of “utility”, there are 8 statements of what evaluations should do to be useful and under the “propriety” category, there are 7 statements of what evaluation should to be ethical, just, and fair.
As I reviewed the standard statements for this blog posting, I noticed that both the CES and AEA, which both list the statements on their websites, include the following note:”Authors wishing to reproduce the standard names and standard statements with attribution to the JCSEE may do so after notifying the JCSEE of the specific publication or reproduction.” So, since I haven’t notified the JCSEE that I would like to reproduce the statements here on my blog, I can’t do so. You can read them over on the CES website though.
But the standards are more than just the statements. There’s a whole book
It should also be noted that, despite the “should” wording of the standards, they aren’t meant to be slavishly followed, but to be applied in context and with nuance.
The standards also exist in tension with each other and you have to figure out the right balance. For example, there is a standard that says you should use resources efficiently, but another standard that says you should include the full range of groups and people who are affected by the program being evaluated. Evaluators need to find the balance between being thorough, but also been efficient in our use of resources.
In terms of my own practice, I think I can be more explicit in my use of the program standards. I’ve been an evaluator for a decade and I’ve integrated a lot of the standards into my work such that it’s just second nature (things like being efficient in my use of resources, using effective project management practices, using reliable and valid information, and being transparent). But there are other standards for which, as I read them I think “I could probably do better” (e.g., being more explicit about my evaluation reasoning or encouraging external meta-evaluation).
The evaluation standards is such a big topic, I’m barely scratching the surface here. So once I’m done this blog series on evaluator competencies, my next series is going to be on the evaluation standards! I think that will be a good way to get me to spend a bit of time reflecting on each of the standards and thinking about how I can improve my practice related to each one. And I’ll be sure to contact the JSCEE to let them know I’d like to reproduce the standard statements here on my blog!
Image sources:
- Book cover from Amazon.ca
- Scales photo posted on Flickr by Hans Splinter with a Creative Commons license.
Thanks for starting with an overview of the JCSEE standards, Beth. I think it is important to add that the CES was involved from the start when the committee was formed back in 1975 and continues to be at the table today. Although the standards were initially intended for use in educational settings, the CES saw their value for evaluations in other settings when they were adopted as the CES evaluation standards in 2012.
You mentioned that you thought you could “be more explicit in my use of the program standards.” You are not alone. Based on the interest of participants in the standards session at the CESBC 2018 Evaluation Conference, I think you share this intent with others across the evaluation community. One action I have taken based on the same reflection has been to incorporate references to one or more key standards that will be particularly relevant when writing proposals or contracts for evaluation work. This simple step elevates the priority and awareness of one or more specific standards for the duration of a project. Two more actions were to order a copy of The Evaluation Standards: A Guide for Evaluators and Evaluation Users (3rd ed.) to replace my well-thumbed copy of the 2nd edition (1994) and to read the scenarios presented in the 3rd edition. The 3rd edition amended the earlier standards and updated the case studies of the 2nd edition so those who may have used the previous volume as a text in grad school, as I did, should consider a fresh look at the most recent version. Others who may only use the standards as a checklist do themselves a disservice by not taking a deeper dive into the practical wisdom of the case studies presented in the 2011 guide. Kudos to you, Beth, for raising awareness of the utility of the standards for reflective practitioners via this blog entry.
Thanks for these practical ideas about how to be more explicit in the use of the standards, as well as for the information about the history of the standards, Sandra!