Blog Series! Evaluator Competencies

So I had an idea. As I ease my way into blogging in a more reflective way, I thought that perhaps I could do a blog series about the Canadian Evaluation Society (CES) evaluator competencies, where each post I reflect on one of the competences. The competencies have been recently revised, so it seems like a good time to do this. Plus, having a series will give me ideas for topics – and hey, let’s make it every Sunday, so that I’ll have a deadline as well. This seems like a good way to get me into the habit of writing here.

What Are Evaluator Competencies?

Competencies are defined as “the background, knowledge, skills, and dispositions program evaluators need to achieve standards that constitute sound evaluations.” (Stevahn et al, 2005) 

Cited by CES

The competencies were created as part of the program for the Credentialed Evaluator (CE) designation. To get the designation, one has to demonstrate that they have education and/or experience related to 70% of the competencies in each of the five domains. I got my CE under the original set of competencies, but anyone applying now would use the new set. It was a few years ago that I did my CE application, so it’s another reason why it’s a good time for me to reflect on where I am now with respect to the competencies.

The Domains

The five competency domains are:

1. reflective practice
2. technical practice
3. situational practice
4. management practice
5. interpersonal practice

Reflective Practice

“Reflective Practice competencies focus on the evaluator’s knowledge of evaluation theory and practice; application of evaluation standards, guidelines, and ethics; and awareness of self, including reflection on one’s practice and the need for continuous learning and professional growth.” (Source)

1.1 Knows evaluation theories, models, methods and tools and stays informed about new thinking and best practices.

I have taught Program Planning and Evaluation at both SFU (in the Masters of Public Health program) and UBC (in the Masters of Health Administration program) in the past couple of years, and I find that teaching is a great way to both deepen my own understanding of evaluation theories, models, methods, and tools and to stay informed about new thinking and best practices. In deciding what to include in a course, and how best to present it, and coming up (whenever possible) with activities the class can do to learn it, I learn more every time I prepare, update, and deliver a class. Also, students ask great questions (sometimes even after a class has ended and they’ve gone on to work in places where they are involved in evaluation) and sometimes it’s things that I’m not familiar with and I have to go and do some research to find out more.

I think my main reflection related to this area is that I am a firm believer that there is no one “right” way to do evaluation, and that it is best to start with what the purpose of an evaluation is and then figure out what approach, design, and methods will best help you achieve the purpose. Oftentimes, those requesting an evaluation come to it with assumptions about methods or design – like “I need you to do a survey of the program clients” or “how can I set up a randomized controlled trial to evaluate my program?” So I often find myself saying things like “Let’s begin at the beginning. Why do you want an evaluation? What do you want to know? What will you do with that information once you have it?”

Given that I think it’s important to find the best fit of approach, design, and methods to the purpose of an evaluation, it means that I need to be familiar with lots of different theories, models, methods and tools!

In terms of new thinking and best practices:

• I’m currently reading Principles-Focused Evaluation ¹Expect to see a blog posting on that once I’m done the book..
• I attend evaluation conferences and pick sessions where I can learn about new things – and deepen my understanding of things I’m familiar with. For example, at the most recent CES conference, I took a workshop on reflective practice to deepen my skills in that area (which I’m now actively working on integrating into my life), I attended a session on rubrics to learn more about those (next step there is to try applying rubrics to an evaluation!), and I attended a session on a realist evaluation (next step there is to have the presenter come to my class as a guest speaker so that I and my students can learn more!)
• I include a section in my course on “hot topics” in evaluation, which gives me the opportunity to explore the latest thinking in evaluation with my students. Recently, I’ve included complexity and systems thinking, and indigenous evaluation ²Except to read more about indigenous evaluation when I get to competency 3.7 in this blog series.. I also try to demonstrate reflective practice and humility to my students by telling them that I am exploring new areas, so I’m not an expert in these topics (especially indigenous evaluation), but that I’m sharing my learning journey with them.