Evaluator Competencies Series: Well-Being of Human and Natural Systems

1.4 Considers the well-being of human and natural systems in evaluation practice.

For this competency, I would say I’ve focused much more on the “human” than the “natural”. I see some overlap with the stuff I talked about last week, as considering the well-being of humans includes ethical concerns like, for example, maintaining confidentiality for participants in the evaluation.

But looking at this competency, it’s gotten me thinking about the well-being of human systems – which leads me back to my thoughts on learning more about equity, which I also mentioned last week. Within human systems, there are some groups that disproportionately receive benefits of programs/services/initiatives/systems than others – and similarly, some groups that disproportionately are harmed by programs/services/initiatives/systems. Coincidentally, in the AEA eStudy webinar last week, Jonny talked about how in systems there are often disproportionate distributions of benefits and that’s something that we as evaluators need to pay attention to and think about the values underlying our evaluations when we thinking about how we decide if a program/initiative is a “success”. I think Michael Quinn Patton also talked a bit about this on the Eval Cafe podcast episode that just came out this week on Principles-Focused Evaluation (PFE) , where he was talking about the difference between rules and principles using the example of “do no harm” – but if we are interested in making systems more equitable, then aren’t we technically “harming” the more advantaged by taking away some of the power/benefits that they currently have in order to distribute benefits more equitably. I think that if you care about equity, you’d say that in in the interest of fairness/justice, that’s OK and the rule of “do not harm” is actually too rigid ¹It’s entirely possible that I’m misremembering that was where I heard that example – I’m also in the middle of reading the PFE book, and having been reading/listening to a bunch of other stuff, so I may be conflating things. My apologies to all if I’ve mixed that up.. Is having some people benefit good enough to say a program is successful, or should we be looking at who is benefiting -and who is not – and who is being harmed – and who is not – across the whole human system? And how do we include that in our evaluations?

Similarly, I think that paying attention to the unintended consequences of a program/initiative is a really important part of an evaluation. If we are only looking for the ways in which the program designers hoped that the program would be beneficial, but didn’t hold space in our evaluations to look out for ways that the program may cause harm, we aren’t really doing a very comprehensive evaluation.

As for considering the well-being of natural systems, this is an area that I have not done a lot of work. Like with the equity stuff I talked about last week, I think the types of evaluations that I do (in the healthcare sector), don’t have an obvious link to environmental issues like they would if I were doing evaluation work with organizations that are working directly in the environmental sector. But every program/initiative exists within, and interacts with, the natural world, and we are in the middle of a climate crisis. So I think it requires some time to reflect on how I can better consider the well-being of natural systems in my evaluation practice.

There are definitely ways that I try to do environmentally-responsive things in my day-to-day work – taking transit to the office instead of driving, not printing things unnecessarily, using a travel mug for my coffee every day, recycling and composting, not drinking bottled water. But honestly, these things are easy to do, and I don’t know how much impact my individual actions in these regards really have. And there are other things that I do that I know are harmful to the environment – flying to conferences and to sites to collect data, for example, have a huge carbon footprint.

And then there is the idea of how to incorporate considerations of the environmental impact of the programs/services I evaluate. For example, in the project I’m evaluating on switching from paper patient charts to electronic patient charts, one thing that could be evaluated is the saving of paper by going electronic vs. the vast energy costs of the server space required to go electronic. Is that something I could include in an evaluation, especially considering that isn’t really within the scope of evaluation per se? And how would you compare those two impacts? Clearly, this is a space where I have lots of room to grow.

Image credits

• Earth image posted on Flickr by Kevin Gill with a Creative Commons license.
• System image posted on Flickr by Jurgen Appelo with a Creative Commons license.