Evaluator Competencies Series: Self-Awareness and Reflective Thinking

I didn’t write a blog posting in my series last Sunday – the weekend was busy and time got away from me! But it’s now this Sunday night and I’ve got cup of tea and I’m ready to reflect on reflective thinking!

1.7 Uses self-awareness and reflective thinking to continually improve practice.

Often I do my my reflective thinking over a a cup of tea – whether sitting on my own to do some reflective writing, or chatting with colleagues (As an aside, if you want to read some brilliant thoughts on reflective practice, check out Carolyn Camman’s fabulously titled blog posting “The coffee is largely metaphorical“). I’m an external processor and I find that I tend to come up with a lot of my great “a ha!” moments when I write my thoughts down or talk to a friend or colleague. I also don’t have a great memory, so when I have an insight, I need to write it down to cement it in my brain (or the very least, so I can look it up again later.)

Journalling

I write a lot of reflections as I go about my work. Whether I’m collecting data, analyzing data, in a meeting, or whatever activity I might be doing, if I have an “a ha!” moment, I write it in my reflective journal (which for the project I’m currently working on is typed up and saved on a shared drive with the rest of my team’s reflections, as these “a ha!” moments are about the content of the evaluation that we are working on together). A reflection might be about a pattern I’m noticing in the data, or a connection I’m making between different parts of the evaluation, or a surprise that I wasn’t expecting, or thoughts on some of our longer-term evaluation questions. My general rule is “if it’s interesting enough for me to want to tell my team about this cool thing I saw or thought of, it should write it down as a reflection). This improves my practice because it helps me to identify things that are important to the interpretation of the data, which allows me to develop accurate and comprehensive evaluation findings.

I also keep some separate reflections that are more for myself than as part of the evaluation data. For example, since I’m the team manager, if I have reflections that are about my work as a manager, and I might not want to share those with the team right away – especially if I’m trying to work through a challenge or figure out a way to be a better team manager. Some of those reflections might become things that I do want to talk about with my team later, but sometimes I need some time and space to work through stuff first. This helps improve my practice because being an effective leader will help my team be effective in its work.

Team reflections

Speaking of my team, we’ve taken to having group reflection sessions after we complete any big chunk of work where we debrief on:

• what worked well
• what didn’t work well
• how might we have done things better
• what can we glean from what worked well/didn’t work well to improve our practice for our next task

These are some pretty standard evaluation type questions, but we’ve definitely been able to continually improve our practice by doing this reflection together.

For example, in our first big round of data collection, we didn’t do nearly enough documentation of our data analysis. And with having a big team of people all working on different pieces of the data analysis, it meant that we had a lot of files that we’d all named in different ways, with our spreadsheets set up in different ways and often not very well labelled. So when it came time to write up our findings, it was quite difficult to find the data we needed, and we sometimes had to reproduce some of the analyses to ensure we had the correct data. So my big lessons learned for future rounds of data collection were:

• we needed standardized naming convention that we all used
• we needed all steps of analysis clearly documented so that another person could pick up the file and understand exactly what was done (without having to sift through formulas and pivot tables to figure out what it all meant)

These seem like pretty basic things – and they are – but this was the first time for all of us working on a big team. We each had our own individual naming conventions and ways of setting up our analyses in our spreadsheets that had served us well working as individual and what we hadn’t realized was how many different ways people could do the same task! Since the project is being implemented in a phased approached, we are now entering a period of time where our work will be a bit cyclical (collect baseline data for a site, monitoring data at the time of implementation, collect post-implementation data 3-6 months later, and repeat for the next site). And I can see that we are getting better and better each time because we’ve been reflecting on how we do our work and finding ways to be more efficient and more effective.

Another reflection that I shared in a team reflection session recently was something that I think links to the “self-awareness” part of the competency. Working in healthcare, even as a non-clinician, you get exposed to situations and information that can be quite emotional. For example, even when doing a chart audit, you get exposed to stories of serious illnesses/injuries and deaths. Or when interviewing healthcare workers who are exposed to traumatic situations, you also get exposed to those traumatic situations. As human beings, this can bring stuff up for us (like similar illness, injuries, patient journeys, and deaths of loved ones, for example) and it’s important to be kind to ourselves when stuff like this gets to us. I am extremely lucky that I work in a large team made up of kind and caring colleagues, so we know that we can go to each other if we need to debrief, or if today is just not a good day for us to do that particular observation or interview. Being aware of situations that might bring up things for me and being aware of my emotions as I’m experiencing them can help me to manage those, ask for help when I need it, and thus help to ensure that they don’t negatively effect the work. It can also help me to be empathetic to my colleagues and the people I interact with as I do my work.

In addition to reflection with my team of evaluators at work, I am also part of a co-op inquiry group that meets monthly to reflect on a particular topic (for us, it’s “boundaries in evaluation”) and that has been an amazing experience to hear the reflections of a group of evaluators from different sectors and locations – I have left every meeting having expanded on ideas I’ve been having and having learned new ideas or perspectives from my colleagues that have resonated with me.

Teaching

Teaching is a fantastic opportunity to reflect. Whenever I prepare to teach an evaluation course, I’m dedicating time to stepping back and thinking about the big picture of evaluation – what it is and how to do it well. I find it also brings me back to the basics and it gives me the opportunity to think about whether there are ways that can improve what I’m doing. I use a lot of storytelling and examples when I teach – I’ve had many students tell me that they really appreciate that I do that because I tell them “what really happens, as opposed to what the textbooks tell you it’s going to be like”. But it also helps me because, again, it gives me an opportunity to think about how I’ve done my work, how it links to concepts, theories, standards, etc. and how I might do my work in the future.

In addition to getting back to basics, I also like to tell students about whatever the “hot topics” are in the field at the time, which means that I have to keep abreast of what the hot topics are, and typically do a bit of research to be well versed enough in the topic to discuss it with the class. This is an opportunity for me to identify gaps in my knowledge and do some learning.

Another aspect of teaching that I think is reflective is that students tend to ask really great questions. And since they are coming from a different perspective, sometimes those questions are things that I haven’t thought about before, which forces to me to reflect situations from a different angle. Sometimes they ask questions that I do not know the answer to – when that happens, I tell them that I’ll go do some research and get back to them. This links to that notion of self-awareness – knowing the limits of my knowledge, having the confidence to say “I don’t know that right now, but I will find out”.

Blogging

And finally, this blog is something that I’m using as part of my reflective practice now. I’m glad that I decided to write this blog series on the evaluator competencies as a way to provide some structure and timeline to get me in the habit of reflecting here on a regular basis ¹Last Sunday notwithstanding.. I’m finding it quite useful to spend a bit reflecting on the extend to which I have each of the competencies and areas for each where I can continue to learn and grow.

Image Source:

• Pot of tea photo posted on Flickr by Jack with a Creative Commons license.