As part of EvalYear 2015, the School of Public Affairs at the University of Colorado Denver, is holding a webinar series called: “Practical Applications of Systems to Conduct Evaluation: Cases and Examples“. They have a great list of 30 minute webinars running from January to June related to use of systems thinking in evaluation. Systems thinking is something I’ve been reading up on lately, so these webinars are very well timed from my perspective!
Title: The Use of Rich Pictures in Evaluation
Speaker: Judy Oakden, independent evaluation consultant
Date: 8 April 2015
This was a very interesting webinar as it introduced me to a methodology that I’ve not heard of before: Soft Systems Methodology and explained a bit about the use of one of the methods from that methodology: rich pictures. The webinars are only 30 minutes long, so they are really meant to just whet your appetite and give you some practical tips on applying methods related to evaluating complex systems, and this seminar definitely archived that objective for me!
Here’s the notes I took during the webinar:
- “rich pictures” is a tool to help visualize problematic situations that you want to unpack to aid in the development of an evaluation framework, the evaluation questions, etc.
- rich pictures help you to:
- isolate key issues quickly
- avoid being overwhelmed
- manage emerging or changing circumstances
- represent a wide range of stakeholders
- understand interconnections
- understand problematic situations
- need to have the right people in the room to facilitate the creation of a rich picture
- rich pictures can be easily merged into existing evaluation practice
- use rich pictures at the start of an evaluation to help understand the territory
- comes from “soft system methodology” (been in existence for ~40 years – but relatively unknown to many evaluators)
- don’t need to use the entire methodology – can just use some of the tools
- how do you make a rich picture?
- address context and dynamics when working in groups
- work best when the group has gotten through form, storm, norm
- don’t want to start with this – groups need to form
- 3-6 people working on a picture
- consider the power dynamics of the group (e.g., mix up managers with staff, etc.)
- consider confidentiality – some of the things people will draw are the “elephants in the room”/”the unmentionables” – since we know these will come up, need to consider how we will manage confidentiality (state at the start – “what’s said in the room, stays in the room”)
- instructions for participants
- systems.open.ac.uk/materials/T552/ – open university material on instructions on rich pictures (and some other diagramming techniques)
- make sure the participants know what they need to do
- make sure the participants know what question they are being asked to answer
- test the question before you run the session (to make sure it’s not ambiguous)
- picture needs to include structure, process, issues/concerns/yourself (and roles/relationships of people)
- don’t need to map the entire system – just the part you are dealing with
- set expectations of ambiguity
- often a big pause as people think about what they need to do
- trust the process – and ask the participants to trust it too (“I promise you it will work!”
- set the expectation that ambiguity is OK
- provide reassurance and timely feedback during the process
- doesn’t matter where you start on the picture, just get started
- often one (or a few) person will
- looks for ideas, not perfect drawings
- messy drawings is fine
- doesn’t need to be an artistic masterpiece
- some people feel that having to draw slows down their communication (they are used to being articulate, but struggle to draw even a stick figure)
- can be overwhelming (when you see how complex the situation is) – reassure them that this is normal
- work on drawing for 30-50 minutes (depends on the complexity of the situation – e.g., one group they’d given the group 45 minutes, but they could see they weren’t done, so ended up giving them 55)
- then analyze!
- drawing itself is often unintelligible – those who made it can tell a compelling story of the picture, so ask the participants to explain their picture – “this is where the magic happens!”
- give 5-8 minutes per drawing
- audiotape the explanation
- don’t read into the pictures – get the explanations from those who drew it
- she doesn’t show the pictures in their reports – it’s the raw data that gets interpreted (not meant to be a product itself – it’s a way to get to the ideas/emotions)
- address context and dynamics when working in groups
- one audience member asked how “rich pictures” compare to “anecdote circles”, but the speaker and facilitator hadn’t heard of that before (and neither have I!), so I googled it (and it turns out that they are a completely different thing than rich pictures).
So, it looks like I’ve got some homework to do: reading up on Soft Systems Methodology, checking out these tools, and checking out the references on the last slide of the presentation slide deck.
Other webinars from this series that I attended:
- “Emergence: Detection and Interpretation in a Leadership Program” by Michael Quinn Patton
- “Cross Boundary Insights: Plans, Critical Incidents, and Outcomes” by Jonathan Morell
- “Systems Dynamics: Computer Models to Anticipate and Plan for Surprise” by Jeff Wasbes
View the slides here and view the whole presentation on YouTube.
