As part of EvalYear 2015, the School of Public Affairs at the University of Colorado Denver, is holding a webinar series called: “Practical Applications of Systems to Conduct Evaluation: Cases and Examples“. They have a great list of 30 minute webinars running from January to June related to use of systems thinking in evaluation. Systems thinking is something I’ve been reading up on lately, so these webinars are very well timed from my perspective!
Here’s the notes I took during the webinar 1Technically, I took these notes after the webinar as I experienced a technology problem and the site wouldn’t let me log on during the session. I viewed the recording later.:
Systems Dynamics: Computer Models to Anticipate and Plan for Surprise
Speaker: Jeff Wasbes
Date: 12 March 2015
- computer simulations can help us anticipate surprise
- not talking about statistical models, but rather talking about dynamic computer models that help us to understand how structure of a system affects its behaviour
- endogenous = arising from within
- system dynamic models are “rich with feedback loops”
- when you examine a system deeply enough, you can find “leverage points” (places where making small changes –> big results)
- when you find “leverage points”, you can use them (change the structure of the system at those points to get the results you want)
- “Models don’t need to be complex. Models need to be useful”
- don’t model a system for the sake of modeling a system – model a problem – make the model useful
- computer modeling is more cost (and time) effective than running a system in the real world
- three pieces of a system:
- elements
- interconnections (to link the elements to each other – with simple rules [e.g., an increase in A leads to an increase in B])
- function
- e.g. of systems – schools, cities, policies, programs
- elements pass information to each other through interconnections
- complexity – when cause-effect relationship change and adapt (and what causes these changes/adaptions often arises from within – feedback loops)
- feedback behaviour is dynamically complex
- combinatory complex – has lots of parts, but linear interactions – relatively easy to predict
- dynamically complex – nonlinear interactions – difficult to predict
- models help us separate the signal from the noise
- why are systems relevant to evaluation?
- programs and policies (which are complex systems) are implemented into complex environments
- sometimes what we think is a solution can actually be exacerbating a problem
- without understanding the system, the feedback loops, etc., we might be causing unintended consequences
- models help us communicate about complexity
- many social systems are in a “comfortable equilibrium” – stable, tends to go back to that equilibrium when perturbed
- having an understanding of how a system might be expected to work can help us to, for example, decide what/when/how to measure things
- example: let’s say you have a system with a feedback loop between effort and effect, such that you put in effect, see an effect, you relax your effort (because you saw the effect), effect drops a bit, so you increase effect – result: an oscillation in the result around the level that you want it to be:
- but let’s say you only take a measurement of the effect every 6 months:
if you don’t know about the underlying oscialation pattern, you’ll see this:
- which will lead you to quite different conclusions than what is really going on:
- so modeling the system to get an understanding of what might be the underlying pattern can be useful in helping us to think through our results and our measurement plans
- stakeholders should be involved in the development of the model and a model should be seen as a “living” entity that is subject to change as you learn more Vensim.com – a free software for personal or academic use (or license for commercial use) to do systems modeling
Other webinars from this series that I attended:
- “Emergence: Detection and Interpretation in a Leadership Program” by Michael Quinn Patton
- “Cross Boundary Insights: Plans, Critical Incidents, and Outcomes” by Jonathan Morell
Images: screenshots from the webinar. View the whole presentation on YouTube.
