In last week’s newsletter, we looked at using short cycles for learning quickly at the individual, team, and org levels. This week, we’re taking a deeper dive into designing good org change experiments (or probes, in Cynefin terms).
Most orgs approach org change like this: leaders do analysis and planning, convince themselves the change will be successful, and then manage a big rollout. But since org change is a complex, human problem, this approach rarely works smoothly. Instead, we favor an experimental approach to org change.
When is an experiment the right move versus just making a change? Use an experiment when:
- Your change depends on risky assumptions—i.e., things that you need to be true for your change to be successful, but that might not actually be true.
- Being wrong would be expensive—not just in terms of money but also time, morale, credibility, etc.
The most common source of risky assumptions is the future preferences and behaviors of people. It’s notoriously difficult to predict how people will behave in the future (including ourselves; see New Year’s resolutions!).
So, how do you surface the assumptions, find the risky ones, and design tests for them?
Surface Assumptions
Start by brainstorming the assumptions behind the change you’re considering. This can be tricky because assumptions, by their nature, seem obvious to us. This makes them hard to see.
Our favorite way around this is to use resistance as a resource. Imagine announcing your planned change today. What does the resistance sound like? Capture the quotes you might hear.
-
“Why do we need to change? Things work fine now.”
“I don’t think that change will produce the effect you want.”
“That might work, but it’ll cause [some other bad thing].”
“We don’t have budget for that.”
“We’re too busy to make this change right now.”
“This’ll delay project XYZ too much.”
“Senior management will never let us do this!”
Now, let each quote reveal the things you’re assuming but that the imaginary resistor doesn’t believe. For example, the objection, “Why do we need to change? Things work fine now,” suggests assumptions like, “We believe X problem exists,” and, “We believe X problem causes Y negative effect today.” The objection, “That might work, but it’ll cause [some other bad thing],” reveals the assumption, “We believe this change will not cause [some other bad thing],” or, “We believe we can prevent [some other bad thing] from happening,” or even, “We believe [some other bad thing] isn’t actually that bad and doesn’t need to be prevented.”
Once you’ve brainstormed resistance quotes and the assumptions they reveal, look over the assumptions to identify ones worth testing. Which ones are risky and would be expensive to be wrong about? Those are the ones to consider testing.
Design an Experiment
So, you’ve decided an assumption is worth testing, how do you design an experiment? This can (and does) fill whole books, but broadly, you’re looking for something quick and cheap (relative to making the whole change) that could prove your assumption wrong. Or, to put it differently, something that could convince you that your assumption is wrong.
“Wait a minute,” you may be thinking, “why do I want do prove my assumption wrong? Don’t I want to validate my assumption?” Confirmation bias is why. Our brains naturally notice the evidence that validates our existing beliefs, and we’re largely blind to evidence that falsifies our beliefs. A good experiment is able to stop you from wasting money following a bad assumption. So, it needs to be able to falsify that risky assumption quickly.
In our experience, a good experiment has 5 parts:
- A clear statement of the hypothesis
- A description of the test you’ll run
- What you intend to measure
- The threshold for concluding your hypothesis is true
- The threshold for concluding your hypothesis is false
Let’s look at an example experiment…
-
Change proposal: We should adopt agile!
Key Assumptions Include:
We believe…
…our current approach isn’t providing the results our business needs.
…an agile approach would produce better business outcomes.
…an agile approach will be better for the people doing the work.
…we can make an agile approach work well in our unique context.
…key stakeholders will support the change.
etc.
Any of these could warrant an experiment, but the one that seems hardest to resolve from others’ experience is, “We believe we can make an agile approach work well in our unique context.” That one would benefit from a real test. Here’s one way you might structure it:
Notice how this is not only different from the big rollout approach, it’s also much more deliberate than the typical “just do a pilot and see if it works” approach.
Also note the gap between the validation and invalidation criteria. We try to set validation criteria so they’ll convince the skeptic that the hypothesis is true and the invalidation criteria so they’ll convince the advocate that the hypothesis is false. If the measures land in between, the experiment didn’t produce a strong enough result—you may need to run another test.
By the way, check out our episode on how to choose a pilot team if you’re considering this kind of experiment.
Here are a couple of other example org change experiments…
Of course, the details would need to be tuned for a particular situation, but these illustrate the typical shape of an org change experiment.
To help you create your own org change experiments, we’ve created a Probe Design Cheat Sheet. Click on the image to the right to download a copy!
Want to learn more about designing org change experiments? In our Leading Organizational Transformation workshop, we spend a half-day just on experiment design. We also explore what changes are worth making if you want a more human-centric, high-impact org; what leaders should focus on in this kind of org (and what they shouldn’t do); as well as the personal leadership development required to lead meaningful change. Our next public workshop is August 7-10. Use coupon code, ORGDESIGNCHANGE, for $200 off!
Last updated