Snapshot

Corporate counsel can help clients by bringing an independent perspective and calling out potential survivorship bias.

What is the model?

The ‘survivorship bias’ mental model suggests that we flawed humans tend to consider only successful outcomes in a representative sample, which leads us to over-estimate the chances of our own success.

The easiest way to demonstrate and remember this bias is with the ‘returning bombers fallacy’ — unfortunately I can’t remember where I read this example initially but it’s certainly not my own.

The story goes as follows: In World War II, some very smart designer was asked to imagine a better bomber for the Air Force. The Air Force presented him with their research, including photographs of bombers that had returned from air raids over Germany. The photos showed damage caused by German anti-aircraft defences, and the Air Force told the smart designer that any aircraft she came up with would need additional armour in those spots - because clearly that’s where the aircraft often get hit.

The smart designer saw through the survivorship bias, though. There was no need for extra armour in those spots, she told them, because the photographs proved that aircraft could be shot in those areas and still make it home safely. It was those parts without any damage that needed the extra armour - none of the aircraft that were damaged in those parts made it home at all.

The Air Force was using ‘returned aircraft’ as their sample group, when really they should have used ‘departing aircraft’.

How does the model apply to corporate counsel?

Corporate counsel can add value by identifying survival bias and protecting decision makers from relying on skewed assumptions.

For example, when reviewing due diligence attached to a new business case, consider whether the market analysis considers only existing market players. That is, has the team overlooked threats that have taken competitors out of the market entirely?

Or when a counter party offers a track record of performance to justify an onerous contract. Has that track record been carefully manicured to be presentable? Is your consultant going to go missing should your matter start to look a bit dicey?

And we’ve all been told before how something might be the ‘market standard’, but subscribers will be alert to just which market is the relevant one. Agreements released to the open market might contain clauses very different to those that settle privately.

Be on the lookout for..

Any time someone looks to use a representative sample to support their argument, we should automatically consider the potential for survivorship bias.

The tricky part is that people rarely refer to their ‘representative sample’ in the real world. They say things like ‘every customer we spoke to thought..’ or ‘we’ve done this plenty of times before and it has always worked out.’

If we’re clever enough to pick out a sample being used in the wild, we should consider: if I was trying to make the same point that’s being suggested to me now, what evidence would I collect to support it?

And if that’s not the evidence being provided, we should be very cautious.

Where to read more..

The Survivorship Bias Model is covered in Rolf Dobelli’s excellent book, the Art of Thinking Clearly. You can also read more about survivorship bias at the Farnam Street Blog.

Thanks

Thanks very much for subscribing to and (hopefully) reading defined terms.

There’s a couple of questions we’d love your feedback on:

• Would you prefer one consolidated email (say, once or twice a month) with multiple features in it or would you prefer each feature to be sent as its own email once it’s ready?

• Is there any content you would be particularly keen for us to create?

• Is there anything we could do to make the life of the legal counsel easier?

You can send us your thoughts just by replying directly to this email :)

And if you’re enjoying defined terms, please consider recommending it to your colleagues. They can sign up at this link and they can read past features at this link.