Outside First, Then Inside: Start From the Base Rate and Adjust

The psychologist Daniel Kahneman once told a story from his own work. A year into writing a high-school textbook with a team, everyone estimated it would take about two more years to finish. Then he asked the curriculum expert on the team: “How long do teams like ours usually take?” The expert thought for a moment and said that about 40% of such teams never finished, and those that did took 7 to 10 years. He was taken aback himself — he had also just guessed about two years. The book ended up taking 8 years.

Everyone on the team knew their own project inside out, but nobody had thought to ask how projects like it usually go.

The two-step method

Good forecasts usually take two steps:

• Step one, outside: find a suitable reference class, write down the base rate, and use it as your anchor.
• Step two, inside: look at what’s special about this case and nudge the number up or down.

Order matters. If you start with the details, a story forms in your head first, and a base rate you look at afterward tends to get a polite glance without actually changing your mind.

How far to adjust

An example: a friend is renovating, and the contractor promises it’ll be done in 3 months. Will it be?

• Outside first: suppose that among renovations you know of, only about a third finished on schedule. Starting point: 30%.
• Then inside: your friend hired a well-established crew with a great reputation, and the apartment is small. Both help, so moving up to around 40% is reasonable.

The size of the adjustment should match the strength of the evidence. “The contractor says it’ll definitely be fine” is something every contractor says — it shouldn’t move you from 30% to 90%. Only strong, specific evidence, like “the main work is done and only the furniture is left,” justifies a big jump.

Using it in forecasts

When you create a forecast, fill in the “Base rate” field first, then your reasons for and against, and only then your probability. If your Brier score keeps coming out high, this is often the first habit worth changing.