How Much Should New Evidence Move You?

Before leaving home, you put the chance of rain today at about 30%. Downstairs, the sky has darkened and the wind has picked up. Where should you move to? 40%? 80%? "It's definitely going to rain"?

Most people update on gut feel: big changes for news that sounds dramatic, none for news that sounds dull. But there is a principled way to decide how far evidence should move you.

The key question: how lopsided is the evidence?

Ask yourself two things:

• If it is going to rain, how likely am I to see a sky like this?
• If it isn't going to rain, how likely am I to see a sky like this?

The ratio of the two is the likelihood ratio. Say a dark sky shows up before 60% of rainy days but also on 20% of dry days. The likelihood ratio is 60% ÷ 20% = 3.

Do the math with odds

Write your 30% prior as odds: rain vs. no rain is 3 to 7. Multiply by the likelihood ratio of 3 and you get 9 to 7. Back to a probability: 9 ÷ 16 ≈ 56%.

So a sensible new estimate is around 56%: not unchanged, but not a leap to 90% either.

Same evidence, different starting point, different move

• Prior 50%, likelihood ratio 3 → odds 3 to 1 → 75%, up 25 points.
• Prior 90%, likelihood ratio 3 → odds 27 to 1 → about 96%, up only about 6 points.

The closer you are to 0% or 100%, the less a given piece of evidence can move you. And if a piece of evidence is about equally likely either way (a likelihood ratio near 1), it shouldn't move you much, however exciting it looks.

Try the sliders below: adjust the prior and the strength of the evidence and watch the new probability change.