I Am Not an Optimist

I’m with statistics guru Hans Rosling. “I am not an optimist,” he said in the New Year’s Eve edition of More or Less on BBC Radio 4. “I am a probabilist. I can see that progress in the world is possible, and even further and better.” He is displaying an important trait shared by skilled decision analysts and thinking in terms of possibilities and ranges, rather than fixating on a single point forecast of the future.

If you are in search of meaningful and reliable information to support an important decision, it is crucial first to admit to a range of possibilities, preferably a wide range. The failure, for example, to allow even for the possibility that US house prices might go down, let alone attach any probability to the event, was a significant contributor to the recent near-meltdown of the global economy.

Statements of optimism and pessimism should then be cast in terms of the probabilities you assess for the ends of the range. It is hard to be absolute as to what optimism (or pessimism) means. Someone can only be more optimistic (ie, a relative comparison) than another if she assesses a higher probability on the upside than the other (and of course a lower probability on the downside).

I have often had to assess an important input variable where there were two alleged experts with vehemently opposing views about the forecast. The trick has always been to get the experts together in the same room and get them to explain the reasoning behind their forecasts. It has always turned out that the two experts believed the same thing about the variable under discussion, but each was making different assumptions about other related variables. Decompose the assessment and make the underlying assumptions explicit, and it was a lot easier to get agreement. They still disagree as to which scenario will occur, but they agree about the probability distribution that governs it.

See also:

• More or Less is a great programme, often debunking some of the scandalous misuse of statistics by our politicians and media. It is presented by Tim Harford.
• Hans Rosling runs GapMinder. This is a great site with some fabulous software, now available as the Motion Chart gadget from Google, for displaying global development statistics and how they trend over time. Check it out! (And Gapminder’s documentation for Motion Chart.)
• Rosling is a frequent (and enthusiastic!) speaker at TED. He also presented the recent BBC TV programme The Joy of Stats which contained the following 4-minute presentation of 200 years of global development.