Lies, damn lies, and statistics

STEPHAN LEWANDOWSKY

ABC, The Drum Unleashed, 14 JANUARY 2011

http://www.abc.net.au/unleashed/43020.html

When Mark Twain uttered his memorable phrase, the science of statistics was in its infancy, and he could not have anticipated that statistics and data analysis would take centre stage in many of the 21st century's controversies.

Was Mark Twain correct?

No.

Properly executed statistics are, in the words of statistician W. A. Wallis, "a body of methods for making wise decisions in the face of uncertainty." Alas, the ease with which statistics can be abused for deception and manipulation challenges the public to remain alert and, indeed, alarmed.

When George W. Bush announced in 2003 that under his policy, "92 million Americans receive an average tax cut of $1,083," those numbers were not, strictly speaking, incorrect. However, they camouflaged the fact that some 45 million people would receive less than $100 tax relief, whereas the top 1% of income earners were gifted a whopping $30,127.

Lies, Damn Lies, or Statistics? Take your pick among the first two, but don't blame the science of statistics: President Bush "cherry-picked" a number (namely, the mean) that made his intended point while obscuring the full implications of his tax policy — this is the exact opposite of proper statistical practice.

Oh the joys of cherry-picking! Your Volvo happens to lose a fan belt and so all Swedish cars are poor quality. You don't like Collingwood supporters and you find one fan who served time for shop-lifting — what a club of hardened criminals!

And global warming is a hoax because it's snowing in England.

Really?

Of course not. Global warming is inescapably real despite the snow in England or the cold wave in India or that icicle in a Mongolian cave.

To understand why, one must understand the proper role of statistics, and in particular the difference between meaningless anecdotes and real data.

Pictures are often said to be worth a 1,000 words. Perhaps, but pictures are worse than useless when used in lieu of real data. The nether regions of the internet are awash with the photo of a U.S. submarine surfacing at the North Pole in 1959, which is taken to disprove global warming because … who knows, but it is definitely the FINAL NAIL IN THE COFFIN (all IN CAPS!!!!) of climate change for sure. The involuntary humour of this (approximately 132^nd) final nail is highlighted by the fact that the submarine surfaced on 17 March, which is before sunrise at the North Pole. While darkness rather curtails photographic opportunities, it apparently does not limit the appeal of random daylight pictures of sleek submarines posing against a backdrop of ice to disprove global warming.

The real data, incidentally, show that the Arctic icecap has shrunk by an area roughly equivalent to the size of W.A. since 1980.

Lest one think this idolatry of pictures is limited to internet worshippers, our very own national tabloid recently decorated its front page with the headline "Wong wipeout doesn't wash with locals." Having exhausted its intellectual prowess by the alliterative headline, the piece made reference not to data but instead boasted a picture of a bronzed Aussie swimmer, whose untroubled excursions to Bondi beach during the last few decades somehow put another (by then 154^th) final nail into the coffin of climate change.

The real data, incidentally, show that sea levels have risen some 4-5 cm globally in the last 15 years.

Silly pictures aside, what about the indubitable reality of the current snow, ice, and miserable cold in Europe and parts of North America? Does this not disprove global warming for the 185^thtime?

No.

Single events carry little information — one shoplifter tells you nothing about Collingwood fans and one broken fanbelt does not undo decades of quality manufacturing.

So what about the heat wave in Russia last year, does this not prove that the climate is warming?

No.

Single events carry little information — a single heatwave by itself tells us little more about the climate than a single blizzard. (Actually, forensic statistical techniques exist that permit more robust interpretation of single extreme events, especially when they are virtually unprecedented such as the heat on Victoria's Black Saturday, but that is carrying us too far afield).

Notably, the very same extreme observations can be legitimately revealing if they are considered not as anecdotes but as data. That is, instead of cherry picking one or the other observation, a proper statistical treatment requires that all extremes be considered together.

When this is done, the picture is unequivocal: The last 5 decades have seen an inexorable decrease in the number of record cold days and an equally steady increase in the number of record hot days — in Australia, there are now more than two record hot days for every one record cold day. If the climate weren't changing, there'd be the same number of each.

Yes, there are still record cold days, in Australia as well as in England or India or Mongolia — but when all the data are considered, heat records far outpace cold records. Indeed, in 2010, nearly 20 countries broke all-time heat records whereas at most one broke an all-time cold record (as of November, subject to confirmation by the World Meteorological Organization).

The real data show 20 heat records to 1 cold record.

That is data, not an isolated anecdote or a silly picture.

Yes, it is snowing in Britain and cold in Germany or wherever else, but cherry-picking those anecdotes in lieu of real data obscures what is actually happening on the planet, at considerable risk to our children and grand-children.

Cherry-picking data in support of one's hypothesis is to proper statistics as Enron is to proper accounting: It is ignorant at best and criminally fraudulent at worst.

The pernicious nature of cherry-picking sometimes reveals itself, innocently or otherwise, even in the way a question is phrased: The question, "Has there been significant warming since 1998?" is not a meaningful question — it's a cherry-picked question that seeks a cherry-picked answer which must therefore remain meaningless.

Why?

Because the questioner chose the year 1998. (Or 1995, or whatever year happens to make the desired point.) And whenever someone chooses anything in statistics, be it a date for a question or an extreme cold in England for a newspaper headline, then they are engaging in cherry-picking.

They are playing Enron with the data and Pravda with your mind.

Of course there has been significant warming during every decade since the 1970s. But that doesn't mean that some years aren't cooler than others, and one can always find a particularly hot year (such as 1998) that sticks so far above the trend line that a few subsequent years appear cooler until the trend catches up. These few years present a (short) dream come true for those who deny climate change for their own agendas or emotional needs.

In fact, if one chooses a brief-enough period, there will never be any "significant warming"— in the same way that you never gained measurable height from one day to the next during childhood. Miraculously, this lack of "significant growth" has never prevented a baby from growing into an adult.

Statistics is subtle and confusing, no doubt about it. That's why it takes a decade of study to become a true statistical expert.

So what should an alert and alarmed, but understandably confused, member of the public do in light of conflicting messages and statistical nonsense in the media? Fortunately, there is an easy answer, and it is one I report in a peer-reviewed paper that is about to appear in Psychological Science. All you have to do to avoid confusion about climate change is to look not at a picture but at a graph.

A graph of all the data, for the entire globe, and across all available years.

In my study, I showed people a graph with climate data and asked them to predict the future trend — in one condition the climate data were labelled as such and in the other condition they were presented as fictitious share prices. Regardless of condition, and regardless of people's attitudes towards climate science, the results of the study were crystal clear: Everyone — including the few individuals who deny climate science — recognized the trend for what it is, and everyone knew where it is heading; people's responses (averaged across 100 observations in each case) are shown by the red triangles in the figures below.

So your best defence against the cherry-picking Enrons of the internet and the picture-boasting Pravdas of the media is to look at the data yourself.

All the data, for the entire globe, and for all available years.

Like the 200 participants in my study, you will recognize the trend without equivocation.

So was Mark Twain right after all?

No.

Statistics, when done properly, provide a robust and revealing tool to understand reality. Statistics are anything but damn lies. There are only some damn liars telling untruths based on incompetent or malicious abuse of statistics.

Stephan Lewandowsky is a Winthrop Professor and Australian Professorial Fellow at the University of Western Australia and an award winning teacher of statistics..