Publish at April 17 2016 Updated July 02 2026
Backing up one’s arguments with numbers and statistical data apparently boosts credibility. And yet, misusing these mathematical tools can lead to the most erroneous claims...
A few websites help us avoid mistakes stemming from a poor grasp of statistics. They also present us with some paradoxes that challenge our intuitions with a few simple calculations.
The media regularly reports that eating a certain fruit or vegetable may reduce the risk of disease. Is it really that straightforward? People who consume antioxidants are less likely to develop cancer or heart disease. Does this mean that antioxidants reduce the risk of developing these diseases? Not necessarily, says Catriona Maclean, a mathematician specializing in geometry who is also passionate about statistics.
And to illustrate her point, she chooses to show us where identical lines of reasoning might lead. For example, we see that children whose parents own two cars perform better than average. Does that mean that cars make you smart? Should we take the kids for drives around the neighborhood instead of encouraging them to read and conduct experiments?
Unfortunately, newspapers routinely use this… logic. Aspartame supposedly causes miscarriages; a certain exotic fruit supposedly reduces the risk of disease. Catriona Maclean warns us: perhaps the people who consume antioxidants aren’t exactly the same as those who don’t. They tend to be from higher socioeconomic groups; they’re people who pay attention to the products they consume, and so on. They often engage in sports, and so on. And it may well be that the results obtained are more closely linked to this lifestyle than to antioxidants. We don’t know.
The only way to settle this would be to take two samples, selected at random, and give antioxidants to the first group and neutral products to the other, then follow them for several decades to obtain a result.
Statistical paradoxes abound. ... Catriona Maclean asks the group if they think it’s likely that two people in the room were born on the same day of the year. Everyone offers their own guesses, but most participants think it’s unlikely that, in a group of 50, two people were born on the same day. And yet.
This is the “birthday paradox,” an explanation of which is provided by the website Science Étonnante. For a group of 50 people, there is a more than 95% chance that two people were born on the same day. An explanation that relies less on statistical details is offered by “Curieux² Savoir.”
Simpson’s paradox was formulated by a statistician who gave it his name in 1951. A patient has the choice between treatment A and treatment B. Treatment A leads to a cure in 78% of cases, and treatment B in 86% of cases... And yet, treatment A is the more effective one. Why?
Once again, *Science Étonnante* will provide the explanation. For those who can’t wait, just imagine that treatment A is reserved for the most severe cases. It is rarely prescribed for mild cases, which have a success rate of over 90%, but is frequently prescribed for severe cases, where the success rate hovers around 75%. Treatment B is almost exclusively prescribed for mild cases. Although less burdensome for the patient, it is less effective. Thus, because Treatment A is more effective, it is used for the most serious cases, where success is less likely... The blog“Mole.net” offers a more mathematical explanation of this paradox.
Other errors stem from the samples. They may be too small, resulting in findings with a confidence interval that is too wide.
The sample may not be representative. Pollsters survey people they anticipate will agree to respond, for example. Self-administered online surveys, which are answered only by those who choose to do so, provide a stark illustration of what a sampling error is. Only motivated internet users respond… and the most motivated ones respond multiple times. Influencing the poll results is sometimes their main motivation!
Catriona Maclean poses another question. A disease affects one in every thousand people on average. The test that detects it is 90% reliable. Maclean tells us that we’ve tested positive. Should we stay calm, be concerned, or panic?
Charlatans.info also asks this question and provides the information needed to answer it.
Out of 1,000 people, 10 are affected. Statistically, 9 will be detected, and one will go undiagnosed by mistake. And out of the 990 healthy people, about 99 will be diagnosed, or 10%. So out of 108 people (9 + 99), we’ll have 9 who were correctly diagnosed… or 8.9%.
I don’t know about you, but I feel better!
Catriona Maclean and the participants in her training session offer explanations for these statistical errors. First, it’s a cultural issue. Statistics are rarely taught, and some journalists or scientists aren’t particularly interested in the mathematical methods that lead to the results. Their readers aren’t any more interested, for that matter.
More seriously, some researchers are determined to prove a result at all costs, so they’ll only use data that supports their thesis. All it takes is having hundreds of studies on a topic, and statistically, you’re bound to find some that are more favorable and that you can rely on.
illustrations: Frédéric Duriez
Sources
David Louapre on *Science étonnante*: The Birthday Paradox
https://sciencetonnante.wordpress.com/2012/05/28/le-paradoxe-des-anniversaires/
Raghi on Mole.net Statistical paradoxes are more common than you might think
http://blog.m0le.net/2014/06/14/des-paradoxes-statistiques-plus-repandus-quon-ne-croit/
Charlatans.info: “Statistical Pitfalls,” accessed April 15, 2016 – Website no longer available (2026)
You might also like:
Bonds in human groups endure over time according to a variety of strategies, but AI is reshuffling the deck. Lasting human groups share more than data...
Contemporary approaches to facilitation in the context of collective intelligence
Trends in collective intelligence facilitation are marked by AI, new managerial roles, diversified collective approaches and formats, and the institutionalization of research.
How Has the Pandemic Changed Our Language?
Has the pandemic and its invisible barriers changed our approach to language? Who are the most affected? For how long? Has containment changed the way we learn and speak?
Wise use of myopic and monomaniacal A.I.s
In the world of artificial intelligence, anthropomorphisms aren't very popular: we prefer "narrow-minded" to "mono-maniacal" and "focused" to "myopic". Where do we place ethics in the practical use of artificial intelligence? Where else but at school could we learn to do so?
Self-evaluation, a positive reflective exercise
Self-assessment raises a number of questions. Is a learner in the best position to analyze his or her understanding or that of others? Yet, this metacognitive exercise has many benefits and can become one of the many assessment tools used in a course.
