When our biases disarm us: the illusion of the unpredictable

Remember the Subprime crisis and the economic cataclysm that followed? Most experts exclaimed, "It was unpredictable!" Yet, in retrospect, the warning signs were there, hidden by our own judgments.

This feeling of surprise at such an event is not just the result of chance! It often stems from our cognitive biases, those mental mechanisms that help us navigate through everyday life but blind us to break-ups. In this article, I propose to explore how these biases transform potentially anticipatable events into unexpected shocks, drawing on philosophical, economic and technological concepts.

Similarities and identities

The "tyranny of similarity" refers to our tendency to project the past into the future, assuming that tomorrow will resemble yesterday. Our biases and preconceptions are not isolated errors; they are evolutionary acceleration tools, designed to simplify a complex world. They enable us to make quick decisions, but in a volatile environment, they become blinkers.

For example, we see time as a straight, predictable, linear line, whereas it is punctuated by unexpected breaks - the Black Swans described by Nassim Nicholas Taleb in his book The Black Swan. These rare and significant events defy the Gaussian curve and the architecture of our way of operating, revealing our blind spots.

Our need for security leads us to ignore uncertainty but, as Taleb points out, the unpredictable can happen even if, in theory, the probability of occurrence is almost zero. Yet in retrospect, we can see a logic in these events, once we've abandoned our rigid reading grids.

Turkey syndrome: when the past lies

Let's dive into a striking analogy from thework of Nassim Nicholas Taleb. Imagine a turkey raised on a farm. Day after day, for 364 days, it is fed, cared for and protected by the farmer. Each morning reinforces its confidence: the world is predictable, benevolent and the future looks bright. Her similarity grid, based on repeated experience, dictates that tomorrow will be like yesterday. But on Day 365, the eve of Thanksgiving, the farmer shoots him. The shock is total for the turkey, but predictable for the outside observer.

This story illustrates the turkey syndrome, where the past induces a mistaken belief in eternal continuity. Taleb uses this metaphor to criticize our dependence on the rear-view mirror: we extrapolate historical patterns without considering possible ruptures. Hindsight bias comes into play here: once the event has occurred, we rewrite history to make it coherent and predictable. "Ah, of course, it was obvious!" we say, forgetting our initial blindness.

"The problem with experts is that they don't know what they don't know", says Nassim Nicholas Taleb in The Black Swan.

Narratively, the contrast is poignant: the turkey on day 364 radiates certainty, surrounded by accumulated evidence of safety. On day 365, everything collapses. This reflects real crises, such as the market crash of 2008, where years of growth masked systemic risks and signs, including mortgage valuations that far exceeded real value. Our biases reassure us, but they lie by obscuring real uncertainty, as opposed to simple calculable probability.

• Advantage of biases: They speed up decision-making in normal times.
• Risk: They create illusions of stability, making Black Swans all the more devastating.

Blind optimization: when performance kills resilience

The economic and societal world in which we live is obsessed with efficiency, and we optimize everything: finances, health, education. But this quest for maximum return is eroding safety margins, those essential cushions for absorbing shocks. According to an OECD report on systems thinking for public policy, over-optimization weakens systems by eliminating redundancies. In calm times, this works, but in times of crisis, it becomes a death trap.

Take the example of global supply chains: to minimize costs, companies reduce inventories to a minimum. This boosts short-term performance, but an unforeseen event, such as the COVID pandemic, reveals the weaknesses. The OECD warns against this "blind optimization", which prioritizes efficiency over resilience. Our biases lead us to see the future as a linear extension of the present, ignoring systemic blind spots.

This optimization analysis grid is suited to stable periods, but it kills the ability to adapt. In finance, models based on historical data underestimate extreme events. The OECD advocates systems thinking to integrate uncertainty, cultivating margins that absorb rather than deny disruptions.

1. Identify vulnerabilities: assess potential breaking points.
2. Building redundancy: adding margins for survival.
3. Adopt a holistic vision: beyond immediate efficiency.

Optimization, so seductive, makes us vulnerable to the unpredictable, turning manageable risks into disasters.

AI or the industrialization of the rear-view mirror

Artificial intelligence (AI) embodies the apogee of our biases: trained on past data, it excels at recognizing similar patterns, but fails when faced with the unprecedented. As Microsoft's research on optimization with uncertainty underlines, classical algorithms work with complete inputs, but in evolving environments, they have to deal with uncertainty. AI, the champion of similarity, industrializes the rear-view mirror by projecting history into the future.

The problem of "Out-of-Distribution" (OOD) is central here. Think of AI as a driver who knows only known roads. So, faced with an unknown path, it forces entry into its existing maps rather than admitting "I don't know".

A study by Google Research explores the limits of OOD detection, showing that even models pre-trained on large datasets, such as Vision Transformers on ImageNet, improve performance but struggle to identify truly novel scenarios.

Pedagogically, AI reinforces our biases by offering an illusion of probabilistic precision. It calculates risks based on history, but ignores uncertainty, which cannot be quantified. Microsoft focuses on frameworks for evaluating algorithms in stochastic [chance-dependent] environments, while Google suggests exposure to a few examples to improve detection.

• AI benefits: accuracy in known scenarios.
• Risks: amplification of human blind spots, leading to costly errors in healthcare or finance.

Ultimately, AI doesn't invent the future, it recycles the past, inviting us to question our dependence on these tools.

Towards intellectual vigilance

Managing our biases doesn't mean eradicating them, because that's impossible, but cultivating a cognitive humility that accepts the incompleteness of our perception and analysis grids. Instead of aiming for perfect prediction, let's prioritize resilience: let's build flexible systems, ready to absorb the unpredictable. Taleb, the OECD and research by Google and Microsoft all agree on this point: excellence lies in preparation for error, not in the illusion of mastery.

In conclusion, our biases reassure us in times of peace, but disarm us in times of upheaval. By adopting an intellectual vigilance, we transform the illusion of unpredictability into an opportunity for adaptation. The future is not to be foreseen in its entirety, but faced with an open mind. Let's cultivate this posture to navigate a world of Black Swans with greater wisdom.

References

Nassim Nicholas Taleb - The Black Swan - https://fr.wikipedia.org/wiki/Le_Cygne_noir_(book)

Systemic Thinking for Policy Making - William Hynes, Martin Lees and Jan Marco Müller - OECD
https://www.oecd.org/content/dam/oecd/en/publications/reports/2020/02/systemic-thinking-for-policy-making_a95b3226/879c4f7a-en.pdf

Optimization with Uncertainty - Microsoft
https://www.microsoft.com/en-us/research/project/optimization-with-uncertainty/

Exploring the Limits of Out-of-Distribution Detection - Stanislav Fort, Jie Jessie Ren, Balaji Lakshminarayanan
https://research.google/pubs/exploring-the-limits-of-out-of-distribution-detection/