Are We Preparing for an AI Black Swan - 
Or Preparing the Public for One?

A few days ago, a headline caught my attention.

Leading artificial intelligence companies, including OpenAI and Anthropic, were reportedly preparing for the day after a potentially catastrophic AI incident.

Not simply how to prevent a serious event. But what happens afterwards.

The public reaction. 

The political consequences. 

The demands for answers. 

And perhaps the calls to shut down the very technology that so many people are now beginning to depend upon.

I found myself wondering about something. Are we preparing for an AI Black Swan - or are we also preparing the public to expect one? Because those are two different activities. And both may be happening at the same time.

I’ve Been Here Before…

In September 2026, I published a TFP Field Note titled What Could an AI Black Swan Actually Look Like?

Drawing on my earlier experience in corporate technology and cybersecurity marketing, I explored the kinds of serious incidents that might occur as AI becomes more capable and autonomous. The article examined eight possibilities, ranging from disruption to critical infrastructure and financial systems to misinformation crises and unintended behaviour by autonomous agents.

But the central question wasn’t whether artificial intelligence would destroy humanity. It was much more practical.

What does AI materially change about risks that already existed?

• Cybercrime existed before AI.
• Financial crises existed before AI.
• Fraud, sabotage, propaganda and human error existed before AI.

What artificial intelligence might change is the speed, scale, accessibility, adaptability and autonomy with which certain activities can occur.

I made another distinction in that article. Strictly speaking, many of the risks we describe as black swans are no longer completely unforeseen. Once an event is being studied, modelled and prepared for, it is better described as an emerging risk or a potentially severe event whose probability and consequences remain uncertain.

And I wrote:

‘Preparedness is not prediction. A fire alarm does not mean the building is burning’.

Now, a month later, that distinction feels particularly relevant.

The Day After

On 9 October 2026, Axios published an investigation reporting that executives at major AI companies were considering how to respond to the public and political consequences of a catastrophic AI event.

The scenarios included serious cyber incidents potentially disrupting financial services, communications or essential infrastructure. The article also described discussions about how governments, lawmakers and the public might react.

OpenAI confirmed that it conducts preparedness exercises involving different scenarios, while making clear that those scenarios are not treated as inevitable. That qualification matters.

Some researchers and executives reportedly believe a serious incident is inevitable. But their expectations are not the same as demonstrated certainty about what will happen, when it will happen or how serious it might be.

I have previously said that I personally expect a significant AI-related incident at some point. That’s my assessment of the direction of technological development. I don’t know its scale. I don’t know when it might happen. And I certainly don’t know whether it will resemble the dramatic scenarios circulating in today’s headlines. But I think preparation is sensible. The question is how that preparation is communicated.

When Risk Management Becomes a Narrative

In my earlier cybersecurity work, the objective of risk communication was rarely to terrify the people sitting around the boardroom table. It was to help them understand consequences.

• What might go wrong?
• How likely is it?
• What controls are already in place?
• Where are the weaknesses?
• Who is responsible?

And what happens if the safeguards fail?

Those questions remain relevant today. But the public conversation around AI has another dimension – Emotion!

Headlines compete for attention. Fear, urgency and uncertainty can make a story feel more important, more immediate and more compelling.

An article explaining that researchers are conducting controlled evaluations may attract less attention than a headline suggesting that autonomous AI is preparing to escape. Both might refer to the same underlying research. Yet they can leave the reader with very different impressions. And that’s where I begin to wonder about the consequences of the narrative itself.

Are We Becoming Conditioned to Expect Disaster?

I don’t mean that somebody has necessarily designed a coordinated campaign to frighten the public. There is no evidence presented here that establishes such a campaign. But repeated exposure to alarming predictions can influence how people perceive risk. Consider two messages

Message One: Researchers are testing increasingly capable AI systems to identify potential vulnerabilities and improve containment.

Message Two: Scientists warn that dangerous AI could escape and cause catastrophic disruption.

The second message may be based on legitimate research. But it also introduces a much stronger emotional response. And if people encounter similar warnings repeatedly, a future incident might be interpreted through the expectations those headlines have helped establish.

Perhaps that would encourage sensible preparedness.

Perhaps it would create unnecessary anxiety.

Or perhaps it could make the public more willing to accept sweeping interventions without fully examining their consequences.

Those possibilities deserve investigation. We shouldn’t confuse them with proven outcomes.

The Other Side of the Story

There is a danger in becoming too sceptical of frightening headlines. Sometimes the warnings are justified. And sometimes early intervention prevents a serious incident precisely because somebody raised the alarm.

The history of cybersecurity, aviation, medicine and infrastructure safety contains many examples of organisations preparing for events they hope will never occur. Responsible contingency planning is not evidence of a hidden agenda. Nor should companies be criticised merely for asking how they would respond if something went badly wrong. Indeed, I’d be more concerned if organisations developing increasingly powerful systems were making no preparations at all.

The important questions concern the quality, independence and transparency of those preparations.

• Are safeguards being tested?
• Are failures reported?
• Can external researchers challenge the findings?
• Are emergency controls reliable?

And do the relevant authorities understand what they can actually stop?

What Happens to Public Trust?

Here is where I think the October reporting introduces something different from my September article. My earlier investigation focused on the mechanisms through which a serious AI incident might happen. The newer reporting draws attention to what might happen after public confidence has been shaken.

Imagine a major disruption. Perhaps a bank temporarily loses access to essential systems. Perhaps a hospital or telecommunications provider experiences a serious cyberattack involving AI-assisted tools.

Perhaps investigators initially disagree about whether the incident arose from malicious human activity, unintended autonomous behaviour or ordinary security failures. While the technical investigation continues, headlines spread. Politicians demand explanations. Social media fills with speculation.

Some people insist that AI must be shut down immediately. Others argue that the incident has been exaggerated to justify tighter regulation. And organisations dependent on AI begin asking whether their own systems are safe.

Now we have two potentially interacting problems:

The original technical incident.

And the public response to what people believe has happened.

The second could amplify the consequences of the first. Especially if inaccurate information spreads faster than investigators can establish the facts. But a well-informed public response could also strengthen accountability, accelerate necessary safeguards and prevent further harm. The direction is not predetermined.

Who Controls the Response?

This brings me back to two other Field Notes I published on 12 October: Is an AI Kill Switch Possible?

And: As AI Becomes More Autonomous - Who Will Keep Watch?

I explored whether autonomous systems could be reliably monitored, interrupted and contained, and whether independent AI oversight might help humans recognise problems before they escalate.

My hypothetical Queen Bee would not govern AI. She would examine evidence, identify unusual behaviour, assess possible consequences and alert accountable humans. But the October developments add a further question.

Who oversees the response when an AI incident becomes a matter of public confidence?

• A technology company might have the authority to suspend its own service.
• A government might have legal powers to intervene within its jurisdiction.
• Independent experts might investigate what occurred.
• The media might report emerging findings.

But these parties do not necessarily have the same responsibilities, incentives or access to information. And they may disagree.

The challenge is not simply to communicate quickly. It is to communicate accurately when the evidence is incomplete and the pressure to provide answers is enormous.

Could Preparing the Public Be a Good Thing?

Yes. Public education can be an important part of resilience. People benefit from knowing how to recognise scams, verify information, understand technology’s limitations and respond sensibly to disruption. But education and emotional conditioning are not the same thing.

Education explains the evidence, uncertainty and available choices. Fear-based messaging can encourage people to reach a conclusion before they understand the underlying risk.

And reassuring messaging can be equally problematic if it creates unjustified confidence.

Neither extreme serves human discernment. I don’t think the answer is to stop talking about serious AI risks. I think it’s to become more careful about how we talk about them.

TRUE · FALSE · 📌 PIN IT:

TRUE:

Major AI companies are conducting or considering preparedness exercises involving serious AI-related incidents and their possible consequences.

Axios reported on 9 October 2026 that industry executives were discussing not only technical failures but also the potential political and public response.

AI can increase the speed, scale or sophistication of certain existing cyber and information-security threats.

Reporting on possible catastrophes can draw public attention to legitimate safety questions.

FALSE:

The existence of emergency planning proves that an AI catastrophe is imminent.

A prediction made by an AI executive or researcher establishes when a major incident will happen.

A dramatic headline is, by itself, evidence that a worst-case scenario has occurred or is inevitable.

The reporting examined here proves that technology companies or governments are deliberately coordinating a campaign to frighten the public into accepting future controls.

📌 PIN IT:

Are repeated warnings changing how the public perceives AI risk?

Could expectations of catastrophe influence the political response to a future incident?

Will crisis communication distinguish between confirmed facts, emerging evidence and speculation?

Can independent investigators communicate their findings without undue commercial or political influence?

And how do we prepare society for genuine risks without making fear the dominant story of technological progress?

The Fifth Beat - Consequences

There are benefits to preparation. Earlier detection. Stronger safeguards. Better incident-response planning. More resilient infrastructure. Clearer responsibilities. And perhaps greater public confidence when people know that serious risks are being investigated rather than ignored.

But there are possible harms too. Repeated catastrophic messaging could heighten anxiety, encourage distrust or create the impression that disaster is unavoidable. An actual incident could trigger demands for disproportionate intervention.

Equally, dismissing warnings as sensationalism could leave organisations and the public dangerously unprepared. And we must consider another consequence.

What happens if the response to a crisis causes additional harm?

An indiscriminate shutdown could interrupt essential services.

Overly broad restrictions could affect legitimate researchers and businesses.

Poorly supported accusations could damage trust in institutions that are trying to help.

Yet delaying necessary intervention could allow a genuine threat to spread. There is no universal answer.

The appropriate response depends on the evidence, the systems involved, the available safeguards and the consequences of acting, or failing to act.

That is why the Fifth Beat matters.

Reflection

I find myself returning to the same question that began my September investigation:

Are we looking at something genuinely new?

Or are we looking at familiar human behaviour responding to a powerful new technology?

Risk planning is familiar.

Political positioning is familiar.

Corporate reputation management is familiar.

Sensational headlines are certainly familiar.

But AI introduces capabilities that could change how quickly events unfold, how much information can be generated and how difficult it becomes to distinguish genuine evidence from convincing fabrication.

That combination deserves serious attention. And perhaps the public’s greatest protection isn’t simply being told that safeguards exist. It’s understanding what those safeguards actually do, where their limitations lie and who is accountable when they fail.

I want AI to succeed. I also want the people developing, deploying and governing it to be answerable for their decisions. Those wishes are not contradictory.

Reader Reflection

When you see a headline predicting an AI catastrophe, what happens first?

Do you feel frightened?

Sceptical?

Curious?

Do you share it, dismiss it or investigate the evidence?

And if a serious AI-related incident occurred tomorrow, who would you trust to explain what had actually happened?

• The technology company?
• The government?
• Independent scientists?
• The media?

Or would you want to compare their accounts before deciding?

Perhaps that’s where human discernment becomes most important. Not when everyone agrees. But when events move quickly, emotions rise and certainty is in short supply.

Preparing for the worst can be an act of responsibility.

Preparing people to understand the risks can be an act of public service.

But preparing people to fear an outcome before the evidence has earned that conclusion is something else entirely.

Perhaps the real question isn’t whether an AI Black Swan is coming. It’s whether we’ll retain the discernment to recognise what has actually happened if one does.

TRUE · FALSE · 📌 PIN IT

Truth rarely shouts. Discernment begins when we learn to pause.

The book is closed. The conversation continues…

Research Receipts

1.      Gill Malfin - 15 September 2026. What Could an AI Black Swan Actually Look Like? The original TFP Field Note examining eight potential scenarios and distinguishing preparedness from prediction.

2.      Axios - 9 October 2026. Scoop: AI companies plot “the day after”. Reporting on contingency planning, industry expectations and possible public and political reactions. Includes OpenAI’s statement that its preparedness scenarios are not treated as inevitable.

3.      Reuters - 13 October 2026. Behind AI industry’s call for independent reviews, a fight over what gets made public. Examination of tensions surrounding evaluator independence, commercial confidentiality and public disclosure.

4.      Reuters - 12 October 2026. UK given access to safety-test latest AI models, minister says. Reporting on independent model evaluation through the UK AI Security Institute.

5.      NIST - AI Risk Management Framework. The Govern, Map, Measure and Manage functions as a practical foundation for understanding and responding to AI-related risk.

Continue the Conversation

This Field Note explores a single question. The book: TRUE • FALSE • PIN IT explores the wider framework behind questions like this - helping us navigate an age where technology, opinion and human judgement increasingly overlap.

The goal isn’t to tell you what to think. It’s to help you become more confident in deciding for yourself.