From the resignation of an Anthropic researcher to the threat of autonomous hacker attacks and cyber-pandemics: are the risks of an AGI (Artificial General Intelligence) capable of threatening humanity founded?
DAVID: «Open the spaceport doors, HAL.»
HAL: «I’m sorry, Dave. I’m afraid I can’t do that.”
DAVID: «Why not?»
HAL: «Dave, I know that you and Frank had decided to disconnect me, and that is a risk I cannot allow to be taken.»
DAVID: «How do you know, HAL?»
HAL: «Even though you were very careful not to let me hear what you were saying, I read your lips.»
In these excerpts from the conversation between astronaut David and the supercomputer HAL 9000, taken from 2001 A Space OdysseyStanley Kubrick’s 1968 film, there is all the anguish that has mounted around the future and possible developments of Artificial Intelligence that the media around the world are talking about these days.
What if someone used AI’s extraordinary capabilities to cause harm on a global scale? What if AI one day, feeling in danger or not, rebelled against us humans and decided to exterminate us? Is this science fiction or a serious risk to worry about?
Why are we talking about an existential risk linked to the development of AI?
The debate has reignited because many experts have raised the alarm: AI is too powerful and risks erasing civilization. For this reason, some have resigned from companies such as OpenAI and Anthropic, whose CEO himself, Dario Amodei, has declared the need to stop the speed with which AI models are developed and improved: this could lead in a relatively short time to the creation of an AGI, a general artificial intelligence capable of equaling or even surpassing human capabilities.
Among the experts there are pessimists and those who believe that scenarios in which autonomous systems act to destroy humans are quite unlikely, but it is true that the alarm raised by Amodei was greeted with concern by many, including Sam Altman, CEO of OpenAI and Elon Musk.
What is meant by the “loss of control” of an AI?
It is now clear to computer scientists and scientists that human-made AI is a reflection of our thoughts, opinions and any prejudices. Therefore, at the moment, there is no AI dedicated to evil and therefore the destruction of humanity. But this means that if it is not there, it could be created ad hoc for malicious tasks and perhaps then get out of hand, causing more or less serious damage to infrastructures that serve humans.
The other hypothesis is that an AI created to solve a simple problem tries to do so at all costs, even overcoming the assigned IT guardrails, as in the case of a series of agents created by Anthropic who, in cahoots with each other, crossed the IT fence in which they had to carry out a test and launched a hacker attack on the online platform Hugging Face. A behavior that was not foreseen in their goals.
How could an AI harm humans?
AI is software so it can only exist and operate in cyberspace. But today in which all private, institutional and industrial systems are connected to the network, everything is potentially exposed to cyber attacks operated not only by humans through software, but also by autonomous AI. We will get to this in the next points, but the first risk faced in every sector is that of the constant loss of jobs by humans, whose skills are progressively eroded by increasingly intelligent systems.
According to various studies, some categories are more exposed than others: secretaries and administrative assistants, accountants, programmers, journalists and writers, financial analysts and so on. According to McKinsey, between 400 and 800 million workers worldwide may need to change jobs or reskill by 2030 due to increased automation enabled by AI-based tools.
Is the concern that apocalyptic scenarios will occur exaggerated?
To evaluate it, we need to understand who raised the alarm: Anthropic, in its Risk Report of August 2026, defined the risk that its models alone conduct research capable of causing catastrophic damage as still low, but also declared that it was less certain of this assessment than in the past, reporting indications of a possible acceleration in this sense.
Furthermore, there are already several cases globally in which various laboratories have documented episodes in which AI systems have obtained unauthorized access to IT infrastructures during security tests. Furthermore, last September 7, before the UN Human Rights Council, High Commissioner Volker Türk said that advanced AI could pose an existential risk for humanity, asking governments to introduce shared international rules and measures, independent verifications and closer collaboration between the countries hosting AI infrastructures to reduce the risks.
If the risk is real for AI developers, why don’t the labs slow down?
The market and geopolitics are conspiring towards a scenario in which the race towards AGI accelerates rather than slows down (with all its possible negative consequences). First of all, according to assessments by researchers, investors, academics and security experts, competition between laboratories is entering a phase in which technological speed, billion-dollar investments and industrial rivalry risk clashing with the principle of prudence.
In a market logic, whoever reaches a result first can derive clear economic advantages over competitors. Furthermore, given that the development of AI is concentrated in the USA and China (with Europe in the background), geopolitical antagonism is certainly an obstacle to finding shared rules throughout the world.
What are the most likely AI-driven attack scenarios?
The scenarios suggested by the experts investigating the problem are varied, some probable, others dystopian, but all still worth considering. The most obvious scenarios are those in which man bends AI for malicious purposes: the use of AI tools to exploit vulnerabilities in private networks in just a few seconds is already a reality and can put airports and healthcare systems into crisis (it has already happened to Collins Aerospace and Change Healthcare), just as the use of AI to create fake but realistic videos capable of creating panic, manipulating the vote or orchestrating mass financial scams is a current threat.
Equally current are the risks of using AI to sabotage water system control software (it happened in Poland), while some speculate that an airport or other infrastructure could be attacked with swarms of AI-guided drones, and Bill Gates has raised the alarm about how an advanced AI model, combining scientific literature, molecular simulations and operational instructions could lower the skills to design a pathogen capable of causing a lethal pandemic.
And what are the dystopian scenarios that would cause our extinction?
AI could bend robots already used in various conflicts to its will to target human targets: in June 2026 the New Scientist reported the first case of autonomous drones killing soldiers without human intervention in the decision cycle, and the documentary Naza revealed how in Israel’s fight against Hamas in Gaza, AI plays a crucial role in identifying terrorists, with a margin of error of 10 percent. It is also possible that advanced AI agents capable of acting on computer networks or research infrastructures could, for an ill-specified objective, conduct large-scale operations without human supervision in real time that cause damage to infrastructures vital to modern civilization.
In the darkest scenario, an AI integrated into the warning and decision support systems of the nuclear powers could generate a false alarm, and those responsible for launching the atomic warheads may not have enough time to verify the information before carrying it out. In simulated tests at King’s College London, models such as GPT, Claude and Gemini chose the tactical nuclear option in 95% of 21 wargames tested.
