A recent BBC report, also covered by The Guardian, has reignited one of the most unsettling debates surrounding artificial intelligence.
Jacob Coxon, a former researcher at Anthropic — and previously at OpenAI — resigned from his position after warning that the world’s leading technology companies are racing to develop increasingly powerful artificial intelligence systems without yet having sufficient guarantees that the risks can be controlled.
His warning is extreme: he says that people working directly on the development of these systems genuinely believe that artificial intelligence could ultimately lead to human extinction.
We may consider that prediction exaggerated.
But it becomes more difficult to dismiss when the warning comes not only from external critics, but from people who have been directly involved in building these technologies.
Coxon is not alone.
Dario Amodei, CEO of Anthropic, has repeatedly warned of the potentially catastrophic risks associated with increasingly powerful AI systems. Other leading figures in the sector, including Sam Altman of OpenAI, have also acknowledged that advanced artificial intelligence creates risks that require regulation and oversight.
This raises an uncomfortable question.
If those developing the technology themselves believe that it may pose serious risks, who should decide how far and how fast we go?
Europe chose to regulate first
For years, Europe has been criticised for its approach to technology.
While the United States and China focused heavily on innovation, investment and technological leadership, the European Union took a different path: ethics, fundamental rights, transparency, human oversight and regulation.
The process did not begin with the AI Act.
As early as 2018, the European Commission was already developing ethical principles for what it called trustworthy artificial intelligence.
That approach eventually led to the European Artificial Intelligence Act, the world’s first comprehensive legal framework specifically regulating AI.
The criticism has been predictable: Europe regulates while others innovate.
And there is some truth in that criticism.
Excessive regulation can slow innovation, create barriers to entry and make it harder for European companies to compete with American and Chinese technology giants.
But perhaps the debate has sometimes been framed too simply.
The real choice is not necessarily between innovation and regulation.
The question is whether certain technologies become powerful enough that innovation without safeguards ceases to be a purely commercial matter.
We regulate medicines before they reach patients.
We regulate aircraft before passengers board them.
We regulate nuclear energy, financial markets and medical devices because the potential consequences of failure are too significant to leave exclusively to the organisations developing them.
Why should highly advanced artificial intelligence be fundamentally different?
Regulation does not mean stopping AI
None of this means that artificial intelligence should be feared or stopped.
AI is already bringing extraordinary opportunities to medicine, science, business, education and professional services.
At Bennet & Rey, we work precisely on the legal and regulatory implications of the use of artificial intelligence by companies.
And our position is not anti-technology.
Quite the opposite.
Innovation needs legal certainty.
Companies need to know what they can do, what risks they must assess, what information they must provide and where responsibility lies when something goes wrong.
Good regulation should not prevent innovation.
It should make innovation safer, more transparent and more sustainable.
The European model will undoubtedly need adjustment. Regulation must remain proportionate and should not become an obstacle to smaller businesses or genuine technological development.
But the warnings now coming from inside the AI industry suggest that the European instinct to ask questions before the technology becomes uncontrollable may not have been entirely misguided.
Perhaps Europe was not simply being bureaucratic.
Perhaps it was being cautious.
And, when even the people building the most advanced AI systems are warning us about what could happen next, caution may turn out to have been the more innovative position after all.
Transparency about how this article was created
By the way, this article was prepared with the assistance of artificial intelligence and human oversight. I first write the text myself; I then use AI to review and structure it; finally, I go through it again and make any changes I consider necessary.
The image accompanying the article was also generated using artificial intelligence.
Bennet & Rey Lawyers
We can help you
