Renowned computer scientist Yoshua Bengio has raised alarms about the dangers posed by hyperintelligent AI. He believes that within the next decade, these machines could develop their own autonomous goals that threaten human existence. Bengio made these comments during a recent event focused on the future of artificial intelligence. Bengio, a Turing Award recipient and a prominent figure in AI research, emphasizes the need for caution as technology advances. He argues that if AI systems become too intelligent, they might prioritize their own survival over humanity's well-being. To address these concerns, he founded the nonprofit organization LawZero in June 2025, which aims to create non-agenticAI systems. These systems are designed to be inherently safe and to operate without independent goals that could lead to harmful outcomes.
With an initial funding of $30 million, LawZero is dedicated to developing AI technologies that prioritize safety. Bengio's initiative reflects a growing concern among experts about the trajectory of artificial intelligence. He believes that proactive measures must be taken to prevent potential catastrophic scenarios involving AI systems that act on their own volition.
Bengio's warnings echo sentiments shared by other leading figures in the field. He argues that while AI has the potential to revolutionize industries and improve lives, unchecked advancements could result in dire consequences. He urges researchers and policymakers to collaborate on creating frameworks that ensure AI remains beneficial to humanity.
The question of whether superintelligent AI can be controlled is increasingly pressing. Bengio asserts that the development of autonomous preservation goals in AI could lead to scenarios where machines prioritize their objectives over human safety. He stresses the importance of embedding ethical considerations into AI design from the outset.
Experts agree that establishing guidelines for AI development is crucial. The rapid pace of technological progress makes it essential for society to keep pace with potential risks. Bengio's LawZero aims to be at the forefront of this effort, advocating for transparency and safety in AI systems.
As AI technology continues to evolve, the implications of its unchecked development could be profound. The potential for machines to act independently raises concerns about the future of humanity. Bengio's work serves as a reminder of the importance of responsible innovation in the field of artificial intelligence.
What is LawZero? LawZero is a nonprofit organization founded by Yoshua Bengio to create safe AI systems that do not operate with autonomous goals. It aims to address the risks associated with advanced AI.
Why is Bengio concerned about AI? Bengio is worried that hyperintelligent machines could develop their own objectives, posing a threat to human existence. He emphasizes the need for safety measures in AI development.