SAN FRANCISCO (The Thursday Times) — AI company Anthropic has warned investors that it may soon cross its own threshold for AI systems that speed up the building of AI, in a new US filing for its share listing that also says it might not notice such a leap until after it has happened.
The company, which makes the Claude chatbot, said in the updated filing with the Securities and Exchange Commission on Monday that its internal estimates and expectations for future models “indicate that we may imminently meet our Responsible Scaling Policy’s definition of ‘dramatically accelerated progress attributable to AI’”.
“We have not designated any of our current models as exceeding this threshold, and we continue to assess the underlying data,” it said.
The idea at issue is known as recursive self-improvement: AI that becomes good enough at AI research to help create stronger versions of itself, which could rapidly speed up progress. Some see it as a route to huge scientific gains. Others fear systems advancing faster than people can control.
Anthropic said it already uses AI in its own research, and that this “may cause the capabilities of our models to advance more rapidly than our ability to develop, validate, and implement corresponding safety, alignment, and security measures”. It said its evidence relies on lagging indicators. “We have in the past not identified, and may in the future not identify, a significant acceleration in our rate of development until after it has occurred,” it said.
The company said it did not expect to be able to safely let Claude “fully autonomously design and build successive generations of frontier models without meaningful human supervision or understanding, if and when that becomes technically possible”, and that it was considering changes to its safety policy “to keep humans in control”.
Risk sections in share filings are written to warn investors of everything that could go wrong, and sceptics point out that they can also work as marketing for a company’s technology. In June, Anthropic said that being able to slow or pause the development of the most advanced AI would be good for the world. On Monday, OpenAI chief Sam Altman told a conference in Abu Dhabi that AI was now “mostly smarter” than top scientists.
