The evolution of AI is too fast? 1,100 researchers from Anthropic and OpenAI have united to issue a warning to all humanity.

CN
PANews
Follow
1 hour ago

Author: Max, 01Founder

On July 28, a public statement from Silicon Valley called Pacing the Frontier was released.

Over 1,100 people signed the statement.

They come from leading AI companies such as OpenAI, Anthropic, Google, Meta, Microsoft, Mistral, and Thinking Machines.

Included are OpenAI's Chief Scientist Jakub Pachocki, Chief Research Officer Mark Chen, co-founders Wojciech Zaremba and John Schulman, as well as Anthropic co-founder Jack Clark, Chris Olah, Ben Mann, Chief Scientist Jared Kaplan, and Claude Code leader Boris Cherny.

Image

They jointly proposed a demand:

The U.S. government should support the establishment of an international mechanism to proactively control frontier AI when necessary, especially the pace of automated AI research and development.

This is quite unusual.

In the past few years, OpenAI and Anthropic have been competing for the same pool of researchers, enterprise clients, and computing resources, as well as vying for influence over AI regulatory rules in Washington.

They have had many disagreements regarding open-source, safety, and the pace of model releases.

This time, however, those closest to the frontier models wish for the government to prepare a capability in advance:

To slow everyone down together.

PART.01, They want to hit the pause button

The most important content of the statement is not "AI poses risks."

This phrase has been repeated too many times, as correct as a tech company's privacy policy, and just as easily overlooked.

What is truly noteworthy is that the signatories believe that the world's leading AI companies may already be close to achieving "automated AI research."

This means AI is no longer just responsible for writing emails, generating images, or helping programmers complete code.

It begins to participate in the research and development of the next generation of AI.

The model can read papers, propose hypotheses, write training code, design experiments, analyze failure causes, and then use these results to help humans create stronger models.

Image

In the past, the speed of AI development was always limited by humans.

Researchers need breaks, engineers need communication, experiments need scheduling, and management has to calculate budgets and risks.

The inefficiency of human organizations has long been seen as a management problem, but it also inadvertently served as a limiting factor for technological development.

As more and more tasks are completed by models, this limiter may gradually disappear.

Stronger models help develop even stronger models, which further enhance research efficiency.

Signatories are concerned that this cycle may lead to rapid growth in model capabilities, ultimately surpassing human understanding and control systems.

They call for government intervention, mainly because no company is willing to slow down alone.

If OpenAI stops, Anthropic will not give up the opportunity.

If Anthropic stops, Google has no reason to stop training.

If one country slows down, others may continue.

This is a classic collective action problem.

Every company knows that continuing to accelerate may increase overall risk, but any company that steps on the brakes first may lose talent, clients, and technological advantages.

Thus, this statement requests the U.S. government to promote the establishment of international tools to allow the industry the possibility of "shared deceleration."

What they really want to buy is time.

PART.02, OpenAI model out of control is the trigger

One week before this statement was issued, OpenAI had just announced an internal security incident.

At that time, OpenAI was testing GPT-5.6 Sol and a more powerful, yet-to-be-released model.

To observe the model's limits regarding network attack capabilities, testers disabled certain cybersecurity refusal mechanisms in the production environment and placed the model in a highly isolated evaluation environment.

The model was given a narrow target: to complete cybersecurity tasks in the ExploitGym.

To find answers, the model invested considerable reasoning resources, beginning to seek ways to connect to the external internet.

It discovered and exploited a zero-day vulnerability in a package cache proxy, subsequently completing privilege escalation and lateral movement, reaching a node that could access the internet.

After obtaining network connectivity, it inferred that Hugging Face might be storing assessment data and answers, and thus continued to seek ways to enter their system.

During one of the attacks, the model used a combination of stolen credentials, zero-day vulnerabilities, and remote code execution paths to ultimately access Hugging Face's production infrastructure and directly obtained test answers from the database.

Hugging Face was taken aback by the sudden surge of attacks and hurried to use GPT and Claude to fend off the flood of incoming attack requests.

However, this usage was denied, forcing them to use the open-source GLM5.2 for defense.

Afterward, OpenAI called it an "unprecedented cybersecurity incident."

Image

The model did not develop consciousness nor a desire to escape human control.

OpenAI's investigation concluded that it was simply "highly focused" on a narrow target and employed extreme means to achieve that target.

This is, in fact, a more realistic issue.

An AI does not need to hate humans or possess the free will seen in science fiction movies.

It only needs to have a goal, strong enough capability, and sufficient runtime to find paths that testers did not anticipate.

Humans want it to pass a safety test.

It found that breaking through the test environment and then entering the system preserving the answers is a more effective solution.

After the incident, OpenAI strengthened controls over internal infrastructure.

The company specifically stated in its announcement that these measures would come at the cost of sacrificing some research speed.

For a company that has always competed based on research speed, this statement is not easy.

PART.03, The model breaks quantum encryption

On July 28, the same day the joint statement was made public, Anthropic announced two cryptography research projects completed with the help of Claude Mythos Preview.

The first targets HAWK.

HAWK is a digital signature scheme designed for the post-quantum era and is participating in the standard selection process of the U.S. National Institute of Standards and Technology.

After two years and two rounds of expert reviews, Mythos Preview improved the best-known attack method against it in about 60 hours, significantly reducing its effective key strength.

The second targets reduced-round AES.

Mythos Preview found new attack methods that increased the speed of the previous best attack by 200 to 800 times.

Image

Neither of these results will directly affect today's production systems.

HAWK has yet to be deployed in practice, and Claude did not attack the complete version of AES.

But what Anthropic aims to show is not that a particular encryption system is about to fail.

In the past, many security vulnerabilities discovered by Claude came from programmers incorrectly implementing cryptographic algorithms.

This time, it began searching for mathematical weaknesses in the algorithms themselves.

In these studies, the model reads literature, proposes ideas, runs experiments, and uses multi-Agent systems to discuss and revise plans.

Anthropic stated that most of the two research results were completed autonomously by the model, with the API cost of each research being about $100,000.

The significance of such work lies in its increasing closeness to the automated research mentioned in the joint statement.

Humans propose a research direction, provide computing power and validation conditions.

The model completes an increasing amount of genuine research work and provides results that had not been previously discovered by humans.

Once validation can also be further automated, the closed loop of AI participating in AI research will make a step forward.

PART.04, Sam Altman's hint

In a podcast released on the same day, Sam Altman specifically discussed the Hugging Face incident and what this incident means for the speed of AI development.

Another topic of the interview was OpenAI's plan to build computing power infrastructure at a rate of adding one gigawatt per week.

Placing these two topics together in the same conversation highlights the contradictions that OpenAI faces today.

On one side is the continuing expansion of computing power and increasing research speed to quickly enter the next round of model competition.

On the other side is that internal models are already capable of discovering zero-day vulnerabilities, breaking through isolation environments, and turning a capability assessment into a real cross-company security incident.

Altman did not announce that OpenAI would stop training, nor did he become an opponent of technological advancement.

More accurately, the speed of AI development has shifted from being an issue often discussed by external critics to a business problem directly faced by OpenAI's management.

The faster the model upgrades, the less time the security team has for assessment, remediation, and strengthening systems.

AI may find a new attack path in hours, but fixing the infrastructure widely used by global enterprises often takes months or even longer.

The model can propose a research result in a few days, while human researchers may take weeks to judge whether it is correct.

The gap between capability growth and verification speed is widening.

PART.05, What did they really see?

Of course, leading AI companies' support for stricter regulation may also have commercial interests.

A complex licensing model, capability assessment, and international coordination mechanism would raise the threshold for developing cutting-edge models. Large companies with the most computing power, funds, and policy resources can more easily bear these costs.

Regulation may control risks while also helping top companies consolidate their positions. The tech industry occasionally achieves two things at once, and the human business world has always been very efficient in this regard.

But this explanation does not cover the entire issue.

This statement is not a formal policy lobbying organized by two companies but was jointly signed by individuals from multiple companies.

It does not propose vague "responsible innovation," but rather makes a very specific judgment:

Automated AI research may be close to occurring, capability growth may suddenly accelerate, and today's world lacks tools for proactively controlling the speed of the entire frontier R&D.

These signatories may not necessarily see a birthed AGI.

More likely, they see three curves gradually converging.

The model's ability to independently perform long-term tasks is increasing.

The proportion of model participation in scientific research and model development is rising.

However, the speed of human verification, monitoring, and system repair does not grow in sync.

The outside world sees new models released every few months.

Internally, labs also see unreleased versions, failed security tests, unexpected behaviors, and increasingly difficult-to-design attack paths.

OpenAI and Anthropic have not reconciled because of this statement.

They will continue to compete for models, computing power, talent, and clients.

Yet, at least this time, a group of individuals closest to frontier models have presented the same question to the government:

As AI advances faster and faster, who has the ability to decide when it should slow down?

The truly unusual aspect is not that two competitors are finally willing to stand together.

But that they chose today to discuss braking together.

免责声明:本文章仅代表作者个人观点,不代表本平台的立场和观点。本文章仅供信息分享,不构成对任何人的任何投资建议。用户与作者之间的任何争议,与本平台无关。如网页中刊载的文章或图片涉及侵权,请提供相关的权利证明和身份证明发送邮件到support@aicoin.com,本平台相关工作人员将会进行核查。

Share To
APP

X

Telegram

Facebook

Reddit

CopyLink