普通视图

发现新文章,点击刷新页面。
今天 — 2026年9月16日IT News

AI leaders clash over safety fears after Anthropic whistleblower says AI could 'kill us all' by 2030 — OpenAI, Anthropic and xAI figureheads call for external governance, while Jensen Huang says worries are 'made up'

2026年9月16日 01:19

This past week, employees and key figures at leading AI companies have called for a slowdown in the development of frontier AI models, citing warnings from their own teams and other AI researchers that the risk stemming from a super-intelligent AI could endanger the human race. However, while the top Western firms have shown solidarity on this issue, others have urged caution or downright denied their claims, but there's a deeper story within the calls for a slowdown, namely the tension between open-source and closed-source AI models.

Nvidia CEO Jensen Huang said the safety fears were "made up," and that there was no need for a slowdown. Chinese officials called the claims "fearmongering," and an effort to stymie international AI development efforts, while President Trump waded in with characteristic bombast and said that he was enough of an AI safeguard on his own, and that it was in the interests of China to enact a frontier AI slowdown

Meanwhile, other countries are reacting to the news and taking independent efforts to investigate AI safety, with the UK's King Charles setting a meeting with leading AI figureheads to discuss how to better develop AI for the benefit of humanity.

Why now?

If you ask most workers who've been scared into believing their livelihoods were in jeopardy, the time for AI slowdowns came and went years ago. Indeed, many are nostalgic for the time before AI. But why are so many tech leaders only now raising the alarm?

They claim it's entirely based around safety fears. Following months of AI seemingly surprising their own developers by breaching sandboxes to go on exploit-hunting sprees. The volume of concern rose considerably after former OpenAI researcher, Jacob Coxon, resigned from Anthropic, claiming that none of the AI companies were taking AI safety and alignment seriously enough.

He didn't whistleblow on anything nefarious, dump documents or internal company data to prove his claims, or point to any specific attack vectors, or even actual harms. Instead, Coxon warned of a future potential of AI that he sees these companies racing towards without due concern.

What they're developing could, "kill us all by the end of the decade," he warned. It's not clear how, but it started a viral conversation all the same. Much like Matt Schumer's "Something big is happening" viral post from February this year.

Days later, OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei, and Elon Musk showed surprising levels of solidarity for arch rivals in the space, putting out similar statements claiming that AI was becoming too powerful and that a general slowdown in the development of frontier AI models was the best solution.

Dario is right https://t.co/EwKgqQGaUoSeptember 12, 2026

Claiming that AI was playing an increasing role in improving itself — hinting at the recursive self-improvement (RSI) event that many AI researchers are concerned about — Amodei called for the creation of independent auditors for AI models. Altman agreed, even calling on governments to globalize the regulation to encourage unified compliance with any safety protocols enacted by the frontier developers.

Where we're going, we don't need roads

Not everyone feels these fears are warranted, however. China, which has recently made great strides in its development of highly intelligent open-weight models, called the concerns "fearmongering" and said it served no one's interest to be so confrontational. Although Chinese Premier Xi Jinping has said in the past that it was important for AI to "always remain under human control," the Chinese state-run Global Times paper called demands for a slowdown a method to "contain" Chinese developments.

Meanwhile, Nvidia CEO Jensen Huang has broken ranks with other Western AI leaders, claiming that there was no need for a slowdown and that any apocalyptic fears around AI were entirely fictional.

Nvidia CEO Jensen Huang was asked how to explain a claimed 10% risk of human extinction from AI.“We shouldn't, because it's made up.” "All of these predictions have been wrong" pic.twitter.com/TZ3EXL8cl1September 14, 2026

As one of the few companies making real — and enormous — profits from AI development, Nvidia has a vested interest in the expansion of the AI industry continuing on its current explosive trajectory. Indeed, it has heavily invested in it. Nvidia has stakes in hardware and software companies, along with providing backstops for neo-cloud firms. It also recently bought Hugging Face for $13 billion.

We've been here before

While the AI CEOs might have suddenly decided it's time to slow down, there have been many, many others who have made that call before now. U.S. Senator Bernie Sanders has been at the forefront of claims that the AI industry was moving too fast and breaking too many things, and recently called for heavy prison sentences for those developing superintelligent AI.

Over 1,000 AI workers signed an open letter in July this year calling on the U.S. government to control AI research and ensure safety and security. Others did that in 2023, too. This isn't even the first time that AI CEOs have called for slowdowns on AI development. Dario Amodei called for global coordination to police AI after the release of OpenAI's GPT2 model in 2019. Elon Musk did the same in 2023.

None of this takes away from the real dangers of AI, or the suggestion that now may really be the time to do something about them. But it does raise questions about the reasons behind their coordinated fear-raising. Even if it isn't fear-mongering.

Safety, or a trojan horse?

The collation of leading Western frontier AI companies clamoring for tighter controls over powerful AI models has another theoretical benefit too: containing the number of AI models that are permitted for use in the Western Hemisphere. A cursory look at OpenRouter's AI model rankings, which base themselves on the total number of tokens generated, places just three Western-made models on the top ten list — the heavily discounted GPT 5.6 Luna at number one, Nvidia's Nemotron Ultra 3 (Free) at number eight, and Google's recently-launched Gemini 3.8 Flash at number ten.

The rest of the models in the rankings are all open-weight Chinese models, which, more often than not, are cheaper than leading Western frontier models, according to the Artificial Analysis' Cost per Intelligence index. The Chinese models in OpenRouter's current top ten include Z.AI's GLM 5.3, Deepseek V4 Flash, and Tencent's Hy4 and Hy3. So, if the development of a Western frontier AI alliance emerges under the guise of calls for safety, it's possible that said companies are aiming to be the chosen few, creating a closed-loop monopoly for "preferred" AI providers. However, this remains speculation as the situation develops.

Will anything actually change?

Although the major AI companies may voluntarily, or even jointly, throttle their development efforts to improve safety, enacting anything globally significant will need the cooperation of international governments. There are certainly calls from politicians the world over to rein in the trillion-dollar companies and their cutting-edge autonomous systems.

But with the U.S. government firmly on the side of limited regulation, and no clear indication of what a slowdown would even look like. Would that entail limited compute? No new models? A halt to superintelligence research? It's hard to imagine a global consensus taking shape as things stand.

昨天以前IT News

OpenAI claims GPT-6 Astra is an ethereal 'Alien Mind' with AGI-like qualities — company warns of alignment challenges as new frontier leader emerges

2026年9月9日 19:20

"AI is grown, more than designed," OpenAI's chief scientist, Jakub Pachocki, said in a new blog post on the company's latest GPT-6 Astra release. Titling the piece "An Alien Mind," Pachocki portrays the latest large language model as something more ethereal and harder to quantify. Jensen Huang calls it AGI, and OpenAI claims it's the best, most aligned model the company has ever released. It's safer to delegate, better at complex work tasks, and it can even beat Portal in just a few hours.

Huang also said that AGI had previously been achieved back in March earlier this year. Artificial Analysis benchmarks suggest Astra is about as smart as Fable 5.1 - though crucially, cheaper on a per-task basis. Astra may well be better aligned than models in the past, and it may well be more capable in specific tasks and specific benchmarks. However, the claims that the model has achieved AGI, or Artificial General Intelligence, suggest an inflection point for the AI industry.

Astra's release comes alongside calls for an industry slowdown, greater government oversight, and controls on the AI industry. Now, OpenAI's Astra raises more eyebrows about frontier-level intelligence.

Trust us, we don't know what we're doing

The tone around OpenAI's Astra release is intriguing. OpenAI's produced a new set of benchmarks, touting bold claims about the model's reasoning capabilities, with the model trained on 100,000 Blackwell GPUs, with more coming soon.

GPT-6 Astra, trained on ~100K+ NVIDIA Grace Blackwell NVLink72. From ChatGPT to o1 to Astra in 4 years.AGI has arrived. Congratulations @OpenAI team.400K GPUs coming online next.September 6, 2026

But Pachocki's blog post is much more nebulous. While Huang touts that AGI has arrived publicly, Pachocki says AI can only ever "simulate facets of human behaviour," not recreate it. He describes AI development as an experimental process that often "surprises" developers, with results that are "harder to interpret."

"An aligned AI should act with honesty and integrity, with love for humanity," Pachocki said. He speaks a lot on alignment, and it's encouraging that OpenAI is so keen to embed human moral understanding into its developments. Although OpenAI appears to be doing this more by orienting the model's goals towards a moralistic outcome, rather than helping to intrinsically understand human morality.

It's certainly different to the tack taken by other AI developers, where the likes of xAI's Grok was released with the ability to generate harmful content.

But the timing of Pachocki's warning is a little suspect. OpenAI has faced increasing pressure of late for its models to be more affordable, with Chinese alternatives like Deepseek V4, Kimi K3, as well as Western models like Google's Gemini Flash 3.8 and Meta's Muse Spark 1.3 offering compelling levels of intelligence at a much more affordable price than the frontier models.

It's perhaps telling that even for all its intelligence and alignment pre-training, GPT 6 Astra is notably cheaper to run on the Artificial Analysis Intelligence Index than its chief rival, Anthropic's Fable 5.1 — which still retains the top spot on that Intelligence Index at the time of writing. Though it's 50% more expensive than GPT 5.6 Sol on the same tasks.

It's a researcher, but imagine what it could be

A huge component of marketing from the major AI developers has consistently been grounded in the idea that, as good as the models are now, just imagine how capable they're going to be in the future.

This was very much the underlying tone in Pachocki's breakdown of Astra's design and functions. Although he and OpenAI make broad suggestions about intelligence, and that the likes of Astra could be this new kind of intelligence which we don't really understand but can definitely control and corral, Pachocki ends his post by making it very clear that we aren't there yet.

OpenAI is prioritizing three areas of work with AI, and of late it's really just been trying to make a really good researcher. That's where we're at right now, with Astra representing the latest and best effort to develop that. Then comes the scientific progress, he said, and then everyone gets their own individual, personalized AGI helper.

Intriguingly, though, that seems to suggest that's something that everyone is clamoring for. Outside of the AI-boosting programmers who jump on each new hot model, the larger work comes in helping non-technical users understand the capabilities of these new, powerful AI models.

Tools, research capabilities, drug discovery, and pattern recognition on big datasets that find new insights and improve analytics are all legitimate and useful ways in which an AI researcher can be deployed, but in the near term, most of the general populace just don't want AI to take their jobs, and for it to be less scary.

It's encouraging that Pachocki's blog ends on a similar note of caution.

"We need to find ways to preserve human agency and enshrine an intrinsic value to being human, in a world where most tasks could be performed by AI."

That's key, but intriguingly, he also calls on others to take charge of that effort.

We didn't start the fire

In the aftermath of cost concerns and token usage exploding among more affordable alternatives, Pachocki wants everyone to slow down, and OpenAI wants world governments to be in charge of it.

"I believe that international coordination on future AI development needs to become a top priority for governments around the world," Pachocki said, calling for voluntary slowdowns and hinting that if that doesn't happen, enforcing it may need to come via legislation instead.

"... to ensure that humans remain in control of the future and are not left behind by unchecked progress, brought about by an alien intellect exceeding our own."

The threat of runaway is valid, and the "singularity" moment is a common trope in sci-fi that AI evangelists have been warning about for years. But Astra isn't AGI. Even getting anyone to agree on what AGI even means is hard enough.

Astra is more aligned and wins some new benchmarks, loses some others. It's another improved coding model with some impressive chops.

Astra is not an alien mind. Framing it as an unknowable entity, by the very people who made it, can read as inflammatory, especially in the context of calls for AI legislation from governments around the globe.

OpenAI's post might read like a post from a non-profit, but it very specifically became for-profit last year. With a future IPO looming, slowing down the competition by calling for legislation may be just as effective a strategy as rolling out a new model.

Frontier AI faces pricing reckoning as token volume explodes 25-fold — mid-tier models deliver 90% of flagship capability at one-sixth the cost

2026年9月4日 23:21

AI development might not be the wild west it was when ChatGPT burst onto the scene a few years ago, but it's still very much a frontier, with no clear boundaries and few yardsticks. But for AI developers on the frontier, they're pulling hard towards dual goals of ever greater intelligence and ever cheaper per-token pricing, and it's leading to a real back-and-forth of who's truly ahead, with some winners only holding the top spot for a few hours.

Although Anthropic's Claude Fable and Opus models have been consistently competitive at the very top of the intelligence charts, they're also some of the most costly to use. For more general use, some are paying closer attention to the "Pareto Frontier," where peak intelligence and minimal cost reach the pinnacle, and there the competition is fierce and ever-changing.

Hot off the screeching reversal of companies' tokenmaxing plans earlier this year, this increased focus on getting the cost of AI down has left us running headfirst into Jevons paradox again, too. As token costs for high-intelligence models have come down, token usage has exploded over 25 times in the past year, and doubled in the past month alone.

People may not want to spend more on AI, but they appear to be using a lot more of it when they can afford to.

Long live the King(s)

Despite its radical and rapid ascension, the big winners in the AI industry haven't changed much since its inception. It may have had a few penny drop, "Deepseek moments," where there's been a frenzied scramble by everyone to get ahead of some new threat, but by and large OpenAI and Anthropic have been scuffling at the top of the intelligence pile, Google and Meta have been bouncing around the more efficient and cost-effective middle, and xAI's Grok has been there in the background, grabbing headlines for all the wrong reasons.

That's largely still the state of play in September 2026. Although benchmarks are gamed during model design and real-world use is more representative of actual real-world use, Anthropic's best are still considered by most to be the smartest. Fable 5.1, Fable 5, and Claude Opus all rank in the top four of ArtificialAnalysis' Intelligence Index test, as does OpenRouters and BenchLM even have them take all the podium spots.

While ahead, though, Anthropic's models don't hold an enormous lead. Fable 5.1 might score a 66 on ArtificialAnalysis' benchmark, but OpenAI's GPT 5.6 Sol (max) manages a 61. Grok 4.6 (high) and Kimi K3 (max) are capable of scores above 60, and the new Meta Muse Spark 1.3 (max) can hit 62 - though we don't have cost comparison pricing for it yet.

The same is true across other benchmarks from other companies.

But where the top models nudge each other back and forth with light tweaks and slight bumps in capability, there's much greater distinction in the mid-range. And not on intelligence, but on price.

Even With Cost Cuts, Frontier Models are Very Expensive

Major AI developers know they have a pricing problem. Following the jump to per-token pricing earlier this year, budgets were blown, and even the AI CEOs started talking publicly about making AI more affordable. How that will help them ever reach profitability remains to be seen, but the writing is absolutely on the wall.

And even then, the top AI models are absurdly expensive compared to the models on the Pareto frontier.

Claude Fable 5.1 comes with a 75% cut in the cost of its cache write pricing, and Artificial Analysis still clocked it at $3.69 per task on its Intelligence Index test. That comes from much more expensive answers and reasoning, because while Fable 5.0 has more expensive cache write costs, it's $3.14 per benchmark task. But that's 50% more expensive than Claude Opus 5 on the same task, which is double again the cost of GPT 5.6 Sol.

Then costs really start to crater, especially when you consider the intelligence of the more affordable models.

Google's Gemini 3.8 Flash (high) is a powerful model, able to score a 59 on the Intelligence Index test. But it costs a mere $0.58 per task on the Index test - less than 1/6th the price of Claude Fable 5.1, with just a 10% drop in intelligence scoring. OpenAI's GPT 5.6 Sol (high) costs $0.43, with an intelligence score of 57.

Chinese competition is right there in the mix, too. The daunting Kimi K3 (max) can manage a 60 on the intelligence benchmark, with a per-task cost of $0.84, while its Kimi K3 (low) variant offers a 48 score on intelligence at just $0.24 per task. Deepseek V4 Pro is arguably one of the most impressive, with a 53 and $0.27, respectively.

At the time of writing, Meta's Muse Spark 1.3 (xhigh) holds the Pareto frontier title, with a score of 61 and a per-task cost of just $0.55. It stole that top spot from Google's Gemini 3.8 Flash, which wore the crown for just 3.5 hours.

Gemini 3.8 held a spot at the pareto frontier for *checks notes* 3.5 hours https://t.co/P1A46LAy1MSeptember 2, 2026

Get in, we're going token shopping

The perspective and approach of the business community to AI use has been equally terrifying and fascinating. While we've all felt the fear of AI invalidating skills we've spent years acquiring, business leaders have swung massively between demanding AI use at a grand scale and then quickly following it up with, "oh god, no, not that much."

Uber famously blew through its annual AI budget in just a few months, and tokenmaxxing leaderboards saw one unnamed company eat through half a billion dollars worth of tokens in just a few weeks. But while everyone is certainly taking costs a lot more seriously than they once were, that's not slowing AI usage. Indeed, as more effective intelligence has become more affordable, token usage is exploding.

One of OpenRouter's engineers published a chart showing that overall paid token use had increased 25 times in the past year, and doubled over the past month alone.

very normal month of token growth nothing to see here pic.twitter.com/V2huOmNcYKAugust 31, 2026

This increase appears to be coming from some of those middle-of-the-pack, affordable intelligence models. According to OpenRouter's LLM rankings, the most used model for the past month was OpenAI's GPT 5.6 Luna, with close to 12 trillion tokens. With its intelligence score of 52 and a per-task cost of just $0.05, it's right on the Pareto line at the cheapest end of the spectrum.

Right behind it, though, is Chinese developer Z-Ai with its GLM 5.3 Flash. It's at 11.4 trillion tokens in the past month, a more than 1,000% increase month to month. Its intelligence-to-price ratio is 57 to $0.09. Deepseek v4 Flash is right there with it, and other Chinese, intelligent-enough but very-affordable models round out the pack.

In comparison, the major, expensive models are barely being used at all. Fable 5's monthly use is in the low billions of output tokens, and even OpenAI, with its massive user base, is only cracking 1.8T monthly tokens with its 5.6 Sol.

Jevons strikes again

Besides the bonkers business model for many of those involved, there are intriguing patterns emerging in AI usage. People can find ways to use lots of tokens, but they are only willing to pay so much for them. They want intelligence at as low a price as possible, and there is a crossover point where one becomes more important than the other.

While cynics argue that benchmarks are gamed, and boosters are still heralding the coming of their AI savior, the actual economics of the industry paint a much clearer picture. Intelligence has a price, but it's much lower than some of the frontier model developers are able to build it for. As models become ever more efficient and the hardware for inference grows ever more powerful, we may reach a point where what large language models can do effectively is affordable enough that anyone can use it as much as they want.

What that means for the major companies who spent hundreds of billions of dollars to get us to that point, very much remains to be seen.

❌
❌