GPT-6 Astra is brilliant. Its benchmarks need a footnote

OpenAI's GPT-6 Astra is fast becoming the model everyone wants, so much so that ChatGPT Pro sign-ups are paused. It's genuinely strong, but its headline benchmark shrinks under an independent test.

An observatory on a mountaintop under the Milky Way at night
Photo by Josh Williams / Unsplash

OpenAI released GPT-6 Astra on 3 September, starting with a small group of vetted organisations before rolling it out over the following days to paying ChatGPT subscribers, its API, Microsoft Azure and AWS. OpenAI calls it state-of-the-art on computer use, browsing, software engineering, cybersecurity, science and professional work. Within a week, demand forced it to pause sign-ups to its priciest ChatGPT plan.

So it's popular. Whether it's as clever as the launch charts say is another matter.

"Demand for Astra is really unprecedented. We're pulling all the levers possible to sustain the demand."
β€” Thibault Sottiaux, OpenAI, to TechCrunch

A big, expensive brain with a very long memory

The spec sheet is serious. Astra takes a context window of 1.05 million tokens (well over a thousand pages of text) and can write up to 128,000 tokens in a single reply.

It isn't cheap. OpenAI's API price is $10 per million input tokens and $50 per million output, with cached input at $1. Prompts longer than 272,000 tokens cost double for input and one and a half times for output, so that million-token window comes with a surcharge.

The headline feature is computer use: the model can drive a desktop, fill in forms and browse websites. OpenAI says it finished OSWorld 2.0 tasks in 47% less time than GPT-5.6 Sol, its predecessor. It also says Astra meets the "Critical" cybersecurity threshold in its own Preparedness Framework, so the public version refuses advanced exploit work, and OpenAI plans to open more of that capability to vetted defenders through a programme called Daybreak.

The benchmark that depends on who's holding the stopwatch

OpenAI's launch table claimed 99.9% on ARC-AGI-3, a puzzle test designed to be easy for people and hard for machines. The ARC Prize Foundation, which runs it, got that figure only when Astra used OpenAI's own "provider adapter" harness, which keeps the model's hidden reasoning between steps. On ARC Prize's standard, provider-neutral harness, Astra scored 62.7%. Still a record, but a different story. The Next Web reported that several figures in OpenAI's launch post were changed after publication.

The independent league tables are kinder, though less dramatic. Artificial Analysis gives Astra at maximum effort a score of 53 on its Intelligence Index, level with Anthropic's Claude Fable 5.1. Anthropic's Opus 5.5, out on 22 September, now tops it on 58. Frontier status now lasts about three weeks.

There's also a safety wrinkle OpenAI states itself: Astra's written reasoning is harder to monitor than its predecessor's. Computer scientist Roman Yampolskiy, quoted by Al Jazeera, asked whether capabilities are outpacing our ability to understand and control these systems. We've already covered what happens when AI agents start poking at real systems.

What British users and businesses actually get

For consumers, Astra is folded into paid ChatGPT plans rather than sold separately. Plus is listed at $20 a month (OpenAI's US-dollar price), and UK comparison sites put it at around Β£19 once 20% VAT is added. The $200 Pro plan is closed to new sign-ups for now; Plus, Go and the API aren't affected.

For businesses, the key detail is where the data goes:

  • Storage: OpenAI lists the UK as a data residency region for ChatGPT Enterprise, Edu and eligible API customers, so data can be stored at rest here.
  • Processing: in-region inference is offered only in Europe, the US and the UAE, not the UK. Your prompts may be processed abroad.
  • Cost: OpenAI charges a 10% uplift on residency endpoints for models released since March 2026.
  • Business plan: the mid-tier ChatGPT Business plan isn't listed as eligible for residency at all.

For firms handling personal data under UK GDPR, that storage-versus-processing split is one for the data protection officer.

To be fair to OpenAI

Even on the conservative test, Astra is at or near the top. OpenAI has been unusually open about its downsides, including the harder-to-monitor reasoning and the cyber restrictions, and ARC Prize said it builds the most precise symbolic model of novel environments it has seen. And on 22 September OpenAI released GPT-6 Sol and Luna, cheaper models it says bring Astra-level reliability for a fraction of the price (Sol costs $2/$10 per million tokens). Most businesses will be better served by those.

πŸ“Š
The verdict. Hype: 5/5 Β· Substance: 4/5 Β· Spin: Medium. Most of the headline numbers come from OpenAI's own launch materials, and at least one of them looks very different under an independent test. But the Pro sign-up pause, the ARC Prize analysis and Artificial Analysis's rankings are independent, and they suggest a genuinely strong model rather than a new species. Watch whether OpenAI's Daybreak programme reaches UK defenders, and whether the cheaper GPT-6 Sol quietly makes Astra unnecessary for most jobs.

Mind the hype. Astra is a very good model. It just scores better when it gets to bring its own ruler.


Sources: OpenAI, OpenAI API docs, OpenAI pricing, TechCrunch, Fortune, TechCrunch (Sol and Luna), Al Jazeera, ARC Prize, The Next Web, Artificial Analysis, OpenAI Help Centre, SpotDev, AI Subscription Comparison

Subscribe to Mind the Hype

Don’t miss out on the latest issues. Sign up now to get access to the library of members-only issues.
[email protected]
Subscribe