Introducing GPT-6.2 Astra and Sol
OpenAI’s GPT-6.2 Astra more than doubles Claude Fable 5.2’s legal-task pass rate on Harvey LAB-AA v1.1 at about one-sixth the cost.
18 comments
I'm pleasantly surprised by the speed.
Opus 5.5 is so good that this is the least motivated I've been in a long time to try out a new flagship model.
On my "impressionistic benchmarks" (standard routines I have for finding high-priority problems and similar), it's comparable to Opus but not clearly better. Its choice for "most urgent problem" is not good, and is the same choice I get from Opus 5.5 and Astra 6.1.
Just following up: I've run a whole bunch of more quantitative assessments of Astra 6.2, and it's way behind Opus 5.5 on those. Now, some of those assessments were constructed in part by Opus 5.5, but I keep a careful eye on them, and it's really not close right now. (I've tried a couple different thinking levels for Astra 6.2, FWIW.)
thanks for sharing. very interesting. Since some of these were built partly with Opus, a cheap check is to have a third model (or a person) rewrite a sample of those tasks and rerun both. If Opus’s lead shrinks on the rewrites, that suggests part of the gap is Opus being good at Opus-shaped tasks.
Agree! FWIW I didn't explain myself clearly enough to show why this wouldn't be quite so cheap: these are eval harnesses, where Opus was involved in finding and drafting the evals, but humans have reviewed, edited, and modified a lot of them. The "Opus-free" version of this is pretty labor-intensive because of the searching and human effort required (and the harness is already fractionally non-Opus, because I use a bunch of models).
This also means that it's relatively unlikely (IMO!) that much of the difference here is Opus bias, but "relatively" is doing a lot of work there.
Also FWIW, Opus absolutely smoked Astra this morning at another qualitative task ("what's the best way for this system to handle X?").
I wonder if releasing a model that benchmarks with a regression in code quality is an indicator that they're trying to win harder in other arenas and are ceding code to Anthropic (for now). Similar to how Claude doesn't have image generation, it just builds really intricate web pages that look like images.
I like hearing that 6.2 Sol was slightly better and somewhat faster on the deck QA, I've been trying to get the turn times back down again and it's the main culprit.
Interesting to read the Harvey number in absolute terms. LAB-AA only credits a task when every rubric criterion passes with no material hallucination, so “more than double” is 7.2% to 15%, and most tasks still miss full credit. The more interesting thing imo: Astra for Law (same model plus a legal search index and research instructions) gets 20.8%, a jump about the size of GPT-6 Astra to 6.2 (8.6% to 15%). On 120 tasks one standard error is roughly 3 points, so maybe it hints that retrieval and workflow matter about as much as the model for domain work. Anyone seeing that in their own evals?
Not that exact effect, but spiritually analogous ones, definitely. So, for example, slightly improving the information we send to decision models tends to make them vastly outperform slow LLMs that don't have the better information.
This is slightly speculative / impressionistic, but I think this effect is even bigger in the real world than the numbers would lead one to believe, just because benchmarks are less and less useful (natemeyvis.com).
The jump in computer use ability is phenomenal and unlocks a whole host of capability and workflow automations. I've been able to leverage this to effectively not touch terrible UIs anymore and also helps with the mental overload that comes with learning new UIs - I can just navigate almost any interface by speaking my goals. It even makes the case that integrations are no longer a major blocker for most workflows
I need to set up some good evals for this so I'm prepared once OpenAI releases this for ITAR compliant channels. I was very impressed with GPT5 Sol but have since switched over to Opus 5.5
My perspective on seeing more recent models work on BenchCAD and EEBench are that they are a little easily "gamed" and those score jumps are a little mis-representative
I'm surprised at how strong Opus 5.5 is still at design + good decision-making in the real-world in that OpenAI's models are starting to feel antiquated. I'm even more surprised that they released this model instead of waiting longer to see if they could at least catch up on some of the gaps that Carbon and Opus/Fable have set in terms of "raw" intelligence.
some unimpressed users on linkedin: https://www.linkedin.com/news/story/users-are-unimpressed-with-openais-latest-models-7800652/
has anyone here run GPTAstra or Sol on production work yet? Would be curious to hear how it compares with Opus 5.5 and Fable 5.2
Access to ultra fast mode for the $200 sub is pretty cool, especially if you need to blaze some specific smaller stuff out in the week next to a larger sub/APIs.
Kind of crazy that ultra fast modes are locked behind consumer plans only, no way to access on any business plan afaict?!


Honestly this release just highlights how high of a bar Opus 5.5 set almost 2 months ago, rumors are probably true that it took OpenAI by surprise and that’s why it took OpenAI so long to release the new Astra (75 days since previous release).
Weirdly it seems like Astra didn't improve and Sol actually regressed at agentic coding?!
Even the framing with cost on the X-axis seems to suggest that OpenAI is now leaning into being the affordable workhorse option (where they're already having a lot of success (tomtunguz.com)).
FWIW running 6.2 sol on a few internal benchmarks that we’ve been using 6.1 sol for: Summarization / writing style - almost twice as fast but slightly wordier and somehow makes more factual errors Visual slide deck QA - slightly better (caught one error that 6.1 didn’t) and roughly 25% faster