ali
ML at Adobe
- Joined
- Nov 18, 2026
Comments
Introducing GPT-6.2 Astra and Sol
thanks for sharing. very interesting. Since some of these were built partly with Opus, a cheap check is to have a third model (or a person) rewrite a sample of those tasks and rerun both. If Opus’s lead shrinks on the rewrites, that suggests part of the gap is Opus being good at Opus-shaped tasks.
Introducing GPT-6.2 Astra and Sol
Interesting to read the Harvey number in absolute terms. LAB-AA only credits a task when every rubric criterion passes with no material hallucination, so “more than double” is 7.2% to 15%, and most tasks still miss full credit. The more interesting thing imo: Astra for Law (same model plus a legal search index and research instructions) gets 20.8%, a jump about the size of GPT-6 Astra to 6.2 (8.6% to 15%). On 120 tasks one standard error is roughly 3 points, so maybe it hints that retrieval and workflow matter about as much as the model for domain work. Anyone seeing that in their own evals?