Not surprised to see Claude significantly higher in scientific intelligence than Sol.
You can tell that Claude really does grasp a wide array of highly specific scientific and mathematical nuances... where's codex is just basically for coding and that's it.
That's the feel I get from the both of them anyways and I've used both on the 20x plan for the past week at length.
Don't get me wrong, codex is great at finding bugs and building games. It's great.
a2ff6eeb0 10 minutes ago [-]
I wonder how long it's going to be before self improvement encompasses hardware and materials science, not just code. It's exciting, soon we'll be able to fully hand off scientific, mathematical, and technical progress over to the machines, and then we can lay back and enjoy the results.
jerpint 53 minutes ago [-]
The fact that opus 5 is outperforming fable is odd to me
From personal experience, opus 5 feels net inferior to fable on almost every aspect (for coding tasks)
jhbadger 33 minutes ago [-]
This is measuring on scientific tasks though. I haven't used Fable lately but when I was playing with it when it was new its "safety" features made it practically impossible to use for biomedical science -- it once refused to work on a pipeline of mine that was analyzing pathogenicity islands in bacteria (presumably because it had guardrails to that effect to stop potential bioterrorists and the like)
akshay_akula 2 hours ago [-]
Evals on actual research workflows is the right direction, most agent benches are toy tasks.
2 hours ago [-]
rubslopes 3 hours ago [-]
I'm glad to see that GPT Sol beats Opus at least in Mathematical Sciences, because that's my need right now, and I much prefer GPT's prose style.
mlmonkey 1 hours ago [-]
Sad to see no mention of Gemini ...
vatsachak 2 hours ago [-]
Damn. These things aren't AGI... but I don't care.
Luna is good enough for me to give a parser spec and have it write one.
Rendered at 03:18:58 GMT+0000 (Coordinated Universal Time) with Vercel.
You can tell that Claude really does grasp a wide array of highly specific scientific and mathematical nuances... where's codex is just basically for coding and that's it.
That's the feel I get from the both of them anyways and I've used both on the 20x plan for the past week at length.
Don't get me wrong, codex is great at finding bugs and building games. It's great.
From personal experience, opus 5 feels net inferior to fable on almost every aspect (for coding tasks)
Luna is good enough for me to give a parser spec and have it write one.