The Astra version seems to have used three.js r170, which is from October 2024. Sol used an even earlier version. GLM's code used the latest version, but I think it's just getting three.js@latest from jsdelivr so it's unlikely to be writing code against that version. Qwen on OpenCode also fetches from jsdelivr, but using a pinned version at r160.
I don't think any of these examples are using things like tone mapping so they're stuck in sRGB (AgX or ACES look much better), they're not using the node materials (good for programmatic texture implementation), and they're not doing anything cool like baking shadow environments or using post-processing effect.
They're nice, but I think they're showing how far behind AI models are on this sort of project rather than how good they are.
polyterative 45 days ago [-]
The fact that these tools are capable of doing anything at all is a miracle to me.
utopiah 44 days ago [-]
That's a very strange position.
There are thousands, if not millions now, of people working on the problem, a vast majority of humanity text output used for training, an enormous infrastructure composed of the most complex human made device (the processor), a gigantic amount of energy running it, for an attempt that spanned decades, if not century, heck if not millennia if you go as far as considering Antikythera.
It might be impressive but it's anything but a miracle when we consider how much effort was poured into this.
andai 45 days ago [-]
How am I driving? How does an engine even work?
avaer 45 days ago [-]
Not tone mapping/shader graph is probably the least of the sins, these kinds of one-shots tend to produce hundreds of THREE.Geometry, massive matrix walks, etc. that explode once you move beyond a screen saver.
The models _can_ do it, but you need to ask for the right things. Most people don't, they'll usually blame the browser for being slow or ugly when they can't break through the THREE demo page wall.
sroerick 45 days ago [-]
Can you elaborate on this? how would you work on a larger THREE project? would you create models independently?
avaer 45 days ago [-]
> how would you work on a larger THREE project?
A larger THREE.js project starts to look more and more like a game engine, so you pick and choose the parts you need. There's a ton of open source libs, most of the heavy components the big players use are open source, things like physics, mesh optimization.
Key AI-specific parts might be:
- a harness (so the agent can drive the thing)
- authoring pipeline (so you can bake/optimize assets)
- some sort of coherent renderer architecture (what are your assets, your passes, what's your shader graph).
Without some fundamentals here you are on the short road to falling off the cliff of tech debt and the AI will gladly drive you off of it until you ask for an expensive rewrite.
> would you create models independently
Yes. Pretty much any proven gamedev/asset pipeline is something frontier models are good at. Procedural systems, asset store, free content, Blender, Meshy.
Literally anything except "make a THREE.js scene" -- which is not a technique used in gamedev, beyond throwaway prototypes or demos. Which is what you will get if you ask for a THREE.js scene.
rao-v 45 days ago [-]
Does anyone have an efficient, reasonably designed three.js or other web oriented mini game engine? Low poly but modern rendering effects should be a good niche for hobbyist stuff but hard to find.
Footprint0521 44 days ago [-]
Three.js with cannon.js is what you want, but the more you work with it, the more you realize how the difficult part is not so much the “game” or “engine” in that sentence but the word “web”… optimizing assets is a massive skill all in its own to be storage friendly and usable without the easy default of laggy and massive download size
rao-v 44 days ago [-]
For what it’s worth rando redditers warns that cannon.js is old.
Both the regular and lite demos are pretty amazing for browserware!
utopiah 44 days ago [-]
I think the cool kids now prefer Rapier for physics in the browser.
rao-v 45 days ago [-]
Really appreciate this perspective. It’s tempting to stop at being amazed at what these models can do, but my experience matches yours - they need help with details to do well.
scroogey 45 days ago [-]
Anyone know of pertinent skill files to nudge the agents into these directions?
avaer 45 days ago [-]
Don't use skills. Point it to a game engine or better yet full game repo or whatever and ask for what you want.
warpech 45 days ago [-]
What’s a good way to drive models to use current versions?
laruss5 45 days ago [-]
[flagged]
SadErn 45 days ago [-]
[dead]
rao-v 45 days ago [-]
It’s really interesting how much little choices make the result better or worse. Astra and one of the GLMs added bright lights, and thus looked so much better to my eye.
Genuinely happy with some of the Qwen 3.8 results (especially since I can run that model at Q8).
Interesting to see how much better (at this task) Pi (OMP) is over Opencode as a harness.
I’d love to see a few more with outcomes that are as easy to judge but less subjective.
I’ve got a toy project going to make a fun to watch battle simulator where an LLM (or two if playing vs) has to write programs that control multiple bots (each with their own line of sight and limited battle context) that have to coordinate and fight alongside each other. Goal is to have the LLM update the code based on current situations maybe 5-10 times in a 5 min simulated battle. Exploring even allow the bots to request new programming and score based on number of reprogram steps.
alvins82 45 days ago [-]
Interesting also that Astra and Sol didn't check visually using the browser. I suspect they would have done even better if they had.
andai 45 days ago [-]
Yeah, most of them made it too dark to see what's going on.
Also your last paragraph sounds like the setup for a late 80s scifi movie...
riversflow 45 days ago [-]
huh. goes to show how much browser selection impacts things. the OMP Qwen result is completely unresponsive in free cam on my iphone, whereas the opencode version is responsive, has multitouch zoom and multitouch pan. the glm models do really well with this as well, but all of the anthropic models have pretty gimped camera controls, even astra, which while pretty wont let me pan and only allows me to rotate about 120 degrees! I’m really impressed with the Qwen 27b OpenCode result.
I suppose the only thing that the prompt asks for is the cinematic view, and honestly they all kinda fail on the “subtle volumetric-style fog planes”, none of them have more fog when you get farther from a light source.
utopiah 45 days ago [-]
I wish there was another column with the estimated cost for each, with a specific date.
Ideally also finding somehow (not sure what would be the right away) what is publicly available before running the test. It's quite a different outcome if there are competitions, e.g. js13k, live code examples from books, even templates, on specific that topic. Visually here the results looks very very similar to the point that I can't help but wonder if it's the result from the short yet relatively descriptive prompt or because some template was always found and relied on.
Thanks, sorry to be asking for more but what about a Web visualization of the transcripts?
alvins82 44 days ago [-]
Added costs.
utopiah 44 days ago [-]
Much appreciated
anotherCodder 44 days ago [-]
[dead]
poilcn 45 days ago [-]
Nice. But these tests raise a question what results are reproducible, the final visual, time, tool calling or it's mostly noise. Like, have you tried same combination or model and harness multiple times?
alvins82 45 days ago [-]
I have on Qwen3.8 27B. Results are close enough across multiple runs.
But I am annoyed at these GUIs implementing features I don’t care about. I want them to just wrap my harness and forward it to my iPhone, but they can’t help themselves from feature creep.
I did have some issues getting it installed on a headless server. I sort of gave up and installed the instance that I use as the remote server on a Debian + xfce machine I had laying around.
dandaka 45 days ago [-]
I found it appalling when working with remote setup — tabs disappearing and appearing after some time, screen hangs up, projects disappear. Tried to fix some issues with PR, but team is not accepting for 2w+.
alvins82 45 days ago [-]
I did try this - I found it overly dev focused - e.g. git +/- changes in left sidebar. I wanted something close to what I am use to with Codex/Claude - just for open models.
sdey11 45 days ago [-]
i liked paseo too but its not optimized at all. let alone the feature bloat.
Axsuul 45 days ago [-]
What do you use then?
mmmmbbbhb 45 days ago [-]
DeepSeek harness is for you
anilgulecha 45 days ago [-]
I created my own in-browser terminal, so it can sit in chrome, accessible, and with some enhancements like animating the tab when the agent is doing some work. It's a layer on top of ttyd/tmux.
I have been building something like this with human voting, elo and ranking on multiple scenes and many models.
reilly3000 45 days ago [-]
I love everything about OpenCode except for the stuff it outputs. On paper it has everything I want in a harness and more, but it tends to struggle to deliver the desired outcome. Last I used it the context was massive and tool calls were reliably unreliable. Is it worth revisiting as a daily driver?
polytely 45 days ago [-]
i moved from opencode to omp and i don't think I'm ever going back
alvins82 45 days ago [-]
Why not ohmypi?
shock 45 days ago [-]
Cache rate for oh-my-pi is significantly lower than opencode's in your results.
meerita 45 days ago [-]
I saw examples on X of people building amazing things with Three.js and models, and I figured out how to build them myself. But the results are mostly low-poly materials and crappy animations (pretty sure because I am not the best at these things). I thought it was my prompt (it was detailed), but it seems to be mostly a limitation of the models.
From all the examples I've seen, Astra does it really well, and I suspect it's because they wanted to attract game designers, so they trained the model more on 3D, animation libraries, etc.
sampullman 45 days ago [-]
Sol and Opus are pretty good with the right prompting and feedback framework.
Or, my standards are lower. It's sometimes hard to tell in these discussions whether people are talking about getting production quality results, or stuff that's good enough for a one off blog post.
aetherspawn 45 days ago [-]
Would be good to include Fable and Opus. Based on the results with Astra vs GLM (not even in the same ballpark), these are probably the realistic alternatives to consider.
bensyverson 45 days ago [-]
Yes, I'd be really interested to see Fable / Opus / Sonnet.
GLM 5.3 Flash Max had an interesting showing. Its Codex version was bad [0], and it completed in 9 minutes. The OpenCode version was much richer [1] and more detailed, completed in 20 minutes. And the OMP version was arguably the most complete [2], completing in 30 minutes.
This is probably the strongest argument for the effect of a harness, and I'd be interested to learn the differences in the prompts and tools between these three.
I’m always confused, are all these shapes programmatically generated or are they downloaded from some source?
Also I think Astra looks the best and has the best functionality. Also shocked how much better GLM is on the Non Codex harnesses. Didn’t think it would make such a difference.
Would be nice if you could include cost in the table
alvins82 44 days ago [-]
Added costs.
karlkloss 45 days ago [-]
What I'm missing in all of those one-shot tests:
How different are the results between multiple runs of the same setup?
alvins82 45 days ago [-]
I have done multiple runs on Qwen3.8 27B. Results are close enough across multiple runs qualitatively.
hudo 45 days ago [-]
Damn its unreal how much Astra is better than anything else :(
Would be cool to see total cost
45 days ago [-]
alvins82 44 days ago [-]
Added costs.
andai 45 days ago [-]
WTF, Astra looks like a Unity demo project. That's way better than I was expecting.
Would love to see Claude and Gemini as well. (And it would be nice to include the total cost in the table.)
alvins82 44 days ago [-]
Added costs
BurpyDave 45 days ago [-]
So where did most of them get the idea of calling it hanger 07?
alvins82 45 days ago [-]
[flagged]
jonasdn 45 days ago [-]
Where did they all get "Hangar 07" from?
fig781 45 days ago [-]
Qwen with open code seemed like the best to me. It seemed like the best balance between performance and visuals
amenghra 45 days ago [-]
Astra has so much more detail. It did take 5x more time, but that's usually negligible when it replaces x hours of human output.
Refreshing to see a post that gets straight to the point and not hide the data behind pages of prose.
_pdp_ 45 days ago [-]
How do you judge which one produces the best result? They all look the same to me more or less.
simonwsimonw 45 days ago [-]
[dead]
alexchamberlain 45 days ago [-]
Am I missing something or does sorting by Duration not work? Chrome on Android mobile
stuaxo 45 days ago [-]
We have differing ideas of what a simple prompt contains.
andai 45 days ago [-]
Build me an actual hangar. Make no mistakes.
webprofusion 45 days ago [-]
This is rad, the Sol one is probably the coolest.
consumer451 45 days ago [-]
Why no Antrhopic models?
chvid 45 days ago [-]
What is DSH / PTC?
alvins82 45 days ago [-]
Deepseek Harness. PTC is one of the 'modes' it has.
faangguyindia 45 days ago [-]
Ptc is programmatic tool calling or code mode
Basically it can run these mini programs where each input might be another toolcall, so it can run without waiting for whole LLM response and ready the parameters async.
yanhangyhy 45 days ago [-]
to me i think the diffrenece is not that big?
alvins82 45 days ago [-]
[dead]
knighthacker 45 days ago [-]
[flagged]
Trapstreetrun 44 days ago [-]
[flagged]
ankitmathur01 45 days ago [-]
[flagged]
imposter 45 days ago [-]
[dead]
quietraster 45 days ago [-]
[flagged]
0xanushshetty 45 days ago [-]
[flagged]
Rendered at 06:51:53 GMT+0000 (Coordinated Universal Time) with Vercel.
I don't think any of these examples are using things like tone mapping so they're stuck in sRGB (AgX or ACES look much better), they're not using the node materials (good for programmatic texture implementation), and they're not doing anything cool like baking shadow environments or using post-processing effect.
They're nice, but I think they're showing how far behind AI models are on this sort of project rather than how good they are.
There are thousands, if not millions now, of people working on the problem, a vast majority of humanity text output used for training, an enormous infrastructure composed of the most complex human made device (the processor), a gigantic amount of energy running it, for an attempt that spanned decades, if not century, heck if not millennia if you go as far as considering Antikythera.
It might be impressive but it's anything but a miracle when we consider how much effort was poured into this.
The models _can_ do it, but you need to ask for the right things. Most people don't, they'll usually blame the browser for being slow or ugly when they can't break through the THREE demo page wall.
A larger THREE.js project starts to look more and more like a game engine, so you pick and choose the parts you need. There's a ton of open source libs, most of the heavy components the big players use are open source, things like physics, mesh optimization.
Key AI-specific parts might be:
- a harness (so the agent can drive the thing)
- authoring pipeline (so you can bake/optimize assets)
- some sort of coherent renderer architecture (what are your assets, your passes, what's your shader graph).
Without some fundamentals here you are on the short road to falling off the cliff of tech debt and the AI will gladly drive you off of it until you ask for an expensive rewrite.
> would you create models independently
Yes. Pretty much any proven gamedev/asset pipeline is something frontier models are good at. Procedural systems, asset store, free content, Blender, Meshy.
Literally anything except "make a THREE.js scene" -- which is not a technique used in gamedev, beyond throwaway prototypes or demos. Which is what you will get if you ask for a THREE.js scene.
I found the remarkable https://babylonjs.com/lite-demos/
Both the regular and lite demos are pretty amazing for browserware!
Genuinely happy with some of the Qwen 3.8 results (especially since I can run that model at Q8).
Interesting to see how much better (at this task) Pi (OMP) is over Opencode as a harness.
I’d love to see a few more with outcomes that are as easy to judge but less subjective.
I’ve got a toy project going to make a fun to watch battle simulator where an LLM (or two if playing vs) has to write programs that control multiple bots (each with their own line of sight and limited battle context) that have to coordinate and fight alongside each other. Goal is to have the LLM update the code based on current situations maybe 5-10 times in a 5 min simulated battle. Exploring even allow the bots to request new programming and score based on number of reprogram steps.
Also your last paragraph sounds like the setup for a late 80s scifi movie...
I suppose the only thing that the prompt asks for is the cinematic view, and honestly they all kinda fail on the “subtle volumetric-style fog planes”, none of them have more fog when you get farther from a light source.
Ideally also finding somehow (not sure what would be the right away) what is publicly available before running the test. It's quite a different outcome if there are competitions, e.g. js13k, live code examples from books, even templates, on specific that topic. Visually here the results looks very very similar to the point that I can't help but wonder if it's the result from the short yet relatively descriptive prompt or because some template was always found and relied on.
I wanted a powerful GUI+harness setup for open models so I could use/test as they came out.
But I am annoyed at these GUIs implementing features I don’t care about. I want them to just wrap my harness and forward it to my iPhone, but they can’t help themselves from feature creep.
https://github.com/stablyai/orca
I did have some issues getting it installed on a headless server. I sort of gave up and installed the instance that I use as the remote server on a Debian + xfce machine I had laying around.
https://github.com/anilgulecha/ttydterm
If you don't have anything working check the console, maybe a WebGL issue.
I have been building something like this with human voting, elo and ranking on multiple scenes and many models.
From all the examples I've seen, Astra does it really well, and I suspect it's because they wanted to attract game designers, so they trained the model more on 3D, animation libraries, etc.
Or, my standards are lower. It's sometimes hard to tell in these discussions whether people are talking about getting production quality results, or stuff that's good enough for a one off blog post.
GLM 5.3 Flash Max had an interesting showing. Its Codex version was bad [0], and it completed in 9 minutes. The OpenCode version was much richer [1] and more detailed, completed in 20 minutes. And the OMP version was arguably the most complete [2], completing in 30 minutes.
This is probably the strongest argument for the effect of a harness, and I'd be interested to learn the differences in the prompts and tools between these three.
[0]: https://alvins82.github.io/hangar-harness-model-tests/hangar...
[1]: https://alvins82.github.io/hangar-harness-model-tests/hangar...
[2]: https://alvins82.github.io/hangar-harness-model-tests/hangar...
Also I think Astra looks the best and has the best functionality. Also shocked how much better GLM is on the Non Codex harnesses. Didn’t think it would make such a difference.
Would be nice if you could include cost in the table
How different are the results between multiple runs of the same setup?
Would love to see Claude and Gemini as well. (And it would be nice to include the total cost in the table.)
Basically it can run these mini programs where each input might be another toolcall, so it can run without waiting for whole LLM response and ready the parameters async.