No benchmarks, no info on which models are used, ai generated video, just a signup page with nothing else.
Anyhow, this kinda reminds me of that quote about architecture: "We replaced our monolith with micro services so that every outage could be more like a murder mystery."
guessmyname 33 minutes ago [-]
> No benchmarks, no info on which models are used, […]
They are not good benchmarks but at least they exist.
j45 18 minutes ago [-]
It's easy enough to copy and paste in a prompt, no?
Eval tests while giving general indicators might not be similar for each use case.
adam_rida 9 minutes ago [-]
[dead]
jmaw 27 minutes ago [-]
I think approaches like this have potential. Only time will tell. This reminds me of the mixture of experts taken by deepseek r2 (I think it was r2, at least), but less specific models I guess.
I have often wondered how tools like GHCP choose the best model for the job when set to "auto".
guessmyname 33 minutes ago [-]
The HumanEval+ tests, the sole code benchmark, consist of thirty-one (31) programming questions designed to create a function that solves a straightforward algorithm problem. I find none of these particularly challenging for a model of Fable caliber, so I’m confused by the necessity of the comparison.
Also, when clicking on the “Inspect” button in each of the 31 code tests, there’s a section that supposedly displays the answer, but it’s always empty so the only comparison one can make is whether the code appears identical or at least similar to the one generated by Claude Fable 5.
And one more, there’s a test where Fable supposedly gave the wrong answer while the @OP’s system was correct. However, when I input the same prompt to my instance of Claude Fable 5, as well as all the Mythos variants I have access to (5 in total), all of them return the correct code, which suggests that the @OP may not have run the prompts with enough iterations.
adam_rida 11 minutes ago [-]
[flagged]
bnjemian 4 minutes ago [-]
I don’t find the recent spate of blog posts and systems delegating and combining LLMs to get better performance particularly interesting. Especially given that anyone who’s taken an ML 101 course has learned about ensemble methods.
While an LLM isn’t what you’d traditionally consider a weak learner, the theorems on learning systems clearly point to them being so in this context. The feigned surprise at combining them to yield better results seems disingenuous.
Even so, the work to predict which models are best suited for which task, how to delegate, and how to combine their outputs is interesting, especially if you’re placing a cost minimization objective on it. That said, this isn’t too far off from what many AI labs are already doing.
Alifatisk 23 minutes ago [-]
This reminds me on OpenRouters report that combining multiple different models gave comparable performance to Fable 5. I think this approach has lots of potential. Maybe OpenAi was ahead of its time with GPT-5 (it being a router to different models rather than just being one new model)
tj800x 32 minutes ago [-]
No single signin. Privacy policy allows training. No try it first without credit card. It's a good idea, but this looks premature.
adam_rida 11 minutes ago [-]
[dead]
jambalaya8 11 minutes ago [-]
Might want to rethink the name to avoid an Amazon issue.
codekansas 11 minutes ago [-]
Fable-level, yea, but can it run gstack?
wizche 8 minutes ago [-]
how does this differs from OpenRouter fusion?
fneddy 28 minutes ago [-]
That’s basically the same idea IBM advertises with Bob?
j45 17 minutes ago [-]
If you copy Perplexity, they let you have the first few rounds of chat for free to get you going before asking to sign up.
theneocorner 3 minutes ago [-]
[dead]
Rendered at 20:16:23 GMT+0000 (Coordinated Universal Time) with Vercel.
Anyhow, this kinda reminds me of that quote about architecture: "We replaced our monolith with micro services so that every outage could be more like a murder mystery."
The benchmarks are here → https://echo.tracerml.ai/eval/
They are not good benchmarks but at least they exist.
Eval tests while giving general indicators might not be similar for each use case.
I have often wondered how tools like GHCP choose the best model for the job when set to "auto".
Also, when clicking on the “Inspect” button in each of the 31 code tests, there’s a section that supposedly displays the answer, but it’s always empty so the only comparison one can make is whether the code appears identical or at least similar to the one generated by Claude Fable 5.
And one more, there’s a test where Fable supposedly gave the wrong answer while the @OP’s system was correct. However, when I input the same prompt to my instance of Claude Fable 5, as well as all the Mythos variants I have access to (5 in total), all of them return the correct code, which suggests that the @OP may not have run the prompts with enough iterations.
While an LLM isn’t what you’d traditionally consider a weak learner, the theorems on learning systems clearly point to them being so in this context. The feigned surprise at combining them to yield better results seems disingenuous.
Even so, the work to predict which models are best suited for which task, how to delegate, and how to combine their outputs is interesting, especially if you’re placing a cost minimization objective on it. That said, this isn’t too far off from what many AI labs are already doing.