This looks super clean. I'm curious about the --judge command. How does it evaluate if the cheaper model's response is a "tie" or acceptable? Is it using a specific LLM-as-a-judge prompt template?
XUEYANZ 5 hours ago [-]
Haha. somehow i just love the naming. it just makes sense :D
jarodrh 1 hours ago [-]
Thanks! I love a good meaningful name
Frugal + on
:P
cyanydeez 6 hours ago [-]
this would be more interesting as a local LLM anlysis; throw out all the costs, and figure out primary-subagent model architecture, and maximize token generation and prefill.
I don't see how anyone can operationalize this information.
jarodrh 1 hours ago [-]
That's an interesting thought, and one I'll take note of, but that would be a different tool. However, if you look hard enough I'd say we're tackling the same issue. I'm just choosing to look at the problem from a cost perspective as opposed to a raw token generation/prefill perspective. The mindset can be applied to both sides, but Frugon prices the cloud side.
The end goal is essentially the same and your mind went to that point, "primary-subagent model architecture". This is what the tool helps you figure out. It's not there to hold your hand and explain what your architecture should be, as I wanted it to be a small simple tool that would give the user insight into triggering your exact thought process. The thought process of breaking down their tasks by type, mapping that to individual models regardless of app or dev tool, regardless of cloud or local (the thought process transfers). It shows the user that they could route a portion of their calls to a cheaper model. It's then up to the user to understand the task type and point those calls to different models.
To directly answer the operationalization statement, it depends on what the call log is from (app/harness).
If harness, then the direction would be to pin models per role per task type; the routing recommendation maps directly to that, and measure/judge allows you to verify this before switching.
If app, then this is a similar shape as above, where you would then categorise and pin those calls identified in the recommendation to the model recommended, and as above, measure/judge allows you to verify before switching.
You're proposing an optimisation for throughput whereas Frugon is for spend. Same issue, just different lens.
arspesk 8 hours ago [-]
[flagged]
21 hours ago [-]
Rendered at 06:30:27 GMT+0000 (Coordinated Universal Time) with Vercel.
Frugal + on
:P
I don't see how anyone can operationalize this information.
The end goal is essentially the same and your mind went to that point, "primary-subagent model architecture". This is what the tool helps you figure out. It's not there to hold your hand and explain what your architecture should be, as I wanted it to be a small simple tool that would give the user insight into triggering your exact thought process. The thought process of breaking down their tasks by type, mapping that to individual models regardless of app or dev tool, regardless of cloud or local (the thought process transfers). It shows the user that they could route a portion of their calls to a cheaper model. It's then up to the user to understand the task type and point those calls to different models.
To directly answer the operationalization statement, it depends on what the call log is from (app/harness).
If harness, then the direction would be to pin models per role per task type; the routing recommendation maps directly to that, and measure/judge allows you to verify this before switching.
If app, then this is a similar shape as above, where you would then categorise and pin those calls identified in the recommendation to the model recommended, and as above, measure/judge allows you to verify before switching.
You're proposing an optimisation for throughput whereas Frugon is for spend. Same issue, just different lens.