Huge +1. We used a custom decision model to improve recall of our retrieval pipeline by ~1.5x while reducing cost by ~70x compared to just Opus. We used the custom decision model to quickly and cheaply drop a larger number of egregious results and have the LLM only look at the cases the smaller model was not sure about.
We believe in this pattern so much that we are building Princi (https://princi.ai/) to help teams train and host their own custom decision models. It is great to see this approach gaining traction.
2 hours ago [-]
Rendered at 21:08:52 GMT+0000 (Coordinated Universal Time) with Vercel.
We believe in this pattern so much that we are building Princi (https://princi.ai/) to help teams train and host their own custom decision models. It is great to see this approach gaining traction.