Grafbase just launched Nexus, an open-source AI Router that unifies MCP servers and LLMs through a single endpoint. Designed for enterprise-grade governance, control, and observability, Nexus helps teams manage AI complexity, enforce policies, and monitor performance across their entire stack.
Built to work with any MCP server or LLM provider out-of-the-box, Nexus is designed for developers who want to integrate AI with the same rigor as production APIs.
CptanPanic 179 days ago [-]
Sounds like litellm which I use, I wonder how it compares?
vid 179 days ago [-]
There is also https://github.com/maximhq/bifrost which apparently overcomes some performance issues of litellm and is easy to get going.
tomhoule 179 days ago [-]
Yeah they definitely belong in the same space. Nexus is an LLM Gateway, but early on, the focus has been on MCP: aggregation, authentication, and a smart approach to tool selection. There is that paper, and a lot of anecdotal evidence, pointing to LLMs not coping well with a selection of tools that is too large: https://arxiv.org/html/2411.09613v1
So Nexus takes a tool search based approach to solving that, among other cool things.
Disclaimer: I don't work on Nexus directly, but I do work at Grafbase.
fbjork 179 days ago [-]
Founder of Grafbase here.
Here are a few key differentiators vs LiteLLM today:
- Nexus does MCP server aggregation and LLM routing - LiteLLM only does LLM routing
- The Nexus router is a standalone binary that can run with minimal TOML configuration and optionally Redis - LiteLLM is a whole package with dashboard, database etc.
- Nexus is written in Rust - LiteLLM is written in Python
That said, LiteLLM is an impressive project, but we're just getting started with Nexus so stay tuned for a steady barrage of feature launches the coming months:)
SparkyMcUnicorn 179 days ago [-]
What's the difference between "MCP Server Aggregation" and the litellm_proxy endpoint described here?
The main difference is that while you can get Nexus to list all tools, by default the LLM accesses tools by semantic search — Nexus returns only the relevant tools for the what the LLM is trying to accomplish. Also, Nexus speaks MCP to the LLM, it doesn't translate like litellm_proxy seems to do (I wasn't familiar with it previously).
Seems quite similar to the commercial nexos.ai platform, which also focuses on routing, governance, and observability for AI workloads, but as a proprietary solution rather than open source
fbjork 179 days ago [-]
From what I can tell they don’t offer a self-hosted router?
johntash 178 days ago [-]
It looks like you're planning on monetizing this (which is totally fine!), do you have any plans on what the enterprise version would do differently?
echelon 178 days ago [-]
And isn't OpenRouter already open source?
johntash 178 days ago [-]
OpenRouter isn't open source that I know of, but there are open source things similar like Litellm.
mbrumlow 179 days ago [-]
I thought it was a phone :/, for developers.
fbjork 179 days ago [-]
That phone was discontinued:)
bentogrizz 179 days ago [-]
This is cool
fbjork 179 days ago [-]
What are you building?
owenthejumper 179 days ago [-]
Another proxy?
fbjork 179 days ago [-]
MCP aggregation is one of the big differentiators
barbazoo 179 days ago [-]
I'm curious, what issue does that solve? I'm only working on agents that make tool calls via HTTP in a home baked way but I can't imagine how resolving the tools from 2 MCP servers is harder than 1.
fbjork 178 days ago [-]
The issue is when you have many MCP tools the context becomes too large for the LLM. So Nexus indexes all the tools and lets you search for the right tool and then execute it.
barbazoo 178 days ago [-]
Thanks, I think I get it now. In our case I've dealt with this problem by refactoring the monolithic agent into smaller agents, smaller, more specific prompts, fewer, more relevant tools.
We've found that monolithic agents just don't perform that well.
barbazoo 179 days ago [-]
> There is no problem that can't be solved by another level of indirection.
David Wheeler
davinoishi 178 days ago [-]
[dead]
Rendered at 12:02:59 GMT+0000 (Coordinated Universal Time) with Vercel.
So Nexus takes a tool search based approach to solving that, among other cool things.
Disclaimer: I don't work on Nexus directly, but I do work at Grafbase.
Here are a few key differentiators vs LiteLLM today:
- Nexus does MCP server aggregation and LLM routing - LiteLLM only does LLM routing
- The Nexus router is a standalone binary that can run with minimal TOML configuration and optionally Redis - LiteLLM is a whole package with dashboard, database etc.
- Nexus is written in Rust - LiteLLM is written in Python
That said, LiteLLM is an impressive project, but we're just getting started with Nexus so stay tuned for a steady barrage of feature launches the coming months:)
https://docs.litellm.ai/docs/mcp
Nexus name already has a taker
We've found that monolithic agents just don't perform that well.
David Wheeler