Sounds cool, I want to try this kind of things out, but do you have or planned to have a sandboxing environment, where the agent can try running the query in let say duckdb first to confirm its validity/result before sending it over to bq? Or use something like tablesample when developing the query to reduce cost?
One more thing, how do you compare with nao ( https://github.com/getnao/nao ), it's something I've followed for a while and seem to answer similar issue as what ktx build
andreybavt 9 hours ago [-]
Thanks! ktx maintainer here, related to the queries sandbox question, we're attacking this problem from a different angle: instead of catching bad queries at runtime we try to make it impossible to generate a bad query. ktx allows agents to generate semantic queries (declarative json) and takes care of translating it to the correct SQL with the help of the join graph that it builds during the data source ingestion (dijkstra path finding + fan-out handling etc.). So the agent doesn't need to think "how" it needs to query the data, it just says "what" it wants and ktx builds queries. Also the join graph allows you combining raw tables and high level ktx metrics.
On nao: it's solving an adjacent but different problem, it's a full analytics app, whereas we believe that the general purpose agents (e.g. Claude Desktop) are getting better at the representation layer generating widgets and that's where people work anyway. What these agents need is a solid foundation - this is what we're focusing on. ktx isn't an agent or UI, it's an executable institutional knowledge that keeps any analyst (nao, Claude, Codex, your own agent) from guessing which table is canonical or which join is safe.
lucamrtl 8 hours ago [-]
Just to complete the answer if that's where your question was going - we actually built a prepackaged project that users can use to experiment with ktx before running any kind of ingestion. In our docs we also have a link to a demo postgres, dbt, metabase, and notion that users can access to try out ktx
And better comparisons from the semantic layer space are tools like Wren, Cube, dbt MetricFlow
9 hours ago [-]
9 hours ago [-]
lifeisstillgood 16 hours ago [-]
So, and I may be oversimplifying, you are creating awesome documents and references that I would have loved to have for my different jobs 5-10 years ago (or more).
It’s just that making such docs had next to no ROI 10 years ago. But today they are the difference between success and failure.
It’s fascinating - thank you
(and who writes the wiki / business rules ? Can they be reverse engineered from existing query stack? )
Sounds great - all the best
Edit: don’t take the above as criticism - just trying to fit new ideas into an old dog.
andreybavt 15 hours ago [-]
Thanks! yes, exactly. But what you described is just half of what ktx does, we call it "wiki" - self organizing collection of ingested markdown files.
The second (equally important) part of ktx is similarly self-organizing executable semantic layer.
It allows agents to send simple declarative "I want X sliced by Y" requests instead of imperative "X sliced by Y that has to be queried this way and joined the other way ..."
To answer your questions:
> who writes the wiki / business rules
most is create after `ktx ingest` pulls raw data from your data stack, it can be edited/created manually, but it's rarely necessary. Also if you plug ktx to your agent it allows that agent to send extra memories for ingestion to keep the context up to date
> Can they be reverse engineered from existing query stack
Yes, that's the case when we analyze historic SQL queries or scan Looker or Metabase dashboards for example
lucamrtl 15 hours ago [-]
Just to clarify what semantic layer is: you can think of it as a set of functions
So you define the definitions of metrics like "monthly revenue" once, and that definition is runnable. If an agent asks for "monthly revenue by region", the semantic layer will compile that request into SQL and run it (the same way everytime so that calculation method is consistent)
15 hours ago [-]
15 hours ago [-]
15 hours ago [-]
15 hours ago [-]
tarun_anand 22 hours ago [-]
How does this compare with Wren 2.0, OpenVikings etc
lucamrtl 21 hours ago [-]
Thanks for the question! Just to complete - OpenVikings is context/memory infrastructure for agents. ktx is analytics context for data agents specifically. So they're in different categories
The way we think about the space: there's a "semantic layer" side (Cube, dbt MetricFlow, Wren's engine) that compiles correct SQL but is hand-authored. And a "company brain" side (OpenViking, Glean, wikis) that indexes prose but can't query data warehouses safely. ktx is built to be both halves at once - a YAML semantic layer and a searchable wiki of business definitions, cross-linked (each wiki page references the metrics it explains) and auto-maintained
andreybavt 21 hours ago [-]
Hey, ktx maintainer here!
Great question, we actually have quite a bit in common with Wren, they're in the same category. The main difference is that with Wren you author the semantic model (write MDL) and it executes SQL through its own Rust engine (similar to Cube as well).
ktx instead builds the context for you - ingesting definitions already in dbt, Looker, Metabase, and Notion, auto-detecting joins and fan/chasm traps, flagging contradictions. It also builds both : the SL and wiki that accumulates and maintains free-form tribal knowledge (that’s also interlinked with ktx SL)
modus-tollens 21 hours ago [-]
How are you measuring the accuracy? Are you running this against any benchmarks?
I see this covers a file based approach, was there ever a consideration for a graph based approach?
For business context, how do you handle context that evolves over time?
andreybavt 17 hours ago [-]
Great questions! we're currently working on Spider 2 submission, hope to have first results soon.
It's true that we took file-first approach and not a graph DB. Main reason is that the ktx wiki and semantic layer entities while being written in plain text files (md or yaml) still contain links to each other.
This allows an agent to find the right entry point (with the help of lexical and semantic searches merged with RRF) and then traverse these links to collect enough context.
As for the business context evolution - that's exactly the reason we have ingestion reconciliation and git versioning. The idea is to give ingestion agent a way to deduplicate/consolidate knowledge during the ingestion and leave complex conflicts to humans to resolve
lucamrtl 16 hours ago [-]
And development was done around link detection and text to sql benchmarks to measure/compare different approaches
andreybavt 16 hours ago [-]
yep, happy to share more details with you @modus-tollens if you're interested
hannune 8 hours ago [-]
[dead]
17 hours ago [-]
17 hours ago [-]
17 hours ago [-]
17 hours ago [-]
17 hours ago [-]
yooibox 12 hours ago [-]
cool
lucamrtl 8 hours ago [-]
Thanks!
hipvlady 3 hours ago [-]
[flagged]
germansaprykin 6 hours ago [-]
[dead]
yashjadhav2102 9 hours ago [-]
[flagged]
sid0707 16 hours ago [-]
[flagged]
MadGodInc 19 hours ago [-]
Interesting approach. Context management for agents is an underexplored area. I've been looking at similar problems - the key challenge is keeping token usage low while giving the agent enough context to be useful. Tiered retrieval (facts first, full text only whenneeded) seems to work well in practice.
andreybavt 19 hours ago [-]
Agreed, in addition to this I'd say it's better to spend tokens once while ingesting the sources and storing the canonical reusable definition rather than having agents do the full exploration over and over again
qasimkhan07 19 hours ago [-]
[dead]
Rendered at 15:39:00 GMT+0000 (Coordinated Universal Time) with Vercel.
On nao: it's solving an adjacent but different problem, it's a full analytics app, whereas we believe that the general purpose agents (e.g. Claude Desktop) are getting better at the representation layer generating widgets and that's where people work anyway. What these agents need is a solid foundation - this is what we're focusing on. ktx isn't an agent or UI, it's an executable institutional knowledge that keeps any analyst (nao, Claude, Codex, your own agent) from guessing which table is canonical or which join is safe.
And better comparisons from the semantic layer space are tools like Wren, Cube, dbt MetricFlow
It’s just that making such docs had next to no ROI 10 years ago. But today they are the difference between success and failure.
It’s fascinating - thank you
(and who writes the wiki / business rules ? Can they be reverse engineered from existing query stack? )
Sounds great - all the best
Edit: don’t take the above as criticism - just trying to fit new ideas into an old dog.
The second (equally important) part of ktx is similarly self-organizing executable semantic layer.
It allows agents to send simple declarative "I want X sliced by Y" requests instead of imperative "X sliced by Y that has to be queried this way and joined the other way ..."
To answer your questions:
> who writes the wiki / business rules most is create after `ktx ingest` pulls raw data from your data stack, it can be edited/created manually, but it's rarely necessary. Also if you plug ktx to your agent it allows that agent to send extra memories for ingestion to keep the context up to date
> Can they be reverse engineered from existing query stack Yes, that's the case when we analyze historic SQL queries or scan Looker or Metabase dashboards for example
So you define the definitions of metrics like "monthly revenue" once, and that definition is runnable. If an agent asks for "monthly revenue by region", the semantic layer will compile that request into SQL and run it (the same way everytime so that calculation method is consistent)
The way we think about the space: there's a "semantic layer" side (Cube, dbt MetricFlow, Wren's engine) that compiles correct SQL but is hand-authored. And a "company brain" side (OpenViking, Glean, wikis) that indexes prose but can't query data warehouses safely. ktx is built to be both halves at once - a YAML semantic layer and a searchable wiki of business definitions, cross-linked (each wiki page references the metrics it explains) and auto-maintained
I see this covers a file based approach, was there ever a consideration for a graph based approach?
For business context, how do you handle context that evolves over time?
As for the business context evolution - that's exactly the reason we have ingestion reconciliation and git versioning. The idea is to give ingestion agent a way to deduplicate/consolidate knowledge during the ingestion and leave complex conflicts to humans to resolve