This looks awesome. As someone working in support for a wide array of Linux Apps, and data dumps from customers where I have no access to the system, plus I also write or backport bug fixes to all sorts of random software, I often want to do this kind of crazy stuff. With exactly these kinds of artefacts.
snthpy 1 days ago [-]
Very cool!
How does this interface with the different tools and how would one add another tool for it to operate on?
I started on something similar last year which was just a simple bash script to interact with things like osquery. Alas it was too buggy for what I wanted to do and it's paused indefinitely for now.
Puchaczov 1 days ago [-]
You can query with --format [raw|csv|json|interpreted_json]
This will output pure json or csv. This way you can use other tools like jq, grep, csvtoolkit or whatever you need further process your data.
After some thought: you could have also asked whether it's possible to add new data sources that you need to query, and the answer is of course yes! It's actually quite simple and there are many examples. Each data source tool is just a plugin implementing the appropriate interfaces. You can look at some example projects and see how they implement their data source here:
https://github.com/Puchaczov/Musoq.DataSources
snthpy 1 days ago [-]
Thank you for your reply.
Yes, I was asking about new data sources, so for example if I wanted to add Github to query my GH issues with Musoq, how I would do that.
Great, I'll check out the links!
johnthescott 11 hours ago [-]
in postgresql syntax for data types is stored in the database. how would this tool parse gis expressions, for example?
Puchaczov 9 hours ago [-]
You actually don't need any special type system or complex infrastructure for this. Each data source can handle its own data representation and operations.
For GIS data, you could create a plugin that naturally handles spatial operations. Here's how it could work:
select
Id,
DistanceBetween(s.FromPoint(-73.935242, 40.730610)) as Distance,
s.IsBetween(s.FromPoint(-73.935242, 40.730610)) as IsInArea
from #gis.shapefile('map.shp') s
You could even combine it with other data sources. For example, if you have geometry data in a CSV:
select
sfg.DistanceBetween(sfg.FromPoint(..., ...))
from #csv.file(...) c
cross apply #gis.ShapeFromGeometry(c.Geometry) sfg
cryptoalex 2 days ago [-]
Hey I like your project, earned a star from me! When time allows, will take it on a test drive to see how exactly it works with Roslyn/C# data. My C# solution has grown to about 80 projects so it would be good test.
Puchaczov 1 days ago [-]
Thanks!
It would be great to ask what kind of usage you have in mind? I've currently been using the Roslyn plugin mainly with the engine repository itself because it helps me extract queries from tests, I use it in combination with GPT, which helps me create documentation. In the longer term, I would like this plugin to help me refactor some problematic code fragments, but for that I will need to further develop the Roslyn integration itself.
re_spond 1 days ago [-]
Where would you place this between osquery and steampipe? It seems to borrow concepts on both sides, but I'm not sure how it could not be plugin for either.
Puchaczov 1 days ago [-]
I see it a bit that way: while osquery focuses on OS-level queries and steampipe on cloud services, Musoq is more developer-centric, I'm using it like swiss army knife for various thing, something like sed or grep. You can ofcourse create plugins that covers what mentioned tools do or even, use that tools as a plugin for musoq but in general - I'm not going to compete with any of them, I'm filling my own niche - flexible developer tool.
re_spond 1 days ago [-]
That helps, thanks! It is all about the framing. I love these kinds of tools and have been using a combination of them together with nushell, but it is a road less travelled it seems. All the more reason to evangelise your tool :) keep up the good work.
cachvico 2 days ago [-]
Ironically I think I'd rather query a database with anything other than SQL ¯\_(ツ)_/¯
Puchaczov 1 days ago [-]
Do you have something specific in your mind? I would love to hear that! Unexpected approach is what driving me usually
Rendered at 16:10:19 GMT+0000 (Coordinated Universal Time) with Vercel.
How does this interface with the different tools and how would one add another tool for it to operate on?
I started on something similar last year which was just a simple bash script to interact with things like osquery. Alas it was too buggy for what I wanted to do and it's paused indefinitely for now.
This will output pure json or csv. This way you can use other tools like jq, grep, csvtoolkit or whatever you need further process your data.
to dig deeper, just look at: https://github.com/Puchaczov/Musoq.CLI
After some thought: you could have also asked whether it's possible to add new data sources that you need to query, and the answer is of course yes! It's actually quite simple and there are many examples. Each data source tool is just a plugin implementing the appropriate interfaces. You can look at some example projects and see how they implement their data source here: https://github.com/Puchaczov/Musoq.DataSources
Yes, I was asking about new data sources, so for example if I wanted to add Github to query my GH issues with Musoq, how I would do that.
Great, I'll check out the links!
select Id, DistanceBetween(s.FromPoint(-73.935242, 40.730610)) as Distance, s.IsBetween(s.FromPoint(-73.935242, 40.730610)) as IsInArea from #gis.shapefile('map.shp') s
You could even combine it with other data sources. For example, if you have geometry data in a CSV:
select sfg.DistanceBetween(sfg.FromPoint(..., ...)) from #csv.file(...) c cross apply #gis.ShapeFromGeometry(c.Geometry) sfg