It's implied, and I'm hoping it's true, that this is a map-less navigation. Which is impressive. This kind of task is much easier if you have a pre-captured map of the environment, but if they are doing this without a map it's great. Historically you were always faced with "The Kidnapped Robot" problem where robots that didn't know where they were couldn't navigate even a little bit. Here the robot appears to be able to follow directions as long as they are interpretable from its current vision (or via dead reckoning).
duchenne 4 minutes ago [-]
I am working in Mistral robotics team. I confirm this is map-less. The only inputs are the text prompt and the front camera rgb image.
ainch 7 minutes ago [-]
Wouldn't modern SLAM or VSLAM address that problem?
humanperhaps 42 minutes ago [-]
This looks to not be an openly available model, but I think if it were, availability of an easy single-camera navigation setup could allow for a lot of cool hobbyist projects.
LurkandComment 40 minutes ago [-]
If you're wondering what prevents or mitigates AI hallucinations on the AI layer from replicating or acting out on the physical layer look up QNX. They manage the deterministic reasonin gof robotics. You know them better as Blackberry.
jonash54 55 minutes ago [-]
Producing specific niche models for 100 year old industries that have mountains of data and warehouses full of folders will be the european take on AI.
It may come late but it‘ll be safe and reliable. It also requires a lot of OCR.
lumost 52 minutes ago [-]
The Niche model story is still fairly week. Evidence points to general models being equally capable to niche models at a more attractive capex (risk is spread across multiple verticals rather than concentrated in a single model capability)
berkes 24 minutes ago [-]
The General Models' business-model is also looking more weak every iteration.
Costs of simple tasks grow extensively: OCR with "Mistral OCR" at $4 per 1000 pages vs OCR with Opus 4.8 at sometimes¹ $1 per "page".
Or just the immense costs when burning tokens in an unoptimized agentic coding environment costing tens of dollars for a few simple classes or functions versus a highly optimized "autocomplete" model costing under $10 for thousands of such classes and functions.
Or the, over ten dollars worth of tokens when some "agent" using a general model, tries to perform the task I gave it to "read the event on example.com/event/1337 and put it in my calendar", include commute time as well"
The "general models" currently only become smarter by growing bigger and having larger context windows - by becoming exponentially more expensive to train and to run and to interact with. Whereas "Niche" models can do the things that "normal code" cannot do, and improve by tuning and tweaking only that. Their goal is then to fill in gaps that traditionally are hard or impossible with normal software. Wheras the goal of a general model (with agentic reasoning)is to replace that entire "normal software".
One example: I am not interested in "chatting with my calendar". I'm interested in a calendar because it is a well known view (UI) of my planning and tasks, but I see a lot of opportunities where AI can improve my working with this calendar. I may be interested in a smarter screen when I hit "+ Add event"; one that has knowledge of my previous events and patterns (some RAG vector db maybe). One that maybe has access to content I just copied, or read (though: privacy?) or can open my camera to let me shoot a pic of something that has the event info on it.
In such a set-up, Niche LLMs perform dedicated tasks: determine patterns (he always books a Yoga class on wednesday or thursday, two days in advance, so lets suggest a yoga class), determine existing content (event is planned 100Km from his home, so lets suggest the commute based on previous commutes like this). Or an OCR model. Or an autocomplete model. Relatively simple, niche models, called from within software to aid me when "calendaring". Not replace the entire calendar with some chat.
philipkglass 44 minutes ago [-]
It seems like a stronger story for robotics, since smaller models can always react to the environment faster than large models at a given hardware budget. Also because robots that keep their models local for latency or reliability aren't going to be carrying many kilowatts of inference capacity.
lumost 24 minutes ago [-]
remote inference should be sufficient for most robotics applications with potentially a small model for safety critical actions running locally.
Unless you are in military robotics or automotive of course :)
AlotOfReading 15 minutes ago [-]
There are many, many factories that still don't have internet access on the floor, and commercial inference generally has response latencies measured in seconds. I struggle to imagine a factory spending hundreds of thousands for the local compute to run a large model either, given how cheap they are about expenses.
I'm also skeptical that you can cleanly differentiate between "safety critical actions" and "actions", though this is less of a practical concern given how laissez-faire some manufacturers are. For context, I work on safety critical robotics (in automotive).
baq 40 minutes ago [-]
I expect the bitter lesson to continue to be bitter. Mistral must at least attempt to catch up to SOTA 6 months ago.
mil22 1 hours ago [-]
> achieves 76.6% on R2R-CE (Room-to-Room in Continuous Environments)
I would like to know what it did the other 23.4% of the time!
dwa3592 1 minutes ago [-]
maybe it did a cartwheel instead of turning right.
semiquaver 55 minutes ago [-]
Presumably it did not make it to the other Room.
mhitza 1 hours ago [-]
For a claim such as state of the art, or claims such as "great at any task" needs something of more substance. I've seen maze-solving robot competitions which can zoom around in seconds. The sped up video in the first part, and the "obstacle avoidance" are too slow for me to believe this is state of the art.
While impressive at 8B, what would the expectation be in real life, that it's run remotely or autonomously with a strapped on GPU and battery?
nancyminusone 52 minutes ago [-]
it is state of the art, those maze solving things are a different art.
mhitza 48 minutes ago [-]
I've used that example as a contrast of what I've seen before. If you can point me at comparable efforts, in the same category as what Mistral is doing, I'd be interested in having a comparative look.
All I can think of are robot dogs, Tesla bots, and whatever flavor of the month Japanese robots show up at trade shows.
ImageXav 1 hours ago [-]
Ok, this is really cool. The fact that the robot can use pointing to decide where to go is a great design decision, and robotics really is the next frontier. Definitely cheering on Mistral here!
gunalx 35 minutes ago [-]
No word on pricing or inference options i could see so not that interresting if it is not available to test.
skaiuijing 1 hours ago [-]
Robots handle clean labs well; messy real‑world environments are still the real bottleneck.
heyheyhouhou 1 hours ago [-]
Maybe their LLMs are not the best but design is top-notch!
montroser 1 hours ago [-]
I'm ready for my home helper robot that makes dinner and does the dishes and takes out the trash.
But I'm scared for when those home helpers get drafted to fight in wars, either for or against me...
ainch 5 minutes ago [-]
I think you'll be waiting a while for the former, unless you're ok with strangers teleoperating a robot around your house whenever it gets confused.
cbg0 14 minutes ago [-]
You should be relieved that they're sending robots instead of you to get blown up by a drone.
stackbutterflow 14 minutes ago [-]
One intelligent humanoid robot per house. What could go wrong really. Possibly the worst idea.
toyg 1 hours ago [-]
I suspect the latter will come way before the former...
NitpickLawyer 9 minutes ago [-]
It is already here. Not humanoid (yet, but it's in the works) but tracked robots with bolted on machine guns have both held and captured positions in UA.
Gecko4072 2 hours ago [-]
Mistral seems to be going wide and niche. Could be a smart strategy going forward.
figassis 35 minutes ago [-]
How long until Tesla buys Mistral?
davidpapermill 32 minutes ago [-]
I don't think so. I think Tesla merger with SpaceX, which has the Cursor team and reportedly working on foundation model there.
I imagine the EU would block any attempted takeover of Mistral given recent Anthropic and US govt actions.
Frontier labs are realizing that software/models themselves don’t have real moats and move to embodied ai.
SOTA 80% means a practically useless robot. What are they really imagining their ICP to be here?
not-kinsale-joe 35 minutes ago [-]
What does this comment mean?
fzysingularity 10 minutes ago [-]
It’s unclear to me what their desired outcome for a blog post like this. If you’ve ever worked in a robotics setting, 80% implies that 20% of your autonomous actions are incorrect. Imagine if this were the case for autonomous driving where your car misbehaves 1 in every 5 actions it takes.
Posts like this just reminds me of the end to end demos AV companies built in the early days using a single camera - only to realize that it’s harder than it looks years later into development.
fzysingularity 6 minutes ago [-]
The ICP question was more around the model itself. Are they looking to license it to robotics companies? Do they imagine that devs at robotics companies would be willing to deploy these models as a black box?
It may come late but it‘ll be safe and reliable. It also requires a lot of OCR.
Costs of simple tasks grow extensively: OCR with "Mistral OCR" at $4 per 1000 pages vs OCR with Opus 4.8 at sometimes¹ $1 per "page".
Or just the immense costs when burning tokens in an unoptimized agentic coding environment costing tens of dollars for a few simple classes or functions versus a highly optimized "autocomplete" model costing under $10 for thousands of such classes and functions.
Or the, over ten dollars worth of tokens when some "agent" using a general model, tries to perform the task I gave it to "read the event on example.com/event/1337 and put it in my calendar", include commute time as well"
The "general models" currently only become smarter by growing bigger and having larger context windows - by becoming exponentially more expensive to train and to run and to interact with. Whereas "Niche" models can do the things that "normal code" cannot do, and improve by tuning and tweaking only that. Their goal is then to fill in gaps that traditionally are hard or impossible with normal software. Wheras the goal of a general model (with agentic reasoning)is to replace that entire "normal software".
One example: I am not interested in "chatting with my calendar". I'm interested in a calendar because it is a well known view (UI) of my planning and tasks, but I see a lot of opportunities where AI can improve my working with this calendar. I may be interested in a smarter screen when I hit "+ Add event"; one that has knowledge of my previous events and patterns (some RAG vector db maybe). One that maybe has access to content I just copied, or read (though: privacy?) or can open my camera to let me shoot a pic of something that has the event info on it. In such a set-up, Niche LLMs perform dedicated tasks: determine patterns (he always books a Yoga class on wednesday or thursday, two days in advance, so lets suggest a yoga class), determine existing content (event is planned 100Km from his home, so lets suggest the commute based on previous commutes like this). Or an OCR model. Or an autocomplete model. Relatively simple, niche models, called from within software to aid me when "calendaring". Not replace the entire calendar with some chat.
Unless you are in military robotics or automotive of course :)
I'm also skeptical that you can cleanly differentiate between "safety critical actions" and "actions", though this is less of a practical concern given how laissez-faire some manufacturers are. For context, I work on safety critical robotics (in automotive).
I would like to know what it did the other 23.4% of the time!
While impressive at 8B, what would the expectation be in real life, that it's run remotely or autonomously with a strapped on GPU and battery?
All I can think of are robot dogs, Tesla bots, and whatever flavor of the month Japanese robots show up at trade shows.
But I'm scared for when those home helpers get drafted to fight in wars, either for or against me...
I imagine the EU would block any attempted takeover of Mistral given recent Anthropic and US govt actions.
SOTA 80% means a practically useless robot. What are they really imagining their ICP to be here?
Posts like this just reminds me of the end to end demos AV companies built in the early days using a single camera - only to realize that it’s harder than it looks years later into development.