Roboticist here. All of this, and he didn’t mention compliance or online adaptation to otherwise un-sensable dynamics. Or massively complex miniature mechanisms.
Current generation tactile sensors cost a couple thousand $ PER FINGER, and have a real world MTBF of hours. The cost can be solved with economy of scale. The fragility is harder.
Onavo 55 minutes ago [-]
> Current generation tactile sensors cost a couple thousand $ PER FINGER, and have a real world MTBF of hours. The cost can be solved with economy of scale. The fragility is harder.
Are we talking about experimental laboratory ones here? What happens when the Alibaba players start getting into the game? They have plenty of humanoid robots.
dyauspitr 4 minutes ago [-]
How much more sensitive do you need to be compared to say a standard mobile capacitive screen and those are essentially dirt cheap at this point.
UltraSane 2 hours ago [-]
I'm picturing humanoid robots having to operate in pairs so they can constantly fix each other.
kooi 2 hours ago [-]
Yes the problem is very hard. Mainly because high DOF generalization is very difficult.
We have self driving cars because what are the control inputs? Pedal, brake, steering wheel. This already took many many years.
Now for a humanoid robot: An action space that is metaphorically Hilbert. (Physically, yes, obviously)
Also, IMO, LLM's can aid the development of robots, but do little beyond a planning, human control interface. Below that it's the domain of control and the solution will be the correct combination of classical, neural, and real time optimization based control.
All the bad-ass biped robots that actually look natural? It's PID controls wrapped with control barrier functions constraining the QPs that are being solved in real time.
But that's annoying to derive per-application. So we'll need neural methods which can be learned (while being constrained by a priori knowledge of dynamics). My hunch is that the Yann LeCunn type of jepa models will be how tasks can be learned.
dyauspitr 2 minutes ago [-]
I don’t think the complexity scales with every additional degree of freedom like you are painting here. I think it’s just a matter of getting the right training data in sufficient quantities for an LLM to output across all degrees of freedom simultaneously without it being some exponential leap.
gugagore 52 minutes ago [-]
> All the bad-ass biped robots that actually look natural? It's PID controls wrapped with control barrier functions constraining the QPs that are being solved in real time.
That's not entirely true. Locomotion is well addressed by RL in sim. It's true that there is still a PD layer, and the RL policy produces setpoints for it.
mr_toad 54 minutes ago [-]
> So we'll need neural methods which can be learned
Data is a problem. LLMs had the advantage of the whole internet to train on. Robots don’t have that corpus of information. And real time learning seems to be something that everyone in AI is studiously ignoring.
robotresearcher 40 minutes ago [-]
The hope is that RL in simulation can fill the gap.
Also there’s imitating humans, via a suitable mapping from the human sensor, control and configuration space to the robot’s. Some groups have gathered video and other data from humans doing tasks, for example with a VR headset.
choonway 31 minutes ago [-]
I can say many things, but most people here will not believe me. I'd say just watch China do the 'impossible'. Then some self reflection should be in order.
akurilin 1 hours ago [-]
Couple of related reads/watches to this I found useful recently, with a similar conclusion:
people very soon as going to look back at all of us and just think "they didn't even have robots yet! how did they even eat?" Well, future readers, it isn't easy! We either go to the store and bring back groceries ourselves or pay a helper to do it. No robot in the loop there. Then when it comes time to cook something we either have to cut it up and otherwise prepare it and then cook it ourselves, or another human can do it, or we can buy fully prepared foods that don't require cooking. And then when we're done we have to put our own dishes away into a dishwasher. We can't just say a few words and have a robot do it for us!
robotresearcher 38 minutes ago [-]
> We either go to the store and bring back groceries ourselves or pay a helper to do it.
Or order it from Amazon, in which case there was likely a robot in the pipeline.
Robots are very widely deployed, but almost entirely invisibly to the customer yet.
Roomba is the main exception.
UltraSane 2 hours ago [-]
The physical world is basically an infinite amount of global state that must be perceived indirectly through imperfect sensors and acted on using imperfect motors and manipulators.
Ozzie-D 24 minutes ago [-]
[flagged]
Rendered at 02:56:02 GMT+0000 (Coordinated Universal Time) with Vercel.
Current generation tactile sensors cost a couple thousand $ PER FINGER, and have a real world MTBF of hours. The cost can be solved with economy of scale. The fragility is harder.
Are we talking about experimental laboratory ones here? What happens when the Alibaba players start getting into the game? They have plenty of humanoid robots.
We have self driving cars because what are the control inputs? Pedal, brake, steering wheel. This already took many many years.
Now for a humanoid robot: An action space that is metaphorically Hilbert. (Physically, yes, obviously)
Also, IMO, LLM's can aid the development of robots, but do little beyond a planning, human control interface. Below that it's the domain of control and the solution will be the correct combination of classical, neural, and real time optimization based control.
All the bad-ass biped robots that actually look natural? It's PID controls wrapped with control barrier functions constraining the QPs that are being solved in real time.
But that's annoying to derive per-application. So we'll need neural methods which can be learned (while being constrained by a priori knowledge of dynamics). My hunch is that the Yann LeCunn type of jepa models will be how tasks can be learned.
That's not entirely true. Locomotion is well addressed by RL in sim. It's true that there is still a PD layer, and the RL policy produces setpoints for it.
Data is a problem. LLMs had the advantage of the whole internet to train on. Robots don’t have that corpus of information. And real time learning seems to be something that everyone in AI is studiously ignoring.
Also there’s imitating humans, via a suitable mapping from the human sensor, control and configuration space to the robot’s. Some groups have gathered video and other data from humans doing tasks, for example with a VR headset.
https://www.youtube.com/watch?v=FUUzmRH5Yi4
https://www.construction-physics.com/p/robot-dexterity-still...
Or order it from Amazon, in which case there was likely a robot in the pipeline.
Robots are very widely deployed, but almost entirely invisibly to the customer yet.
Roomba is the main exception.