Humanoid robots: What Tesla is doing right

Professionally, I develop AI; as a hobby, I develop robots. What fascinates me about robotics: it teaches the software person in me humility. Sensors don’t measure accurately, energy supply is not stable. Motors are not don’t deliver the same power continously. Moreover, everything depends on the environment. But now to the pros:

Tesla has unveiled the humanoid Optimus

Last year, Elon Musk unveiled a humanoid as “one more thing” and the world shook its head: A dancing human in a black & white full-body suit. Does Tesla now have to make robots, now? Add-on adventure? Seariously?

It was obvious to me: he’s able to deliver. You can argue about the sense and nonsense of humanoids, but Tesla will be able to deliver. Why?

• Tesla has to deal with battery technology and power management for a long time, the engineering of a car goes through the whole chain at Tesla, which has to save power, not only the battery and charging logic itself. A useful humanoid must be optimized for power consumption from A to Z. Nobody will buy a robot that runs for an hour.
• Self-driving cars need a high level of social intelligence: at intersections in cities, they need to predict not only dozens of cars at once, but the pedestrians, dogs, and cyclists as well (the occupancy network). This means that there must be predictive models for humans that can be used elsewhere.
• The machine learning chain Tesla is developing is second to none. Tesla collects massive amounts of data, has to auto-label it (i.e. make it learnable for computers), builds its own hardware infrastructure for the data streams, optimizes machine learning infrastructure, has knowledge of all the current “building blocks” such as Transformers but also NeRFs and playfully assembles them into new architectures. Last but not least, Dojo is used to develop a chip and server infrastructure based on the software insights. This infrastructure and the data from it can of course also be used in other ways. The additional costs for this are minimal.

One year later: there he is.

These were the three things that were obvious. But what blew me away Friday night (AI Days live broadcast) is that in that one year, Tesla has gone at an iteration speed in design that I didn’t expect.

Normally, you’d build a robot from off-the-shelf parts and start tuning the software and have a robot that can run after half a year. A completely different issue is hands, which are certainly one of the most complex elements because of the need to move many parts in a small space and also to have a good feeling about what to grasp (weight, extension, hardness). Very few robot builders also solve the issue of on-board computing power and energy management at the same time. For example, the parcours running ATLAS from Boston Dynamics still needs a whole room full of GPUs to be able to move like that.

So does Tesla do differently?

• Manufacturing depth. Musk always means it from A-Z, be it rockets, cars or humanoids. Everything is always completely thought through. Actuators, sensors, mechanics, materials, weight distribution. Where necessary, we do our own development.
• Machine learning everywhere. Mechanics can be optimized by machine learning, computer vision / perception, motion planning, everything. But also just optimization / iteration by consistently recording data. I found it particularly fascinating that within a year they had built all the actuators (“motor joints”), measured them, and overlaid their data so they could reduce them to a few.
• Buildability. How easy can I build the system, how can I reduce the number of parts, how can I easily replace parts later? Lack of investment in this aspect has cost NAO developers Aldebaran its life several times.
• Purpose. We don’t just build any humanoid that can do parcours and Viennese waltz. The solution to a problem underlies the design. Musk clearly wants to use the humanoid on the factory floor first. That reduces the design complexity to solving that problem. And increases intellectual reach because innovation energy can be applied in a top-down focused way.

Like Jeff Bezos with AWS and his own delivery services to support the core business, Musk can leverage the synergy of his investments. This differentiates him massively from Boston Dynamics, for example.

Tesla as a carmaker is not alone in humanoid robot building, Hyundai bought Boston Dynamics, years ago Honda already introduced the Asimo. Tesla, however, started with very different assets and has a clear internal development goal.

I cannot yet forsee the economic impact of a functioning humanoid priced at 20k$. But the speed of development must force us to think about it. Better today than tomorrow.