Humanoid Robots Are Still a Body Problem with Jerry Pratt (Persona AI)

Añadido: 2026-05-02

[00:00:00]Yeah, yeah, I know cooling's cool.

[00:00:01]Cooling's a huge huge deal. Um and um you know, your kind of choices are either using a fan to do some sort of forced air cooling or have some big fins that are open to the the world to to do a lot of cooling or start thinking about things like water cooling. Not too many robots that are using water cooling, you know, it just adds a lot more complexity.

[00:00:26]But it's it's a nice solution if you can do it and it really doesn't take too much extra stuff or weight.

[00:00:34]Um but it does, you know, just more things to break and so you don't see it too often.

[00:00:39]And then the other thing you want to do, of course, is reduce the times where you're using huge torques to reduce the amount of um heat you generate.

[00:00:48]So you want to walk with straight legs rather than bent knees, which is pretty much the big thing. And in in general in general, you don't want any joint to be producing a torque which is there just fight gravity if you can avoid it.

[00:01:02]I think my hunch and my desire is that you know, when as things you know, if you think about 3 to 5 years from now um it would be great to see more tendon driven hands that work well and that and that somehow we as a community or us as a company individually figure out how to make um really good tendons that don't break.

[00:01:26]I would say why it's so hard for us to make hands is cuz our stuff doesn't heal.

[00:01:31]Uh if it did, then we could um then the tendon problem wouldn't be an issue at all. We'd we'd have really amazing tendon driven actuators. I hope that somebody will come up with a really good artificial muscle and and there's a few companies I've been chatting with that that are working on it.

[00:01:45]If I were a DARPA program manager, that is the thing I would fund, I think.

[00:01:50]Um is is artificial muscle or one of the things I'd be interested in.

[00:01:53]Um so hopefully somebody can crack that nut. If they can, then you can make an incredible robot.

[00:02:00]If if things are too um it really comes down to like the natural frequency or the delay in getting your center of pressure from one side of the foot to the other versus the closed loop bandwidth that it's trying to do that at. Right? So if it if you've got too much if it's too spongy and the robot's trying to move its center of pressure from one side of the foot to the other instantaneously, you'll start chattering.

[00:02:24]So it's actually easier to walk on clean rocks than it is to walk on like just a 30° sheer surface. You know, you slip on that. Um I love them all and I hate them all.

[00:02:36]Uh let's see here. So planetaries are awesome because they're just really simple, low cost, don't have much um reflect uh much friction.

[00:02:48]Um and uh quasi-direct drive with low gear ratio, like say up to 20 to one planetaries are just work really well and you see just robots working really well with it. Um one of the things I've been trying to determine for the last 30 years for myself is like at what ratio at what gear ratio and the reflected inertia and stuff can does quasi-direct drive break down where you're just, you know, going from motor current to torque. Um you know, and how backdrivable does an actuator have to be.

[00:03:20]You know, in in my early days and you know, the MIT Leg Lab back in um around like 1994 or so, um so Gill Pratt had come up with the idea for series elastic actuators which was amazing for getting just really good resolution force control.

[00:03:36]Well, our Gen 2 robot just took its first steps about an hour ago.

[00:03:43]Um that's just the upper body. We're putting the putting the or sorry, just the lower body. We're putting the upper body on it um in the next uh week or two.

[00:03:51]Okay.

[00:03:52]Hello, Jerry.

[00:03:54]Hey, Marwa. How you doing? I'm doing well. I'm so excited to talk to you today.

[00:03:59]So same here. Yeah. So I think before going maybe more details about Boston AI what you're doing, I think maybe you can introduce yourself and I want to ask you the first question. Why humanoid feet are flat? I know you have been asked this many times.

[00:04:15]>> [laughter] >> You have Why?

[00:04:18]Okay, let me jump in on that one. Why humanoid feet are flat? Um because every single humanoid robot I've been involved in, at least for me um we ask ourselves that question when it's time to design a robot and we say, "Well, not sure on the first version. Let's just use a flat metal plate and then we'll figure out something better later."

[00:04:39]And later never comes.

[00:04:41]Um or we like try something later and ah, it's not so good. And but I mean there there are you do see some feet that aren't completely flat. You know, you you do see some that are kind of flat with a little lip at the front so that you can go on your toes.

[00:04:56]Um the reason you don't see actuated toes is just the additional weight and complexity that you'd have to have down at your ankle. And the more weight you have more distally, the harder it is, you know, with a human foot and human hands, you know, all your weight's kind of back with tendons and cables and stuff so you can have light light feet and light hands.

[00:05:15]Um now there have been a few robots here and there that do have actuated toes.

[00:05:20]Um and I think you'll start seeing more and more of them. It's hard to say whether or not you really need it for most applications.

[00:05:28]Um but it's it's really comes down to it's just a it's a hard design challenge to get degrees of freedom far out at the end. May I ask you, do you think if we have a good foot design, that will reduce the torque and if affect the battery life? Do you think it would be correlated Um if you if you did have a good foot design with those extra degrees of freedom, you could um walk more efficiently. There have been bio biology studies that show that humans walk more efficiently than say ostriches or emus um or other birds because we can make our um essentially get a longer foot by going on to the toes and by making it kind of passively rigid so we don't have to apply torque to hold um that shape. Um so I do I do think you can get more efficiency. Plus you can like take a longer stride while keeping your foot uh straight and your leg straight. So yeah, you you get speed and efficiency advantages from that. Awesome. So maybe you can introduce yourself. I think I think most people in humanoid space know what I have been doing in the last 25 years. But if you can introduce yourself with people first time listening to you.

[00:06:36]Okay. Well, all right, Marwa. My name's Jerry Pratt. I've been working on different legged robots and robots in general for about 30 years. Um got me started at the MIT Leg Laboratory in grad school where I was working on a couple different robots uh one called Spring Turkey, one called Spring Flamingo. Those were pretty simple, you know, four degree of freedom, six degree of freedom robots that walked uh you know, in on a surface of a sphere, walked you know, had a boom so they didn't have to worry about side to side balance.

[00:07:05]Um back then, what we were trying to do mostly is figure out see what is walking, what are um like heuristic strategies you can use, how can you understand walking better from like a biomechanics point of view.

[00:07:19]Um computation back then, you know, this is 1994 to 2000. Um Yeah, the you didn't even have enough computer power to invert say a 6 by 6 matrix. So you things like inverse dynamics were kind of out of the question back then.

[00:07:35]So we came up with very simple heuristics, um some kind of clever control techniques and showed how you could use force controllable actuators to um to get really natural looking walking without having to have complex control techniques. Um particularly by exploiting a lot of the natural dynamics of of the robot.

[00:07:56]So then I got my PhD um started a small spin-off company called Yobotics uh that we had you know, did a bunch of small business innovative research grants, um couple one-off robots, a robot arm, exoskeleton, you know, one of the world's first uh really simple exoskeleton called the Roboneer. Um just one degree of freedom in the exoskeleton.

[00:08:19]Um a lot of consulting. Um but you know, we're we didn't really have business acumen back then um and you know, we're kind of just doing it for the fun and and you know, hoping that something would come of it. Um and we kind of all started going our separate ways after a while. I I went to the um Institute for Human and Machine Cognition, IHMC, which is here in Pensacola, Florida where I live.

[00:08:46]Um started robotics labs around 2002, ran that for about 20 years. Uh grew it to about you know, about 40 people or so at our peak. Um We're in a lot of different uh DARPA sponsored projects. Um the DARPA Robotics Challenge is a big one. Um Learning Locomotion project where we worked on quadrupedal walking over off terrain.

[00:09:09]Um a lot of uh exoskeletons for um paralyzed individuals.

[00:09:15]Um quadcopters, you know, a whole whole range of things uh over the years.

[00:09:19]Um uh in the DARPA Robotics Challenge, we ended up getting second place overall using a Boston Dynamics Atlas robot, their their older version, but writing all the software for it. So we We worked a lot Most Most of our focus uh during those years was on bipedal walking control. Um balance, looking at different control techniques like model predictive control, whole body inverse dynamics.

[00:09:42]Um different ways of doing force controllable actuators.

[00:09:46]Things like that.

[00:09:47]Uh let's see. Around 2022, I uh joined uh Figure AI. Um was the CTO there for a couple years.

[00:09:55]Um helped build the team, get everything, you know, going in the right direction.

[00:10:00]Um, but uh couldn't move out to California really. My my wife runs a science museum here in Pensacola, Florida. And um and, you know, after after Figure Eight grew to around 100 people, um, it was, you know, it's kind of hard to be a remote CTO. Um, and so we we had a mutual parting of the ways, if you will.

[00:10:21]Um, and then um I had been working with on on NASA for quite a long time and and was uh really engaged with the different people and groups there working on Valkyrie and Robonaut and um uh Nick Radford was good friend of mine and we had been talking about doing humanoid robots company for for a long, long time.

[00:10:41]And we found both of our souls found ourselves available and so we decided to start Persona. Uh quickly brought on uh Giday Yakabiti and um as a third co-founder and grew the team and and [snorts] um things are going well and can tell you more about that. Yeah, I'm very excited about this part because I think for Persona AI, you're focusing on the yeah, industrial use cases. And I think when we talk about the space, you have seen it a lot. So, home use cases or industrial use cases and it seems you have a different, I think, perspective on that.

[00:11:14]Yeah, and um I, you know, I I feel good about both home and industrial. Home is a little trickier. It's, you know, it's more price sensitive. There's going to be a lot more competition and it's and it's in general a harder problem, I think.

[00:11:29]Um, so we're focused on industrial use cases. Um, things things where there's large labor shortages, in areas where there's, you know, significant labor costs, um where, you know, customers are looking for automating it using humanoid robots.

[00:11:47]Um, and there's a lot of lot of really uh good at applications where you could have hundreds of robots in a single location.

[00:11:55]Um, with um um couple shifts a day, you know, just generating tons of value for the customer. And on on use cases that are really hard but possible. And we, you know, we're pretty confident that the things we're looking at we'll be able to do. It's going to take a lot of work, a lot of, you know, a lot of engineering and a lot of um grit and determination, but I think we can um uh get get our use cases working, particularly things like welding. Mhm.

[00:12:26]I'll let you speak about that, but I think one of the things that I hear a lot, the durability. If you have 24/7, you have to do the task, like the hand, how it do the task for 24/7?

[00:12:38]Do you think what's the biggest challenges for humanoid now?

[00:12:43]Yeah, no, that's definitely um reliability and durability. Um you know, we there's a lot of tasks that that people have shown that you can do about 80% reliably without too much effort. It's, you know, getting to those nines of reliability, chasing the nines, you know, and um the the ways that people tend to control robots now with machine learning techniques, behavior cloning, what have you. Um like I said, it's it's really amazing, quick way to get to showing feasibility really quickly.

[00:13:21]But then you do have to do a lot of extra work um to get super high reliability.

[00:13:27]And so that's that'll be a big issue.

[00:13:30]Um, and then durability and and just how how reliable your hardware is. You know, the the thing that keeps me up at night is fingers breaking on robots. I think that's going to be um the big maintenance issue of the entire industry is um being able to quickly and cheaply fix hardware by swapping things out or or maintaining it on site. But um developing hands that have high degrees of freedom uh that don't break constantly is going to be the biggest challenge, I think. Yeah, I think this is a good point. I I think some of the maybe the the recent owners of the robots they they reported like uh the breaking, they break a lot. And when you mention the swappable hardware, how do you envision this as a part of the business? Like it's like Mhm. open sourcing the hardware so you can really rent and do it? How do you envision the breaking parts constantly and you have to replace it? Yeah, well well, for us we're interested in doing, you know, just end-to-end um robots as a service labor.

[00:14:33]So, we'll you know, have maintenance kits or whatever, probably initially have our own maintenance people, um have our own, you know, team that works with the customers really closely. Uh you know, works with the customers developing new applications on site. And um and for the most part, robots as a service, ideally, or in some cases, you know, selling the robot and working with their development teams to um to develop further applications.

[00:15:01]So, as far as maintenance go, we'd probably have, you know, spare parts kits and and um trained technicians who can swap out components and then, you know, ship the broken ones back and and fix them back at the factory. Mhm. So, I'll let you show the video and then I'll ask the question, but I I I would love to hear what you want to share, if if you'd like.

[00:15:21]>> Okay, yeah, yeah. I got I got a few videos I can show. Let me um All right, so this this is a video of our our welding application. Uh this is um a simulation that the robots will be um physically dynamic here doing control and everything is simulated. Um the rest is kind of just animation, including like the sparks from the welding.

[00:15:45]So, this is kind of that the concept of being in a shipyard and doing some of these um long linear welds that they have where they're, you know, the early stages of um a ship coming together.

[00:15:58]These the um various shipyards are are incredible.

[00:16:04]I mean, they're just they're amazing to to visit. Um, I mean, just imagine like hangars that are size of football field with people everywhere, pieces of metal everywhere, and just, you know, when when you make a ship, there's tens of kilometers of welding that has to be done. And it's um and it's all done at about 1 cm/s. So, just tons of time to to weld a ship. This is uh kind of showing how we train that was um through um um oops, sorry.

[00:16:39]Using uh deep reinforcement learning uh and using some of the NVIDIA tools like Isaac Lab and then doing a motion mimic where you capture a person doing the motion and um use that to train to train the reinforcement learning algorithm.

[00:16:55]Let me show you uh where we are currently with um with the robot.

[00:17:02]So, this is our ver our generation one robot.

[00:17:05]Uh this this video is a few months old now. I'm walking over um 2x4s on the ground without sensing the 2x4s. So, being pretty pretty robust to It's it's really incredible just the walking algorithms today trained using reinforcement learning how robust to tripping and disturbances they are.

[00:17:25]Yeah, I've been working like I said, I've been working on this stuff for about 30 years and now you can train in a couple hours in just a black box reinforcement learning algorithm, something that's better than anything we developed over the last 30 years and it's is a little frustrating thinking about it that way, but um but it's but it's but I'm happy that I had a that they do work that well cuz, you know, I I'm I'm done with these problems being hard. I want them to be easy. Yeah. I I'm curious also because you have seen a lot of these designs, like even yeah, before all these humanoid companies in the space. So, what what is your philosophy of design in in Persona?

[00:18:03]>> Yeah, so with with iteration one or our philosophy was entirely design and build it as fast as we humanly can. So, we did like pretty straightforward, just pin joints everywhere. Um let's see here in uh in the ankle, you'll see uh two mechanism, you know, a double bar mechanism, which is kind of common. Uh we got a three degree of freedom pelvis.

[00:18:25]Um um pretty pretty straightforward, you know, standard design here. You got six degree of freedom legs, seven degree of Well, we will have seven degree of freedom arms. There's no wrist joints on this one yet.

[00:18:38]Um couple degrees of freedom in the head. Um so So, pretty straightforward, just about all pin joints, nothing parallel except for the the ankle mechanism there.

[00:18:51]And uh for the hand I'm curious about the hand.

[00:18:54]Uh that was uh off-the-shelf hand. Um right now we're evaluating different off-the-shelf hands, low degree of freedom hands. And then um uh we're designing a couple uh hands internally. One that's like the next generation of the Robonaut 2 hand from NASA.

[00:19:11]Um we've got a a design built built up and we're on doing some control on it now. And then we're looking at an an another design um with high degree of freedom, too, that we just started on.

[00:19:23]So, looking at, you know, kind of different parallel efforts. There will be for some applications, you won't need a large degree of freedom hand, but I think for like the long tail of applications, each new one is going to be kind of um opened up um by having good hands and good hand control. I was curious about because I don't know how much you want to share about the the internal development. May I don't know if you can share a little bit inside of it, overall I think the there's two ways of the hand as as far as we can see that the tendons in the market are back drivable and I'm not sure what's your take about like the bones for different design of Yeah, I mean there's there's a handful so to speak of design choices when you're designing a hand. One is whether or not you have intrinsic or extrinsic actuation, you know, are the are the motors all in the hand or are they in the forearm with tendons?

[00:20:18]Um of course you can get more degrees of freedom by having uh actuators extrinsic with tendon drives or some other drive.

[00:20:26]Um and uh but it's typi- typically more complex and one of the biggest challenges is just tendons breaking. Um probably that's the biggest challenge.

[00:20:37]Um if you if you go with tendons, uh intrinsic lot simpler design, but you have less space for the motors. So each motor is going to be smaller. Um you're going to have more heating issues because that be- being smaller they'll overheat more. Um and then just the weight of the hand uh increases so it just takes more weight at the end of the arm which is which is always problematic. And then you know, being able to move your fingers really quickly uh just having more inertia in in your different um appendages will slow you down a bit. So there there's trade-offs um my I think my hunch and my desire is that you know, when as things you know, if you think about 3 to 5 years from now um it would be great to see more tendon driven hands that work well and that and that somehow we as a community or us as a company individually figure out how to make um really good tendons that don't break.

[00:21:38]In nature you have the advantage that you get to heal you know, constantly or every night.

[00:21:43]If that wasn't the case everybody would have carpal tunnel syndrome in weeks.

[00:21:47]And um your hands would just be useless.

[00:21:49]Also you know, you you tend to sprain a finger now and then, right?

[00:21:53]>> [snorts] >> If that was a robot you'd have to replace your finger. It's not going to heal. Um but with nature you just heal overnight. We don't have that in engineering yet so that's really the big difference is why it's so I would say why it's so hard for us to make hands is cuz our stuff doesn't heal. Uh if it did then we could um then the tendon problem wouldn't be an issue at all. We'd we'd have really amazing tendon driven actuators. So this is very interesting.

[00:22:18]Do you think it's about like artificial muscle in that case if we speak about the Well, I I mean if if you can make an artificial muscle that would be amazing.

[00:22:26]I've been watching artificial muscle development for 30 years waiting for it to happen and and there's a lot of really amazing things that have come down the line.

[00:22:38]But if you think of like the the different design parameters or requirements for muscle or the different features, there's really about 20 different cool aspects of it and a lot of these artificial muscles will like blow away natural muscle on like one or two or three criterion like you know, maybe power to weight or you know, then some density or something. Um but you know, one issue is often you can't scale them up to the size of uh you know, a a quadricep or something or or that they require cooling. Um that just makes it take forever for them to you know, to to cool each time.

[00:23:18]Um or um or they break cuz of small bending radiuses or you know, there there's always some sort of fatal flaw that makes them not not useful. I hope that somebody will come up with a really good artificial muscle and and there's a few companies I've been chatting with that that are working on it.

[00:23:36]If I were a DARPA program manager that is the thing I would fund I think.

[00:23:40]Um is is artificial muscle or one of the things I'd be interested in.

[00:23:44]Um so hopefully somebody can crack that nut. If they can then you can make an incredible robot.

[00:23:50]If you if you think about having an actuator like muscle that does you know, really good linear force um and that has you know, just the right speed and force just like muscle does and that where you can have multiple ones slide on top of each other without wearing out or healing.

[00:24:07]Uh then what you can do is start making things like ball and socket joints.

[00:24:11]Yeah, we can make a ball and socket joint physically. The joint's no problem. The difficulty is actuating it.

[00:24:16]If you look at like your shoulder, your hips, you've got you know, take take your shoulder where you've got in this small amount of volume you've got about 30 different muscles running on top of each other going over that ball and socket joint every different which way.

[00:24:29]So that no matter where you are in your range of motion you don't lose lever arm.

[00:24:36]You know, if if you compare that to a single degree of freedom pin joint that has a linear actuator. As soon as that gets straight you know I've lost your lever arm because you know, the the actuator gets near the the joint so you have no no lever arm with it.

[00:24:52]Um so you got to do like different tricks like maybe you do a four bar mechanism or or a or a pulley with a tendon or a cable or something.

[00:25:02]Um with a shoulder you do run out of lever arm for individual muscles, but as soon as you do another one now gets to position where it has good lever arm to bring you back. And by having redundant actuators like that you never lose range of motion. There's no singularity.

[00:25:19]All your degrees of freedom are centered about one point. Um which which is really nice kinematically. And then in addition you can you know, float to add two more degrees of freedom. So your shoulder really has five degrees of freedom while taking up about that much volume of muscle.

[00:25:35]You know, in contrast when making a robot that's how big one single actuator takes up, right? So so just if you if you could do um artificial muscle well you'd be able to make an incredible robot.

[00:25:50]And I think I I will come to that question in a second, but I think one thing is also stressed about the sensing. Like for example, the welding use case that showed me to me now. Yep.

[00:25:59]Hand manipulation. I'm not sure how much complexity or simplicity is needed in the design that you have to work the task. Like the sensing in the hand. I'm not sure how it's relevant your view about the sensing in the hand if you have to incorporate this in the hand as well. Especially in the welding use case.

[00:26:15]Yeah, well um if the only thing we ever wanted to do was weld we could probably do it in a much simpler way.

[00:26:22]I mean heck you could probably just make the arm be a welding torch.

[00:26:26]Um but what we want to really be able to do is um deploy robots in the field doing things exact same way or as close as possible to how humans do them.

[00:26:37]And so for welding and partially the reason to do that is um so that you can have a small number of robots initially work just augmenting the labor force because they they have these labor shortages. They're they're not interested in like completely changing everything over.

[00:26:54]What they need is 10% more workers so that they can build 10% more ships or whatever you know, 10% more gadgets. And then then a lot of these areas where you have labor shortages the hope would be that we can just augment the workforce and then as people retire and they're not replaced we can scale that way.

[00:27:13]Um in order to do so well we want to be able to use the same tools, the same techniques, kind of do the same process same as how the humans do it.

[00:27:22]So for that it's it's all about tool usage. Can we make a a general a hand that can kind of use 100 different tools?

[00:27:31]And that's that's what's really motivating having a high degree of freedom hand.

[00:27:35]Um and then again applications down the road once you start doing kind of general assembly on a on a car line or kidding operations or whatever it happens to be more more um fine manipulation type assembly tasks you're going to need a a better and better hand. So for the welding it's all about being able to grab that welding torch, pick up pick up your spool of wire walk, start welding, put the welding torch back in some holster or whatever and and then pick up the tools that do the the scraping afterwards, some painting and you know, pretty much just do all the things that the human does. Um uh which would require having having a good hand. Yeah, and the sensing part I think that's the part the sensing embedded sensing also.

[00:28:21]>> Yeah, so for embedded sensors that that's uh one of the enabling technologies that's coming along that I have a lot of hope in. Uh there's you know, probably a dozen or two dozen companies right now working on um advanced tactile sensors.

[00:28:35]Uh there the the most difficult part with those is packaging them into the hand design.

[00:28:42]You know, the sensors themselves are incredible now and if if you're using them for applications where you where you have more space like say um you know, virtual MIDI boards or keyboards or musical instruments and stuff like that. There's some companies that develop uh tactile sensors for those applications too.

[00:29:02]Um and they work great.

[00:29:05]The difficulty in a hand is you know, getting you know, we we've got about 10,000 um sensors per square inch on your on your fingers give or take. Um you know, just getting that density and then getting all the uh wiring routing over all these moving joints without breaking. That's that's the that's the big challenge. So it comes down to a packaging of electronics and packaging of the sensors. Yeah, and for Gen 1 I'm not sure when Gen 2 will be released.

[00:29:36]I'm not sure if there there's something >> our our Gen 2 robot just took its first steps about an hour ago.

[00:29:43]Um that's just the upper body we're putting the putting the or sorry, just lower body we're putting the upper body on it um in the next week or two.

[00:29:51]Um so we'll we'll we'll see when we're when we're confident to put videos out on that guy.

[00:29:57]Yeah. What I'm also the design choices like the shoulder.

[00:30:00]I'm not sure if you can tell more about the Gen 1 and maybe the thing is that me being general compared to Gen 1 and all around in this strain humanoid like you think the design it just could be more different like the shoulder the heater and the and heater angle whatever this choices or the waist.

[00:30:18]What do you think is just maybe different or yeah.

[00:30:22]Yeah, um we're we're still looking at pretty standard designs. You know, there there's been a lot of humanoid robots over the last 30 years and so there's only you know, so many combinations of joints you can use and and as you know, you all have been shown on your podcast um you know, people kind of converge to certain um ordering of joints if you're doing a a sequence of um pin joints.

[00:30:45]And and there's a lot of good reason for it and and it usually comes down to just where are your singularities? You know, what um you know, if you take like your hip um the two types you typically see is you know, and if you're talking about yaw pitch roll like an airplane um you typically see uh let's see here yaw pitch roll or um see here or roll pitch or sorry, pitch roll yaw.

[00:31:18]Well, you know, you you know what I'm talking about but the but the reason is um you you don't want to have it where a common motion will have two of the axes line up.

[00:31:31]Right? So if you have one where it's um uh pitch first and then roll and then yaw.

[00:31:39]Um your axes line up when your second joint is at 90°. It's a if you got three joints it's always when when is your second joint at 90°? So that would be your roll out 90°. Your leg all the way out to the side.

[00:31:53]It's not a very common thing you do, right? So so that's a good reason to put roll in the middle.

[00:31:58]Um whereas if you put yaw in the middle say it then as soon as you rotate 90° um then your then your pitch and your roll joints would line up and you'd lose a degree of freedom and you know, at a singularity. So that's usually the the reason why you'd pick one over the other and then same with like a lot of different designs will have them angled a little bit. It's pretty much just to center your range of motion where you want your work space to be or it might have to be with like you know, making sure you're not colliding students self collisions and things like that. You know, if you think about like when a human can only cross over to the inside a little bit but you can step way out. So you might put your hips that way a little bit like 20 or 30° or something. So that's that's usually the consideration that it comes down to.

[00:32:46]Things like that. Now like I said before if you can do ball and socket joints all those problems go away.

[00:32:53]But it's really hard to do ball and socket joints. So so a lot of you know, in the different robots I've worked on over the years we've had different mechanisms like the Nadia robot at IHMC had a a dual four bar hip mechanism so that you get a lot of pitch motion and not as much roll motion but you don't need as much roll motion and using you know, two linear actuators with two four bar mechanisms. That was pretty cool mechanism but it just adds a lot of complexity.

[00:33:23]You got like 12 extra bearings to to get that. Anytime you're doing parallel mechanisms there'll be bearings and then different cross members that if that if you're in a kind of a bad position you might put huge stresses on those linkages. So it really can limit the range of motion.

[00:33:43]Just concatenate pin joints are nice because they're only limited by how far your actuators can rotate or how far before your pieces of metal rotate. And if you look at like the newest Boston Dynamics Atlas where they're they got continuous rotation and they offset their links like that and like that they can have just about you know, a lot of the degrees of freedom have infinite motion which allows them to do a lot of really cool things. Yeah. There's many questions I think the first question I want to ask you about the heat. Most of things that robot feels that like one of the robot like Unit 2 for example like it's a lot of heat and and break down and I think it's all about the actuators. So I'm not sure you think the heat issue how you think about >> The cooling how how to what the cooling issue?

[00:34:28]Yeah, yeah. No, cooling's cool.

[00:34:29]Cooling's a huge huge deal.

[00:34:32]Um and um you know, you kind of choices are either using a fan to do some sort of forced air cooling or have some big fins that are open to the the world to to do a lot of cooling or start thinking about things like water cooling. Not too many robots that are using water cooling. You know, it just adds a lot more complexity.

[00:34:54]But it's it's a nice solution if you can do it and it really doesn't take too much extra stuff or weight.

[00:35:01]Um but it does you know, just more things to break and so you don't see it too often.

[00:35:06]Um ideally you just have you know, good good surface cooling and no fins, no water, no anything.

[00:35:15]Um and and then the other thing you want to do of course do is reduce the times where you're using huge torques to reduce the amount of of heat you generate.

[00:35:26]So you want to walk with straight legs rather than bent knees which is pretty much the big thing. And in general in general you don't want any joint to be producing a torque which is there just to fight gravity if you can avoid it.

[00:35:41]Um now it's unavoidable on some joints but um but like the knee for instance you you walking with straight legs versus bent legs is the difference in overheating your motors and not overheating them.

[00:35:54]Um and then just having really good design where you have um natural heat dissipation to all of your surfaces. Yeah, and that's which which is um a little bit frustrating cuz that kind of then rules out a large use of carbon fiber cuz it's a really poor heat conductor but you know, aluminum's great heat conductor.

[00:36:15]So it's if you can get the heat from your coils to the shell to to the shell the motor to the shell of the robot and then have good surface area on the shell of the robot then then it's it's a solvable problem. Yeah. And then I will ask you about the like the design objective especially for the home for example the noise and all the thing.

[00:36:37]I'm not sure you think about this with your team like we have to optimize the design for that goal like the heat, the noise or it's I'm not sure how this relevant in the design consideration.

[00:36:49]Yeah, um I'm actually surprised how quiet robots are these days. You know, they are well, ours tends to stomp a bit but that's in the walking algorithm and and we'll we'll get that to go away. Um but just like the operation you know, if if arms are moving around really quick you hardly hardly hear them.

[00:37:06]Um so you know, just actuators have they're surprisingly quiet. Uh you know, and we're not too again, we're not looking at home use right now. So we're looking at industrial use cases where you have ear plugs cuz it's so darn loud. So so noise isn't an issue for us.

[00:37:22]Um uh but the heat is again and um so Yeah, cuz some of the industrial use cases like you think about like a steel factory it's pretty hot hot in there.

[00:37:33]Yeah, yeah. And before moving to the actuator because I wanted to take your opinion about the current actuator and also the reducer. For the hand do do you still think we need the five hand I know maybe we can ask this question many times but you think three fingers enough for industrial Um yeah, for a lot of applications. I mean you you look at some of um the results people are getting with one one finger you know, just just one degree of freedom pinchers incredible like you know, folding t-shirts um doing some kidding operations, putting Lego together. There's a lot of things you can do with with just one degree of freedom hands.

[00:38:14]Um but there's a lot of things you can do with five degree of freedom or five five fingers with you know, 10 or more degrees of freedom that that you can't.

[00:38:24]Um you know, you think about okay, well, you see these you know, people folding t-shirts with one degree of freedom pinchers they're on a flat table.

[00:38:32]Usually when I fold a t-shirt it's in the air, right? And now I'm using my chin and everything else and it would be interesting to kind of see okay, how many fingers are humans actually using to to do that. If you think about you know, there's a lot of manipulation things where you're both holding something and manipulating it.

[00:38:49]Like sometimes you might hold like a water bottle and screw the the top the top off. You probably need at least three fingers to do that but even three would be tough to to hold it maybe four. So you know, it's it's really hard to say.

[00:39:03]Um one of the reasons though to have things to be as human like as possible is that one of the best sources of data from neural networks right now is recording humans. And so if you've got if you can set up system where you can record what the human's doing as they're doing fine manipulation tasks um then if your hands can be as similar to the humans as possible so that doing the translation from one to the other is is possible.

[00:39:34]Um then I think that's one of the major considerations.

[00:39:38]And then also and then also in teleop you know, if you're going to teleoperate a robot, um and you're used to doing the task using all five of your fingers, it's going to be more natural for the teleoperator to do the task. Yeah. And I'm just curious about um yeah, that's a little original question I think that was interesting, but um there seems, maybe correct me, there's two ways to achieve the the intelligence, either through the the brain side or the hardware side. And it seems some of the approaches that as you mentioned, using similar hardware to achieve the task.

[00:40:10]Where do you see the conversion or or where do you see the missing piece?

[00:40:13]Because are we having, I think, uh an a great hardware so far? And we just need to focus on the in the in the brain side? Or where is the issue? It seems there's two divisions here, like Yeah, no, um both are important and you know, over the years you'll see kind of a flip-flop between what is the current limiting factor, hardware or software, hardware or software.

[00:40:36]Um I'd say right now it's it's hard to say where it is right now and that and and it depends, you know, between locomotion, manipulation, and and everything else.

[00:40:49]But I think there's improvements we can do with both hardware and the software.

[00:40:53]I mean, that, you know, machine learning is just going to keep getting better and better and better. Um uh computer vision's going to get better.

[00:41:00]Um just all that stuff is just it's just going to ride Moore's law and scaling laws and everything and and just give it time and all that stuff will be incredible.

[00:41:11]Um hardware moves much more slowly.

[00:41:15]And so even if we're ahead at, you know, even if you can say, "Well, the hardware is fine, you got to let the software catch up."

[00:41:21]Um it's probably going to be hardware that's going to be the limiting factor.

[00:41:24]And and with everything it turns out that material science becomes the limiting factor on everything eventually, right? We're not there Well, I we're we're kind of there if you feel if you think about like just what you can get out of a brushless DC motor.

[00:41:37]They're so optimized, you're not going to get much more out of the best out of the best designs, and it's come down to simply the the resistance of copper, the magnetic fields you can produce with rare earth magnets, and then design-wise how like close you can, you know, how small of an air gap you can get.

[00:41:55]Otherwise, there's not much more you can do there. So that has become a material science problem.

[00:42:01]Um you know, with tendons, it's a material science problem. Can we make a tendon that's well lubricated, that doesn't wear, that we can get, you know, millions of cycles out of it before it breaks. Um but there's also a lot to be done simply on straight-up design. You know, just better and better, more optimized designs. Um reducing the weight of a robot is hugely important. Um cuz, you know, every every gram you're carrying around is another gram of weight you got to fight with, you know, power and everything. So um a lot of these really high-performance robots that you see are are smaller and lighter.

[00:42:42]Um but because of that, they're sacrificing battery weight, cooling, um and arm strength. And so typically if you're seeing, you know, some of these kind of mid-sized robots like the Unitree G1s doing these amazing kicks and backflips and stuff, um their their design is optimized for that.

[00:43:00]Um if you're making like an industrial strength robot that can survive the elements, that has padding, that has upper body strength comparable to a human, that's taller, um uh then it's it's kind of much harder to get those power-to-weight ratios.

[00:43:16]Um so designing out as much weight as you can is is critical. So I think there's a lot of work to be done there.

[00:43:24]Um Yeah, but between hardware and software, um we're going to we we need to make advances in both.

[00:43:33]But I think we're at a point now where some of the applications that people are looking at, even if you froze technology where it is as we stand today, you could achieve a lot of these applications and and dare say I think you could get to a profitable humanoid robot company without inventing anything new.

[00:43:51]That's my hunch. I think with like um you know, some of the applications you see in warehousing, automotive plants, um heavy industry, um I think we um you know, we'll keep we'll keep pushing the envelope on the technology, but I think with what we have today, we can actually do the job, whereas maybe five five or 10 years ago we couldn't.

[00:44:15]And uh um you know, there's been there's been a handful of different enabling technologies over the years. The most recent ones have been computer vision is a big one.

[00:44:26]Um you know, there's there's um if you think about you know, we're we're kind of in this new stage of the industrial revolution, and which is enabled by computer vision.

[00:44:42]Um you know, people have been trying to do robotics in factories, you know, first ones were like in the '60s, and keep, you know, getting check mark, check mark, check mark on the next applications. And um you know, and even starting to use some simple computer vision where like you got a conveyor belt of stuff that you got to pick and you know, see them and pick them just straight.

[00:45:02]But for things like where where uh something arrives at a station completely jumbled, and you got to as a human you got to go in there, pick one, you know, singulate it, pick a single one up, orient it and like put it in another machine and push the button.

[00:45:19]Just that being able to see what you're doing, you know, robot arms if they knew exactly where the parts are are capable of doing that sort of thing. But just being able to see where the parts are, um and where the, you know, they might fall down a little bit or so you need like real-time computer vision.

[00:45:36]Um that wasn't possible till till recently.

[00:45:40]Now it is, so I think that's going to, you know, that and things that go along with it are going to enable this next phase of the industrial revolution. And so all the things that people have been trying to automate over the last 40 years but have been trouble with, a good portion of them are now now going to be automated. There's two important thing I want to ask you. I think the holy grail for robotics AI, I think the the learning. And I think this is the notion you have. If you want to fix something, you should I think that's similar to what you maybe you're applying.

[00:46:09]How do you see the holy grail for the AI robotics? You have to to learn something, the continuous learning. And what's your thinking about all that?

[00:46:17]Yeah, well, I think um I think the things that are exciting to me right now is um force-aware machine learning, where it's actually taking into consideration joint torques and um interaction forces, things like that.

[00:46:35]You know, if you think about most of the machine learning that's been done today with like vision language models, um there's not force awareness in it.

[00:46:42]You know, there might be a little bit like, you know, you might have a video of like a person squeezing a Coke can and the Coke can crushes. And therefore maybe magically the neural network can learn that, hey, there's some interaction going on there.

[00:46:56]But if you look at like somebody grabbing an egg and squeezing it, well, there's nothing visual that you see that's except for maybe a little bit of compression in your fingers, but for the but it's so subtle.

[00:47:06]Um so being able to work in force information into some of these large models so that it gets um kind of common sense reason, I mean, reasoning about force interaction. And then also, you know, just common sense reasoning of of physics in general. Like, you know, if you drop something, if let go of something, it's going to drop. If you like have a book and you move a page more than 90°, it'll it'll keep going.

[00:47:29]Just all those common sense things um are really exciting to see people work on. And if you look at some companies like, you know, Generalist in their most recent model I've talked a lot about um how they're starting to see uh physical common sense evolved out of it. Um that's really cool.

[00:47:47]I think that's what you need for kind of general AI in a physical in a physical world.

[00:47:52]But I think even without that, um you can achieve a lot of the like economically interesting use cases without having general AI yet.

[00:48:02]You know, we don't we don't necessarily need to be able to just tell our robot, "Hey, watch that person grind that obj you know, grind that piece of metal and then you go do it."

[00:48:12]Right? We can still spend a bit of effort programming it in a some sense, even if that programming involves, you know, neural networks here and there.

[00:48:21]Um and they don't need to be fully general purpose. What we're looking for right now is multi-purpose. If we can get to the point where we've got, say, 24 applications going and we're shipping robots, you know, we'll be successful. So we don't need to solve general AI. If somebody does in the meantime, great.

[00:48:39]That'll just, you know, if we can have access to it, it'll make our jobs easier.

[00:48:43]But um but in order to get there, I think it's coming down to um um just uh physical common sense in the real world.

[00:48:52]And then um combining different modalities, which are, you know, people are doing, right? You got language, vision, now you're adding action.

[00:49:00]Um maybe I'll do, you know, sound. A lot of people are starting to add maybe even sense of smell once, you know, you can build the sensors.

[00:49:07]Um what whatever they happen to be, I think as you add more and more modalities, you just get closer and closer to having, you know, dare say cognition.

[00:49:16]Um Yeah.

[00:49:20]But personally, I think there's missing piece so far in because I think there's some way some way to think about what has been done so far. Maybe that's not the right way, but whatever you call it, the spatial intelligence, maybe that's the more of Yeah, I've seen to data and also the realistic measures of the objects but for you do you think there's something missing so far or it's it's need to Um let's see here. Okay, so what is missing in general?

[00:49:46]Um What things I'm waiting for I'd like to talk about is uh really good tactile sensors that you can actually put in fingers and do all the plumbing on and have them fit.

[00:49:55]Um Uh let's see here. Um from the machine learning point of view um Yeah, again having this kind of uh physical intelligence and or um common sense in a in a um foundation model so that you can train things more quickly. Right? Cuz you know, with with a lot of things right now you can train them no problem if you have, you know, 10,000 hours of your own data.

[00:50:20]But if we want to be able to do this efficiently. And then networks that simply learn with less data. And you know, a lot of people are working on that. So we'll we'll see how that goes.

[00:50:30]Um Having a way to generate data more uh efficiently and cost-effectively. So your best sources of data are still teleop but that's, you know, super expensive.

[00:50:43]Um then you have your Yumi Yumi style devices where you know, you have humans hold on to something while they're doing the task and all the sensing is just in that contraption so it doesn't matter that a human's doing it instead of as instead of a robot. Uh you can generate massive amounts of data that way but it's, you know, kind of different from how the robot might be doing it or it doesn't include knowledge of what the robot would be doing to do that. So you still need kind of that the teleop data.

[00:51:10]Um Yeah, so just having more efficient ways of of generating data and and more efficient networks and algorithms for you utilizing that data.

[00:51:21]Um yeah, you shouldn't need 500,000 hours of of data to to train um [clears throat] something that a human can learn in a few hours.

[00:51:31]But, you know, if that's what it takes, that's what it takes and then you know, that's what the industry will do until we make it more efficient. Yeah. And two things I I want to touch about the actuator and the reducer. Like for example, there's harmonic drive and cycle cycloid drive and also the planetary lower screws.

[00:51:49]There's there's many things in the space. And it seemed of course the design choice of you have to linear or rotary in the shoulder or leg.

[00:51:58]What do you think about all of that?

[00:52:01]Yeah, you know, I've tried them all.

[00:52:02]I've had robots with the three you mentioned.

[00:52:05]Yeah, with everything you mentioned, cable drives, harmonic drives, planetary, cycloid, um linear and rotary. Um I love them all and I hate them all.

[00:52:16]Uh >> [sighs] >> let's see here. So planetaries are awesome because they're just really simple, low cost, don't have much um reflect uh much friction.

[00:52:28]Um and uh quasi-direct drive with low gear ratio, like say up to 20 to one planetaries are just work really well and you see just robots working really well with it. You a little bit of backlash, little less precision but maybe that's fine.

[00:52:43]Um Cycloids are really efficient. They tend to be a little bit heavier.

[00:52:49]Um And um and it's hard to get gear ratios larger than say 20 to one. I mean, same with planetary but you can do multi-stage with both of them.

[00:53:00]Uh if you want a high high gear ratio in a small space, you best is is harmonic harmonic drives but then you have a lot of friction and um you go low efficiency because of that friction.

[00:53:13]Um and a bit of complexity.

[00:53:17]And because of that friction and the high gear ratios with reflected inertia, if you want to do good force control, you have to have a torque cell in there of some sort.

[00:53:26]Um One of the things I've been trying to determine for the last 30 years for myself is like at what ratio at what gear ratio and reflected inertia and stuff can does quasi-direct drive break down where you're just, you know, going from motor current to torque.

[00:53:44]Um you know, and how backdrivable does an actuator have to be?

[00:53:47]In, you know, in the in my early days in, you know, the MIT leg lab back in um around like 1994 or so, um so Gill Pratt had come up with the idea for so series elastic actuators which was amazing for getting just really good resolution force control.

[00:54:05]Um cuz most robots back then were just really rigid position control. They'd use like maybe 100 to one harmonic drives with no torque cell. So it was, you know, non-backdrivable, everything position controlled.

[00:54:18]And um so we wanted to make walking robots that had good torque control.

[00:54:22]So by putting a spring in series with the mechanism and and turning the problem into how much to compress the spring, that turns torque control into a position control problem which the actuators are good at doing.

[00:54:34]And back then we were so obsessed with getting really good torque control that I think we used much more compliant springs than we should have so that the torque resolution we got was incredible but it threw away a bit of the um the bandwidth you could get especially at high torque. Like cuz if you wanted to go from zero torque to really large torque and instantaneously, it would take time to accelerate the motors, wind up the spring and and get that torque.

[00:55:04]Um so in retrospect, if you know, I could go back in time, I'd just go and take all the springs we had and have them be about four to 10 times stiffer.

[00:55:13]Um but also back then you didn't really have good sensors. So the way we were sensing these compressions was was with like linear um um uh linear encoders that would have like only a thousand counts per inch. And so as you would if and some of the robots we played with we we designed, you would actually hear those ticks. You'd kind of hear it go "Nyer nyer nyer nyer."

[00:55:39]Um just cuz the resolution was so low.

[00:55:41]Now we have sensors that are so amazing that you can sense much smaller deflections. So it's so it, you know, so part of the problem back then again why we had to use more compliant springs was in order for the sensing to work well, too.

[00:55:54]But, you know, we we learned just, you know, a lot of the advantages of having compliant actuators.

[00:56:00]But you don't need them to be, you know, you don't need to get to zero impedance.

[00:56:05]Like with the human when you when your joints are limp, there's absolutely nothing nothing that you're feeling from that from the muscles.

[00:56:13]It's nice to have but it's not an absolutely need to have. So I think um somewhere in there, somewhere between 20 to one reduction with a kind of a standard size motor for say a knee joint and 50 to one is where you go from the reflected inertia and the and the reflected friction um being low enough that quasi-direct drive works to where it's just not there yet.

[00:56:39]And I don't have good numbers to put on it yet but that's something we're still still trying and struggling to figure out. And part of it is just trying things and, you know, try to see what we can get away with. Cuz if you can do quasi-direct drive and don't need a torque cell, you just save a lot of money and a lot of complexity.

[00:56:56]Um but the uh the trick though is that the sweet spot typically if you look at torque speed curves and maximizing efficiency and stuff like that, it's say a knee given on typical motors you can find somewhere around 50 to one is like the energetic sweet spot but 20 to one is like the quasi-direct drive sweet spot. And that's, you know, a factor of about two but everything goes with the square of gear ratio so you're really talking about like that factor of four energy loss um at high torque.

[00:57:29]Um so So that's one of the tradeoffs and still still trying and struggling to determine what the what the correct answer is there.

[00:57:39]Thank you so much for sharing that. And I'm going to encourage you technique to solve that. Uh how do you do that? Is simulation or analytic modeling Yeah, yeah, simulation is a good way to do it. So a simple way you can do it is, you know, set up a reinforcement learning algorithm in simulation and and start throwing more and more friction and inertia in the joints and see if it can figure it out.

[00:58:01]And you know, maybe part of the problem we've had all along is that we've been, you know, as humans using our, you know, modeling capabilities um but maybe, you know, maybe the a machine can figure out how to exploit the actual design that you have.

[00:58:16]Um you know, of course there's so certain physical limitations that you just can't get around. Like, you know, it's it's really hard to invert something like static friction.

[00:58:25]Um You know, and so here's an interesting way to think about it as far as um reflected inertia. Um And I'll give you a couple analogies.

[00:58:35]Say you're trying to just um push something on a table. Like I'm, you know, I'm pushing a paper clip here.

[00:58:40]And I can push it really kind of precisely and stuff and no problem at all and I'm just exerting, you know, micrograms on it.

[00:58:48]Now if I put a um cinder block in front of it and I'm pushing on a cinder block trying to manipulate the paper clip, becomes a lot harder.

[00:58:57]But if I were to put, you know, just uh uh you know, a small download in front of it, I can manipulate these paper clips still pushing on that download. So there's some so there's some ratio of the like the inertia of the thing you're trying to manipulate and the inertia that your actuator is presenting where once your actuator inertia dominates what's going on, you just can't interact with the world very well.

[00:59:20]So some ratio of those two I think is what uh what what you're trying to shoot for.

[00:59:26]And then as far as friction goes, the analogy would be I guess with the cinder block you got like a lot of just static friction and you're pushing and pushing nothing's happening and then it gives, right? And so there's some minimum force to overcome that static friction compared to the minimum force required to move the object you're really trying to move where that ratio becomes important and um so I think there's some some some mechanical design requirement in there that you can that you can come up with. Yeah. So the last question the the question I want to ask you about soft robotics and I think it again I'm coming to the feed and some of the like I think the common argument I hear like if you add like the for the the feet it's if you have to have truck with the ground and maybe I'm not sure if you have sensing there but do you think adding rubber bed would be enough if you have to have the contact with the ground and overall how do you see the soft element in the design because it seems I think one of the your early robot about passive dynamics I think you have a compliant mechanism or or just passive physical intelligence no motors no act Yeah yeah. And you do this for free and to be honest I'm big fan of that but I'm how you think about which part of the robot if I want to speak about now but you think maybe we can do this with compliant mechanism or passive physical intelligence and you have to rely on the actuation and the torque and energy all things that come with traditional way of doing things.

[01:00:59]Yeah yeah I think okay so you kind of have two questions in there I think or two topics that are related but a little bit different let me let me go ahead and play the video of the robot you mentioned um so this is we we had a IHMC um a project called the fast runner and this is one version of it this is the planar elliptical runner.

[01:01:22]Um so Johnny Gadawski was one of the main concept designers of this robot and Chris Watty was the main designer of it this is a um running robot and here it's going slow mo it's running 12 miles per hour on a on a treadmill.

[01:01:39]There's no sensors no feedback control no computer it's just a motor with a with a throttle and as you just squeeze the throttle it runs faster and and you let off the throttle it runs slower and we came up with some really clever ideas on how to make the spin self-stabilize.

[01:02:00]So if the robot pitches forward it will the pitch will just naturally correct itself.

[01:02:07]And so I think there's a lot of um really cool things you can do like this to put intelligence in the mechanism.

[01:02:15]And and you can make the argument that in biology some animals like ostriches are doing something like this because their motion is so quick that the time it would take for something to be sensed and something to be reacted upon um oops I'm sick here would um is slower than the time it takes to take a step.

[01:02:41]Like just in an ostrich you know like nerve conduction is something like 75 miles an hour or whatever and so you know given how fast they're stepping and whatnot the nerves going up to the muscle the muscle delay firing mechanical things happening that actually correct an action would happen [clears throat] two steps later.

[01:03:00]So there there has to be something that's maybe not perfectly stabilizing it but but making the stability be at least partially mechanical.

[01:03:09]And so that that robot was kind of an example of how you can do that now that was only planar it's not three-dimensional but we've also done simulations that have started to show that you can get 3D stability by kind of cycling your legs in such a way that if you start rolling you'll your inside leg will land in such a way that you'll turn into the roll which will then straighten you back up again because of centrifugal force as you go around the curve straightening you back up is one way of looking at it.

[01:03:39]Yeah so I've I've you know I've always been interested in passive dynamic stability and you know back in my early days when we didn't have a lot of computation and trying to come up with heuristics um that was one of the things I showed in my PhD thesis is how you can build stuff into a walking mechanism for a planar bipedal walking where you get a lot of the stability properties um through some open loop control mechanisms.

[01:04:05]For a 3D humanoid that's doing pretty active balance you need pretty active control.

[01:04:10]Um you you can simplify some things with some of the dynamics of built-in mechanism but for the most part you need to kind of have a sense of how you're falling and get your foot out there now and and I don't think like humans are using too much passive dynamics for for that.

[01:04:30]Other things they are have been shown to probably be doing like if you're doing like a layout somersault um back Rob Plater's PhD thesis from quite a while ago he showed that he can do a somersault with a twist by simply moving your arms in a certain pattern and then you can stabilize a layout somersault just by stiffening your arms and that passive dynamic stiffness will make it such that when you start twisting which is a natural thing to do you have to take a tennis racket and spin it about that axis it'll twist when you do it but if you just put some springy arms on it it won't and the arms will do this corrective action that looks really intelligent but really it's just a passive mechanism.

[01:05:12]So the the hypothesis is that humans are doing that when doing layout somersaults. So there's certain things we'll use passive dynamics for but I think for walking and running we're probably not too much.

[01:05:23]Some things where we're just letting the natural dynamics happen like swinging your leg you know in the middle of swing your muscles are turned off and it's just and I think a little bit of kind of just that natural dynamics of that where your foot happens to typically land helps stabilize your speed a little bit but for something like when you get pushed your brain's kicking in your your spinal cord's kicking in and you're you're stepping out there actively. Very interesting. So do you add those the rubber bed that's I'm I'm just curious you had rubber through the feet of the robot?

[01:05:54]>> Oh yeah yeah so that was the second part of the question and is is um for um for soft materials um I think there the kind of the general concept is anytime you're acting with interacting with something that's compliant you need to be stiff and anytime you're compliant interacting with something that's stiff you need to be compliant.

[01:06:17]I think it's kind of a general concept and and you see that a lot in like prosthetic design where wherever something's touching like a bone you want there to be foam padding and anywhere it's touching muscle or or spongy stuff you want it to be a hard harder shell.

[01:06:34]Um >> [clears throat] >> and so if you make a robot with just with metal foot and try to walk on a piece of metal it'll be slipping all over the place. Um so you do need some sort and since the world tends to be hard what we're walking on you know if if all we were walking on was um trampolines then our feet would could probably be better to be rigid actually.

[01:06:58]Um so it's kind of this you know stiffness matching or or whatever. Yeah so um we've tried different things you know you the other day we just threw a a shoe on the robot for the fun of it and it walked just fine.

[01:07:10]Um What did this do? I'm just curious what the shoe did maybe differently like did you notice the differences in the torque by the joint or not at all?

[01:07:20]Not much um you do need to make sure that your walking algorithm is able to um adapt to your ankle moving around to be robust to that. Most you know most walking algorithms now are cuz when you know if you train them using reinforcement learning in simulation you'll force it to walk on a lot of different terrains so that it's not expecting flat ground.

[01:07:43]Um and if you have good compliant actuators if you land on like a slope it'll go like that right?

[01:07:50]Um so so the actual ankle and stuff tends to comply just fine.

[01:07:58]If if things are too um it it really comes down to like the natural frequency or the delay in getting your center pressure from one side of the foot to the other versus the closed loop bandwidth that it's trying to do that at right? So if it if you've got too much if it's too spongy and the robot's trying to move its center pressure from one side of the foot to the other instantaneously you'll start chattering.

[01:08:22]And we you know what at at IHMC when we were doing a lot of um control using center pressure control we would see that a lot so we had to make sure that the controller rate limited how quickly it could move its center pressure from one side of the foot to the other or it would start doing this the shaky thing.

[01:08:40]Um so you don't want it to be too compliant you know like you know walking on a on on a trampoline is really hard to do.

[01:08:48]Um so there's some range of durometer you want in there but I think it's a really large range.

[01:08:55]And then same with fingers right if you're if you're grabbing rigid objects um you need some give in your fingers or else it'll just you know just slide and also by having that given it you can you can kind of you know kind of surround the object you're grabbing just by pushing on it so now I can move it sideways without you know without it slipping just because you know I've surrounded on it. I think a human foot does that a little bit when you're barefoot um like if you're walking on rocks and stuff you'll tend to kind of grip them just because of the way your bones and your your um spongy material goes around the rock. And there's been some people who design I think DOR did one um kind of a foot that does that where it's got some um different um mechanisms in it that will comply to what it's stepping on. So now it's got really good um co- um you know, well it can push it at from different angles.

[01:09:50]And if you think about um you know, suppose you're walking on pointy rocks.

[01:09:56]Okay?

[01:09:58]If you can step on that on that pointy rock and have and comply a little bit to it, now you can apply forces this direction even though you're just on a line contact.

[01:10:09]And then if you want to apply forces this direction, you just rotate a little bit and push that way. If you want to apply forces there, you just have to rotate a little bit and push that way.

[01:10:17]So it's actually easier to walk on pointy rocks than it is to walk on like just a 30° shear surface. You know, you just slip on that, but pointy rocks are no problem and and that's the reason why is because you get to kind of choose your coefficient of friction or your your your surface normal.

[01:10:36]Um just by moving your foot a little bit or by having compliant material, you you get that whole range of potential surface normal that you can that you can push without slipping.

[01:10:47]And what was the follow-up in this part because human feet the shoe are designed for yeah, human human feet and it seemed again the the robots are at flat feet.

[01:10:56]So do you think that we need a shoe for a humanoid in that case?

[01:11:00]>> I mean you you something that allows you to comply um throughout the whole foot so that if you if you're walking on like gravel or something, you need it to like just comply and otherwise you'll be slipping all over the place.

[01:11:13]Um so whether or not you're actually wearing a shoe or just designing that as the sole of the foot, it it essentially is a shoe.

[01:11:21]Um And then I do think having either your foot shaped in such a way that it makes it nice to get up on your toes when you're doing toe off at the end um is nice and that can either be um just a shape cut out into the foot so that it kind of rolls onto it um or or pre- more preferably a spring so that you get um some torque at the joint too.

[01:11:45]And then most ideally actuating it so that you can you know, continue to balance even if you're uh even if your surface contact is only a few square centimeters.

[01:11:56]I mean human humans are incredible, you know, when you're standing on your toes on one foot like I am now my surface area is only about that big.

[01:12:04]And I can keep my center pressure under my center of mass and it's just it's incredible. So being able to you know, having really good uh state estimation too is important for that.

[01:12:16]Mhm. So you're thinking that there's something maybe in Gentoo I'm not sure I'm just curious you would be considered designing that like that.

[01:12:24]>> we're we're not currently thinking about designing a foot. Uh it's it's not currently in the near-term roadmap.

[01:12:29]But um but maybe uh sometime in the coming coming generations it could be.

[01:12:36]Yeah. So the last question maybe before closing, I don't know if you have any final words for people listening in the robotic community.

[01:12:43]Uh but yeah, maybe the challenging part of your career I'm not sure what the thing that you learn at the hardware or our hard moment for you as well. What what Sorry, what were some of the hard moments?

[01:12:55]Yes, hard moments here.

[01:12:58]>> [snorts] >> Oh.

[01:13:01]I mean not you know, noth- nothing about the problem is easy and it's really like I I like to look at this whole enterprise as a 100,000 piece puzzle.

[01:13:11]And you have to like you know, a bunch of them are on the floor, you got to find them, you know, you can't you don't know where some of them are.

[01:13:18]Um and but each one can be figured out, you know, you can work the problem, figure out just about everything.

[01:13:26]Um some things would just just aren't feasible yet because of either material science or you know, you just need a couple more generations of computer whatever, so you got to be patient. But I think now is the time where where we can um get commercial viability and actually have humanoids be useful for people.

[01:13:47]Um and and it's only been recently, you know, sometime in the last 10 years or so.

[01:13:52]Um and because of that, you know, you're you're now starting to see customers come out of the woodwork. I mean and and it's pretty target-rich environment as far as like how many customers you see um that have different applications just all over the place.

[01:14:09]Um and because of that, um we're starting to see a lot more investment which allows it to happen. So kind of everything's coming together with where um where it seems like now might be the time whereas, you know, like I said, 5 or 10 years ago was just way too early. So um so I think there's tons of opportunity.

[01:14:29]And um you know, there's there's a lot of humanoid companies that you see and you know, you see a new new one almost every week, new robot doing something amazing almost every day.

[01:14:39]Um so it's so it's super exciting area to be in right now. And I think you know, I think it's similar to car companies' early days where you had couple hundred car companies and then you know, now you got a few dozen. Um and I think it'll be kind of the sim- similar situation where you know, those who can successfully do useful stuff for their customers will be the ones who survive and then um and then there'll be you know, mergers and hyphenated names in company names and and let's see how it goes. But I think you know, I think we have as good of a shot as anyone.

[01:15:12]Uh we got amazing team, we're about 60 people now.

[01:15:15]Uh looking to grow to about 100 this year.

[01:15:18]And um uh mostly we're located uh mostly in Houston and Pensacola. Considering making another office, but we'll we'll see how that goes. Um but uh so if any of you listeners are have um reason to be in the Houston or Pensacola or south southeast area in general and looking uh we're looking for talented people.

[01:15:41]Awesome. Again, Jared, it was such an honor talking to you and I'm I'm it was really great. I enjoyed talking to you and I it's uh yeah, I think you have been building this uh entire field of legged robots. So I I all all the respect what you have been doing and wish you all success. Thanks, Marat.

[01:15:56]It's nice talking with you. Bye.