...how can robots work together?

Ever Wonder? / October 14, 2020
Photo of Kirstin Petersen, roboticist and professor of Electrical and Computer Engineering at Cornell University.
Image attribution
Courtesy of Kirstin Petersen

Courtesy of Kirstin Petersen

We are starting a new series on robots! We've seen some pretty amazing advances in robotics in just the past few years. Maybe you've seen videos of humanoid robots that do backflips or robot hummingbirds that can hover in midair. Now, a video of a single back-flipping robot is pretty amazing. But what if you could get a whole team of robots—hundreds or even thousands of robots—to cooperate with each other?

Do you ever wonder what would happen if robots worked together?

Kirstin Petersen is a roboticist and professor of Electrical and Computer Engineering at Cornell University. And in her lab, she builds cooperative teams of robots, called robot swarms, that are inspired by insects like ants, termites, and bees. We talked about the challenges of getting huge numbers of robots to communicate with each other and how studying nature can offer solutions.

Have a question you've been wondering about? Email the Ever Wonder? team to tell us what you'd like to hear in future episodes.

Subscribe to our show on Apple PodcastsSpotify, or Google Podcasts. To see a full list of episodes, visit our show’s webpage.


Perry Roth-Johnson: 00:06 

Hello! This is Ever Wonder? from the California Science Center. I'm Perry Roth-Johnson. Today on the show, we are starting a new series on robots! We've seen some pretty amazing things in robotics in just the past few years. Maybe you've seen some videos of humanoid robots that can do backflips or robot hummingbirds that can hover in midair. And these examples are just the tip of the iceberg. So for the next few episodes, we're going to talk to some of the engineers who design these robots and learn more about what's happening in the field today and what might come in the future. Now, a video of a single backflipping robot is pretty amazing, but what if you could get a whole team robots--hundreds, or even thousands of robots--to cooperate with each other? Do you ever wonder what would happen if robots work together? Kirstin Petersen is a roboticist and professor of electrical and computer engineering at Cornell University. And in her lab, she builds cooperative teams of robots, called robot swarms. They're inspired by insects like ants, termites, and bees. We talked about the challenges of getting huge numbers of robots to communicate with each other and how studying nature can offer some solutions. It was a fascinating conversation--check it out.

Perry Roth-Johnson: 01:24 

Professor Kirstin Petersen, you are a roboticist and a professor of electrical and computer engineering at Cornell University. Welcome to the show!

Kirstin Petersen: 01:32 

Thank you. Great to be here.

Perry Roth-Johnson: 01:34 

Absolutely. So, uh, your work involves getting robots to work together often in quite large numbers. Um, but, but before we dig into your work, I want to just get a sense of the state of the art now. So do we already have robots working together in our society? Like in some business sectors, like w where are we now?

Kirstin Petersen: 01:55 

Well, the most obvious example out there is a Amazon robotics, Amazon robotics, and biggest warehouses right now have many thousands of robots working together to shuffle packages around very efficiently and warehouses in order to very quickly get them packaged and sent to you with very few days away.

Perry Roth-Johnson: 02:14 

I'm looking at a picture right now, and it's like this kind of souped up orange Roomba, almost looking devices, like kind of squat rounded square.

Kirstin Petersen: 02:23 

Surprisingly simple. Yeah.

Perry Roth-Johnson: 02:25 

Yeah. So it's not like um at a particular, uh, loading dock that you might see it at, at a business or a warehouse where someone has to come in with a forklift or a pallet Jack from the size. This thing just drives right under it doesn't stick out at all. And it lifts the whole thing up. And then it drives around the warehouse. Is that how it works?

Kirstin Petersen: 02:41 

Yep. That's exactly it. But, you know, there's like a, a huge planning problem there, right? Like you have these thousands of robots and you need to figure out where all of them go. And one thing is figuring it out what you would probably pre compete, plan that out. Um, but you know, with all the robots, there are practical things that happen on shelf can topple over, maybe something spills out, maybe one robot runs out of battery or something goes wrong. And so planning position avoidance for all of these robust and efficient navigation is very interesting.

Perry Roth-Johnson: 03:10 

Yeah. Tricky problem. So I think that's a nice segue into, like, what is your vision for the future of robots? Uh, I know you work on swarms, uh, like how is your research a little different than Amazon's approach? Or like, how are you trying to move that, that work forward?

Kirstin Petersen: 03:26 

So I think first of all, I think, you know, single robot systems are amazing and there's so many fantastic examples out there. Uh, but I worked on a, sort of a special subset of that problem, which is what can we do if we didn't have one or 10 or a hundred? What if we had a thousand, 10,000? Um, and so, you know, and everything changes a little bit, right? So if I had 10,000 or 100,000 robots, um, you know, there's opportunities there, right? So every single robot realistically would have to be much, much simpler and much, much cheaper because otherwise you couldn't support that biggest swarm. Um, but if you have many robots, they could potentially work together. They could cover really large areas. They could do things that single robots just couldn't do. And you might end up with a really robust system where even if some of them fail or half of them fail, right. The rest sort of reorganize, um, and still take on challenges. So there are opportunities in swarms where we can apply them to applications that just don't currently merit robot systems.

Perry Roth-Johnson: 04:25 

That's crazy to think like 10,000 or a hundred thousand robots working together. Can you give me some examples of like these things that, you know, maybe are over a big area that you might want a swarm to tackle?

Kirstin Petersen: 04:37 

Yeah, sure. Uh, so, uh, say the deep horizon oil spill, right. If I wanted to try to contain a massive oil spill, it just makes sense to have a lot of simple robots, try to collaborate on containing that oil spill, if I needed to, uh, search out a big earthquake for victims. Um, the earthquake site, right? You want coverage and that's really something new, robust can give you. Every single robot doesn't necessarily have to be really applicated, but they need to spend the large area. Um, you can also imagine if you're trying to build a really big building that has some really big spires in there, right? You need many robots to collaborate on pairing these big unwieldy things, because a single robot would just have a hard time doing that. Um, but you can also imagine that you're sending your robots out to do a task that we just don't know a lot about. And so chances are that something is going to fail and you don't want just, you know, one wheel to take the example from the Mars Rover, right. One wheel will give you second sense to damage the entire robot, you want just that part to fall off. And then the rest of it continue.

Perry Roth-Johnson: 05:39 

Right? Yeah. That it's like so heartbreaking when you spend all that time and money to ours, it breaks. I want to go back, uh, to the Amazon warehouse, example for a second, like, um, there's this notion in, in science, people like Star Trek, the board where there's like this centralized entity that communicates with the rest of the, the entities and the, and the board collective, like are the Amazon warehouse robots, would it be fair to say they're kind of like the board, there's like a central thing talking to all of these robots picking up product?

Kirstin Petersen: 06:15 

Yeah, absolutely. So for the record, I'm not from Amazon robotics, right? I'm sorry, I'm talking from what I hear from their engineers. Um, they are mostly centralized and for, I think up until they hit around two thousand robots, that's what they did. They had a fully centralized control. Um, but I think even them, uh, now as they keep pushing that barrier have hit a limit where they're actually starting to look into more distributed controllable.

Perry Roth-Johnson: 06:38 

What does that mean to have distributed control? Like, is it more like a hive of bees? Like where each bee can think on their own?

Kirstin Petersen: 06:44 

That's exactly right. Yeah. So, so many people, right. They think of a hive of bees as like, well, there's a queen. She kind of tells him what to do, but that's obviously not true.

Perry Roth-Johnson: 06:53 

It's not yeah.

Kirstin Petersen: 06:53 

Every single bee has some kind of free program, the, you know, desire intelligence. And they use a lot of cues about what they see in the collective, and then they figure out what to do. And so there's a lot of these little emergent collaborations happening, and that somehow gives rise to this whole collaborative behavior, which is incredibly robust and incredibly efficient and incredibly adaptable. Um, and so that's the kind of, kind of methods we're looking into, especially interested in this notion of like, if I have a robot and it doesn't have the ability, or it doesn't want to communicate with everyone else in the collective, because that's a large communication. What, how, what other ways can we use to make everyone have this emergent behavior? And it turns out if you have physical robots, we're willing to implement them. Well, then they have the physical substrate that they're operating on and they can use that substrate in like issues in that substrate to try to coordinate. So they're essentially using their environment. That's sort of a shared database.

Perry Roth-Johnson: 07:54 

Yeah. Can you talk a little bit more about there is this project, uh, you and your colleagues became pretty famous for, I think it was back in 2014 where you had a robot collective, uh, or swarm called termies. Right? It was like, they were, they were kind of modeled after termites and they...

Kirstin Petersen: 08:10 

Right. Right, right. So, uh, these are, these are not, uh, North American termites for the record.

Perry Roth-Johnson: 08:15 

Oh ok.

Kirstin Petersen: 08:15 

These are Southern Southern hemisphere termites that build very big mounds out of mud. Um, if you haven't seen them yet you should absolutely look them up. They're quite amazing. You can sort of think of it as like this, the size of a termite compared to the size of one of the big mounds of similar to, or bigger than a person compared to the size of the Eiffel Tower.

Perry Roth-Johnson: 08:33 

Whoa. Ok.

Kirstin Petersen: 08:33 

As incredibly complex, uh, and fixed structures. Um, what we were trying to get at there was, uh, one of the big problems right now, which is in construction, right? So the future of construction, uh, we really need to automate more things. There's um, it's, it's a big source of risk for workers. Um, there's a lot of energy wasted. There's a lot of people that don't have adequate homes. And so we're trying to look into how many robots work together to build structures. Um, so we took inspiration from these termites and millions of which he comes together to build these big, big mounds.

Perry Roth-Johnson: 09:07 


Kirstin Petersen: 09:11 

Millions, yes.

Perry Roth-Johnson: 09:11 


Kirstin Petersen: 09:11 

So, uh, so the idea was that again, you don't actually need a centralized controller and you don't actually need robots that are super capable because what you have here is the opportunity to, um, you know, the structure that you want to build up front. Right? So you can sort of plan around that and try to figure out, okay, which are actions that we can execute in parallel, and which are actions that have to execute sequentially, uh, such that we still end up building a structure, right. And if you don't do that, then you're gonna end up with problems where like one robot builds a tower, then no other robot is going to come up to you. Right. Because they just don't have the kind of mobility necessary or another robot ends up building all these gaps that can't really fill in later. So there needs to be someone who has been that many rules. So if you understand the hardware you're building, you can pre-plan a lot of these things. Now, once you have this pre-plan you can give that to the robots, and then because the robots are building and literally modifying the environment, they can use the modifications to try to guide what other robots should be doing. So here, the example is I just place a brick, but I don't need to tell you that I placed that brick. It's enough that at some point later on you come by and you'll see that that brick was placed. And that tells you because of this pre-planned we have, whether or not you should add a brick next to it. Um, and so this is what I mean, when I say the worst of leveraging the environment as a shared database, we're literally modifying the environment and that tells other robots that come by later on what they should do.

Perry Roth-Johnson: 10:35 

Got it. So is that the same kind of behavior you saw in those termites in the, in the Southern hemisphere and other insects?

Kirstin Petersen: 10:44 

Yeah. So it's a concept called stigmergy in biology, and is quite well known, um, that, uh, many, many social insights, so that work together in large colonies will actually alter their environment by adding pheromone or by adding little physical artifacts, like little dollops of mud, like the termites do, and that prompts other termites to do other things or specific things. So again, the sort of behavior emerges because they're literally shaping the environment to fit their own purposes.

Perry Roth-Johnson: 11:13 

So, so like the image I have in my mind is like, when I haven't cleaned the kitchen and a line of ants comes out to get the food, but if you're kind of a jerk and you put your finger in the middle of the line, that sometimes the ants get confused. So like they're picking up on the cues, the pheromones that other ants have left on the counter so that they know how to get to the food.

Kirstin Petersen: 11:36 

Yeah, that's exactly right.

Perry Roth-Johnson: 11:36 

Is an example of like social behavior in insects?

Kirstin Petersen: 11:39 

That's, that's a great example. Right. And what's super cool about them is that when you place your finger in there and you don't take it away, the ants will start exploring. Right? And so when one ant then finds the food again, a tracer steps back and lay pheromone as they go back. And so now if another ant comes by and says, Oh, well, there's a pheromone trail here with some probability, I'm there was some chance I'm going to pick that trail, I'm going to move along it. And then also find food they're going to reinforce that trail. So it's real reforms and is constantly adaptable to preservation, like you putting your finger on it.

Perry Roth-Johnson: 12:12 

Um, so, so you're getting your inspiration from these, we're calling them social insects. Um, how do you study them? Like, do you have ant colonies in your lab or are you going outside and getting dirty to observe them? And then you come back in your lab to build something, or you collaborate with other people to study the bugs, like, like what's your approach.

Kirstin Petersen: 12:30 

Yeah, that's actually exactly what we do a first year in my lab. We, uh, we bought a shovel and we went out with some biologists, field biologist, and we got a bunch of ant colonies and stuck them in the lab.

Perry Roth-Johnson: 12:43 

Oh really? I was joking.

Kirstin Petersen: 12:45 

That's actually what we've done. Um, later on, we have a transition to honeybees because Cornell University has a great track record of honey bees and they're easy to keep and we kind of know how to deal with them. And they're, um, it turns out very, very important for agriculture pollinators. Um, but you know, we're not biologists. Uh, we love to study them because they are at the end of the day, our inspiration, but what we really are, um, you know, we're engineers that are capable of building tools that the biologists don't necessarily have access to. So we work with the biologists to try to figure out what kind of tools they need so that they can study the better. Right. And our end goal is whatever insights they then come up with with those tools we can use, um, to try to program robots better.

Perry Roth-Johnson: 13:30 

Hmm. So like what, what kind of tools are you building for the entomologists?

Kirstin Petersen: 13:34 

A lot of visual trackers, a lot of, uh, you know, detecting when honey bees go in and out of their colonies, uh, we're trying to, uh, build little, uh, backpacks that we can stick onto the bees to actually report.

Perry Roth-Johnson: 13:47 


Kirstin Petersen: 13:47 


Perry Roth-Johnson: 13:47 


Kirstin Petersen: 13:48 

Honey bee backpacks. So that's that collaboration with one of my colleagues, Al Molnar. Um, and we have, we have very good results so far, so we're quite excited. Um, so we're sort of starting to enter a field where you can almost think of swarms as these bio hybrid things where like, we don't just try to operate around what is already going on in the field. We're actually trying to incorporate knowledge on what is going on in the field with all the insects that are there, and then just sort of applying robots to, to fill out the gaps where the insects aren't, for example, pollinating.

Perry Roth-Johnson: 14:21 

Hmm. Are there any challenges you run into to like trying to work across fields? I, I know that's more and more common these days. Um, but at least my impression in engineering, it's, it's still a little, a little unusual to, to force yourself, to like, have to learn, you know, two different fields and be able to cross talk between them.

Kirstin Petersen: 14:39 

Yeah, definitely. There's some barriers to break down. Uh, I think mostly it's a mental barrier, right? Like, so every time you take something new on and you're like, Oh my God, okay.

Perry Roth-Johnson: 14:46 


Kirstin Petersen: 14:46 

Is this going to go well? Uh, but I think, um, as we said, more and more people are starting to appreciate all the great things that can come from interdisciplinary research, but also when more people are starting to appreciate, uh, work that comes out of it, you know, for the hard work that it is. And for the ingenuity that goes into actually completing things that take into this work. And so, uh, it takes a little bit of zeal, a little bit of backbone, but if you're willing to fail a bunch and try a bunch, um, there's, there's great things to be discovered.

Perry Roth-Johnson: 15:23

I have to ask you about some of these things that have popped up in pop culture. Uh, like the Robobee episode in Black Mirror, uh, in like your experience, what are people usually most worried about when you present your work and there's these kind of dystopian pictures in pop culture, like, and what do you wish people knew or understood better?

Kirstin Petersen: 15:44 

Um, that's a great question. Uh, so I think most people just see my work and they're like big hero six, right? So I think soft robots and your swarm robots that kind of all fits in there. Um, and it's not a bad movie to compare it to, right. So people are worried that the robots turn evil and take over. Um, but also, you know, actually I think movies generally today do a pretty good job of capturing the fact that it's not that the robots turn evil is that evil people inflict, um, whatever intentions they have onto them. I don't think any roboticists sets out to make a robot that, you know, in any way steals or harms human race. Right. All roboticists, uh, are trying to do things that better in the world. I think the biggest, uh, sort of realistic thing that people worry about is sure do, are robots going to turn evil? And I think, um, anyone who's ever worked with a robot knows that they're pretty far from that. If they take over the world, when they, I think might be a job has done, then we've created a really good robot, but I don't think we're going to get to that. Um, I think the biggest problem is, um, are they going to take over our jobs? Right? So people are worried that we're just, we're just, we're trying to get rid of agricultural workers. We're trying to get rid of construction workers is that's not the case at all. Right. We're trying to take away the things that are really hard or dangerous or, or dull, or things that people just can't do today. Those are the things we're trying to, um, to alleviate

Perry Roth-Johnson: 17:11 

Just before we wrap up. Uh, are there any other stories you'd like to share? Uh, what else do you want people to know?

Kirstin Petersen: 17:18

So we kind of jumped from topic to topic in my lab. We do a lot of different things. Um, you know, we, we, we, we work with these swarms, but it's swarms applied to soft robots for exploration, but it's also swarms applied to agriculture. Um, and anything else you can really imagine. So that's why we covered all of these different topics. Uh, I do think people should appreciate if they can, if you at all have the opportunity to go into research, it's a super exciting thing to do. Um, because you get to combine these drastically different fields, um, you know, into one thing that you think will really change the world. Um, and I think it's, you know, engineering, robotics, um, research, some of these areas are places where you really actually, you know, they enable you to actually do something, something pretty hardcore.

Perry Roth-Johnson: 18:11

Awesome. So where can people follow you online and find your work? Uh, professor Petersen?

Kirstin Petersen: 18:16 

Uh, so we have a website, uh, spelled CEI (it stands for Collective Embodied Intelligence). So, cei.ece.cornell.edu.

Perry Roth-Johnson: 18:25 

Awesome. Well, professor Peterson, uh, it's been amazing talking to you, uh, learning how robots can work together, especially like thousands of robots. I'm still kind of reeling, you know, thinking how that might work. Uh, thanks so much for coming on the show and sharing your work with us.

Kirstin Petersen: 18:39 

Absolutely. Thanks for having me.

Perry Roth-Johnson: 18:42 

Well, that's our show and thanks for listening. Until next time, keep wondering. Ever Wonder? from the California Science Center is produced by me, Perry Roth-Johnson, along with Jennifer Castillo. Liz Roth-Johnson is our editor. Theme music provided by Michael Nikolas and Pond5. We'll drop new episodes every other Wednesday. If you're a fan of the show, be sure to subscribe and leave us a rating or review, or tell a friend about us. Now, our doors may be closed, but our mission to inspire science learning in everyone continues. We're working hard to provide free educational resources online while maintaining essential operations like onsite animal care and preparing for our reopening to the public. Join our mission by making a gift at californiasciencecenter.org/support.