 Okay, thanks very much. Thank you very much for the invitation. So I want to talk about a couple of different topics here under the broad umbrella of collective construction. So large numbers of agents building interesting things together. And the core inspiration for all of this work has been mound building termites. So the photo on the left here is an example of a termite mound. This is actually a particularly small one. The record for these is more than 13 meters tall. From the outside it looks like a pile of dirt. From the inside it's got all kinds of complicated internal architecture. So if you fill the mound with plaster and wash away the dirt, you get the photo on the right. This is showing the network of tunnels. And these are performing a role for the colony. They're helping with atmospheric regulation. And the whole thing is built by millions of independent insects. Each one is limited. They don't know what the others are doing. This is a bit dark. I guess you can't quite see them. But they don't know what the overall state of the mound is like. It's not as though they're getting instructions from the queen. Somehow all of these termites going around, encountering whatever they encounter and doing their thing, wind up producing these huge complex structures. So there's kind of two directions you can go from that. One is to say that's amazing. That these termites are doing and how does that lead to the things they build? And the other is to say that's an amazing proof of principle that that kind of thing can be done. So how do we engineer a system like that? How would you build and program an artificial termite colony to build for you whatever it is you ask them to build? So there's kind of two different sides to this question here. There's the forward question, the scientific one, of given a set of agents and the rules that they're following, can you predict what that's going to produce when they all build together? And then the inverse question is more of an engineering one. Given a particular structure that you want built, can you come up with a set of rules for a bunch of independent agents that will wind up producing the thing that you asked them for? So that's the macro-micro problem that Marco was talking about yesterday. So I want to spend a little bit of time talking about the second of these problems, the engineering one, and then hopefully spend most of my time talking about exploring the first question or studies trying to understand more about what it is that the actual termites are doing. So the project that I want to talk about briefly to start with is what we called the termite system from a few years ago. So this is a collection of independent climbing robots that use this specialized building material to build structures according to what the user asks them for. So the idea is the robots are independent, they're limited, you can give them a picture of what it is that you want them to build, and it doesn't matter how many robots there are, it doesn't matter which one encounters which tasks there are to do, in what order, with what timing. Despite all that kind of unpredictable variability, you can write down provable guarantees that all of them working together and following these automatically generated rules will wind up producing exactly the thing that you asked them for in the first place without getting stuck along the way. So this is a system where we really tried to have on the one hand the firm theoretical underpinning that lets us say, you know, you can ask for a particular result and have a guarantee of getting that, and then carry that through and on the other hand have a physical system showing in hardware that the things that we're asking of the robots are reasonable things to be happening in real life. And the approach, as well as the problem, was inspired by termites. So we've got independent robots, they're limited to what they can sense themselves with their own sensors and their own bodies. They don't have anything like an external camera, they don't have GPS. They're building large scale things, climbing over the structures in progress to get to places they couldn't otherwise reach. So the size of what they build is not limited by the size of the robots. And they're coordinating via this traditionally termite inspired idea of Stigmergy. They're looking at where material has already been put down and using that to help them decide where to put additional material. So I don't have time to talk about this system in a lot of detail, so I thought it might be interesting to talk about it through the perspective of a couple of key ideas that we relied on that I think were really instrumental in getting the system to work. These may be familiar ideas, right? So the first one I want to talk about is to make use of the physical world. So this is an idea you'll sometimes hear talked about under the name of mechanical intelligence or morphological computation or other names. And the idea is that if you can design a system so that the physics of the world handle certain tasks for you, then those are things that you don't need to deal with with an explicit controller. So for instance, the blocks that these robots maneuver on and build with have these sort of bowl-shaped indentations on top. And when a robot is turning in place, those bowls help keep the robot from falling off the structure, so it doesn't need to worry about turning very precisely. When they put down a block, they don't need to worry about putting it down very precisely. They put it down in about the right place and kind of move around a bit and the physical features slide the block into precise alignment and it locks in place. So that's a way that the robots are using the physical world to help with physical manipulations. Another way they're using the world is to help with localization. So on the upper surfaces of these blocks, there are these black and white markings. On the bottom face of the robot, there are six infrared sensors. So this is basically a six-pixel black and white camera that's looking straight down at what the robot is standing on. So as a robot moves around on top of a structure built out of these blocks, it sees different patterns of black and white and that helps it keep track of where it's moving relative to the structure. When it's away from the structure, it has the set of ultrasound sensors and it can use those to tell about how far away the structure is. So in this video, the robot is using the ultrasound to start with to walk around the perimeter of the structure. So using the structure as a reference to figure out where to go. No other odometry or anything. And then when it finds this marking on the ground that tells it it should climb onto the structure, once it does, it has these black and white patterns to rely on that let it keep track of its movement in a coordinate system which is physically built out of these blocks. So the structure is playing the role of the structure that they're trying to build but it's also being used as a physically embodied coordinate system for the robots to make reference to. So the second idea we relied on was sort of paradoxically one of the things that made it possible to engineer this complex system was by limiting the complexity of what could take place. So let's suppose that you want the system to build this step pyramid. Here's a different representation of the same thing. The gray squares are just looking down on the pyramid from above and I've added these colored arrows. The colors are just showing the direction. So like all the green is up, all the red is left. And these arrows are a set of traffic laws that we've imposed on the robots. So it limits how they're allowed to move while still leaving a lot of options open. And that restricts the flow of robots and material through the workspace to a set of sort of limited consistent ways. So it's a bit like the difference between bumper cars at an amusement park and driving on a highway. With bumper cars you've got stuff coming in from all directions. You have to deal with a lot of possible things happening. You have to stop all the time. On highway everybody has agreed in advance to drive in the same direction in the same place. And so that lets you drive very fast and very much more safely. With the flow of material, that also means that very simple rules that the robots follow are sufficient to allow building in an effective way. So like if you're building a brick wall, you don't just add bricks in a random order. You build each row of bricks starting from one side and going in a consistent direction. Because that's obviously the way that makes sense to build a brick wall. And if the robots and material are coming from a consistent direction that makes it easy for them to build in the same kind of effective way. So one of the features of the Termini system is that you can ask it for a particular blueprint and it'll build you that thing. And I think that's useful for human relevant construction systems because typically we like to know what we're going to get in advance. We like rectilinear structures, we like regular buildings. And if you had an automated construction system that gave you some weird lumpy organic looking thing, you'd be unhappy with it. I say that. I mean, there are exceptions. But typically we know what we want when we start. But you can also imagine situations where you don't actually want to have to specify a final design in advance. Imagine you've got a robot exploration system and at some point they come to a river and you want them to get to the other side and continue exploring on the other side. Well, you don't want to have to tell them exactly what they need to build this bridge. You don't want to have to get out there and survey the exact details of the terrain on both sides. You just want the robots to deal with it. So maybe they start building a cantilever and as that extends they need to go back and add more material to reinforce the anchor where they started and eventually they make it across. And you don't care about exactly what that bridge looks like, you just want it to be functional. So that's a direction that we've... This is really very dark, but all right, so it is. This is a direction that we've been looking at more recently. We've been looking at a problem of climbing robots building with these triangular truss structures where the idea here that we're relying on is that as robots move over the structure, that's going to change the... And as they add material, that changes the internal forces within the structure. And we're going to let the robots measure those internal forces and use that to help them decide what to do. So the blue and the red lines here are meant to be showing the locations of force sensors. The idea is when a robot comes to one of these nodes, it's able to evaluate the forces on each of the struts coming out of that node. And use that to decide, is it safe to venture out on a cantilever or do we need to go back and add more reinforcement somewhere else? And to summarize the upshot of this, by paying attention to those forces, it's possible for robots to build cantilevers that extend much further out unsupported over gaps than if they're not paying attention to those forces. They can get to much longer distances before the structure fails either because it breaks through poor decisions made up until now, or another possible failure mode is the whole structure can overbalance and topple into the chasm. Those failure modes are put off much longer. And Nathan Mellenbrink, who's the student who's been working on this, has also been developing prototype hardware, both for a robot that could maneuver on these kinds of structures, but also for sensing hardware to make that force information available. So all of this work that I've been talking about has been inspired by termites. I showed this slide earlier. But this inspiration is in a very high level way. The details of what the robots are doing are very different from the details of what termites do. And there are a number of reasons for that. But one of those reasons is that there's actually relatively little known about the details of what termites do. Compared to the amount of work that's been done on ants or on bees, termites are relatively understudied. And I think there are a number of reasons for that. One is that termites are not incredibly charismatic animals. And I think our relationship with them tends to be somewhat adversarial. So here's a book that I took out of the Harvard Library at one point. I think this really sums up the usual human outlook on this. So a lot of the work that's been done on termites is actually from the perspective of pest control. So let's say you do want to study termites. If you want to study the ones that build mounds, another problem is that they're relatively inaccessible. So the mound builders don't live in North America. They don't live in Europe. To study them, you have to go to the other side of the world. We traveled to Namibia. And once you get there, the termites live in an impenetrable fortress. So what do you do? Well, you could put a hole in the mound and put in a camera on a cable and see what's going on inside. And what's going on inside is kind of a mess. Again, this is dark, but you can sort of see it. There's a huge number of termites. It's impossible to keep track of individuals over any length of time. It's a completely uncontrolled environment. Another reason that this is not an effective way to study termites is that this happens within a few minutes. Anything you put into the mound gets covered in soil. So instead, we take termites out of the mound. We bring them back to the lab. And we put them in petri dishes on lawns of soil, which is the traditional way to look at them in the lab and record what they do. So the reason that we're in Namibia in particular is our collaborator, Scott Turner, had identified this very interesting situation where we've got two closely related species of termites. So there's macrotermes Michelsni and macrotermes natalensis. And morphologically, they're almost completely indistinguishable. In fact, they're so identical when you look at them that I haven't actually bothered to show you photos of the two different species. These are both Michelsni. But they look the same. But the mounds they build are completely different. So Michelsni build these tall, impressive spires. And it's a little hard to see the scale of this because of the height of this tall grass, but that's a cow. And macrotermes natalensis build these low, spiral-less, uninteresting looking things. And so our hypothesis was we'll go, we'll study their behavior, we'll identify behavioral differences, and then we can connect the differences in their individual behavior to the differences in what they build. And we keep getting sidetracked because things that we think we know going in turn out to give us surprising trouble. The biggest example, I think, is that the central idea for many decades as to how termites coordinate their building activity is based around the idea of a cement pheromone. So one termite puts down a blob of material. It puts in this chemical. Another termite comes along. It smells the chemical. It says, okay, I'm going to put down my own blob of material here with more chemical. And so there becomes this positive feedback loop with an accumulation of material and accumulation of the pheromone. And you wind up getting pillars and so on built. But in recent years, there's been an increasing body of work that's been casting question on the role of a cement pheromone as traditionally viewed or even actually on whether such a thing as a cement pheromone is traditionally conceived exists. So the traditional view of termite construction activity is very much focused on deposition. So here's a sort of a cartoon flow chart of how termite construction activity is usually modeled. So termites are walking around not carrying anything. At some point they decide to pick up a pellet of soil. Then they walk around carrying it. At some point they put it down and they keep doing that. And the interesting thing here, the highlighted square, is how do they decide where to put it down? And that's the idea is centered around the cement pheromone. Where they get the soil from is not usually considered as interesting. There's a lot of modeling work actually that in some models they just completely ignore the question. The simulated termites are assumed to come into the area where they're working already carrying soil they brought from somewhere else. In other cases they do pick it up from within the area but more or less at random. And there isn't in general thought to be an important relationship between where they're getting the soil from and where they're putting it. So based on that view, I wish you could see this a little bit better but maybe you can kind of make it out. So we put termites into a dish and expected to see accumulations of soil in a few localized places and divots where soil had been removed more or less at random through the dish. And you may have to take my word for it more than I thought but that is very much not what we saw. We definitely saw accumulation in a few places but those places were exactly the same places where they were getting the soil from. So there'd be a... Great. So there's a period where the termites are just milling around before they start digging and then they start digging in a limited number of excavation sites where they've all apparently decided to dig at those few places in common and it's around those excavation sites that material builds up. And I really hope you can see this but if you look at a single termite digging what you wind up seeing is it goes and it gets a pellet of soil from a dig site and it moves to the side and puts it down and goes back and keeps digging. And to be unforgivably hand-wavy and anthropomorphic what this looks like a bit is imagine that you've got a shovel and you're digging a hole. So you take your first shovel full of soil and now what do you do with it? You're going to dump it out to free up the soil for the next shovel full and so as you keep digging you wind up with a pile of soil but not because you're trying to make a pile because you're trying to dig a hole. And what we see is at least consistent with that kind of view. The piles of soil are all building up on the edges of the excavation sites. So it looks as though the excavation sites are the template for where the depositions are occurring. So that suggests a different sort of basic model for how termites are behaving in these early stages of digging. They're moving back and forth between two states in one of which they're moving through the arena without manipulating soil and in the other they're in this sort of excavating mode where they repeatedly get a pellet and put it right nearby. So the termite's apparent focus on excavation made us say let's ask some questions also focus on excavation. So I will talk here only about the second of these, the underlined one. Let's suppose that you're a termite, you're walking through this arena and you come to a place where digging has been going on. You have a choice. You can either stop walking and start digging at that existing excavation site or you can ignore the site and keep walking. How do you make that decision? What influence is that decision? So we considered a bunch of different candidates that might have an impact on that. And these fell broadly into three categories. So the first was characteristics of the individual termite walking by. There's what we called excavation propensity. This is just how much time it spent digging in total in the course of the full experiment. And I'm going to wave my hands again and say this is something like termite personality or mood. Some of the termites just seem to be more interested in digging than others during that experiment. Mobility level, same idea. Some of them were more active in walking around than others. Those might have an impact on whether they joined in digging. And we also looked at these two different species thinking we might see differences. There's a second category of traits that have to do with that termite's memory. So if I come to an excavation site, was I the one who originally started digging here in the first place? Have I been digging at this site already earlier in this experiment? Have I been digging anywhere in this experiment? Maybe that's gotten me into the mood for digging. And finally we looked at characteristics of the site itself. How many other termites are currently excavating there? What's the size of the digging party? How much excavation has already taken place here? And so we used a model selection procedure to statistically look at all of these candidate features plus combinations of species with the other seven. And model selection told us which of these features were actually the important ones influencing that decision and how important they were. And we had a big pile of data, and the thing that I want to emphasize here is that we were only looking at the first 10 minutes of construction because much longer than that and termites are disappearing under the tunnels they're building, which means this overhead camera view of looking at what they're doing stops being useful. So this is really just initiation of construction. It's possible that things change at much later stages, but this is what we've studied. So the upshot of the analysis was that one strong factor influencing the termite's decision should I join in digging or should I keep walking was just its excavation propensity. That again is the personality of how much do I like digging. But just about as important was how many termites are already digging there at that moment. So for every additional termite digging there, my odds ratio of joining in digging increases by a factor of more than four. So that seemed to be a pretty strong effect. And then there were also much weaker effects, but the analysis said they should be considered from the total amount of building that's gone on at the site and also the mobility level, again sort of the termite personality. So the number of termites, and that's it, that was it. Everything else got eliminated by model selection, including species differences. We couldn't actually find a difference in this respect between the two species. So the number of termites seems like a really important factor here, which makes some sense in retrospect. Termites are social cockroaches, and there's a body of work about other cockroaches and their attention to aggregation. So the not specifically social cockroaches like to gather together in the same place. So it's maybe not surprising that the termites, the social cockroaches, would be paying attention to where others are and what they're doing and helping them decide what to do. Okay, but there's also this issue of how much digging has gone on at the site, and that could be influencing what they're doing. And so we wanted to try to disentangle these two features. So in order to control for what an individual termite might remember, to control for the size of the site that it comes across, because a larger site might be more attractive, and for any possible chemical cues that digging at a site might leave there. To control for those things, we looked at all the cases we had where the termite had never been to that site before, where at most one minute of excavation had taken place there, so the site didn't have a chance to grow too large, and excavation took place there very recently. So if there is a chemical trace, it shouldn't have had a chance to fade. And of all of those cases, in the cases where other termites were engaged in digging, about a quarter of the time, the wandering termite would stop and join in digging. And when the site was unoccupied, we never saw the termite stop and dig. So the number of termites working there really seemed to be the most important factor. So we've got then an alternative to this traditional deposition-focused flowchart. So here's another cartoony flowchart, but this is one now focused on excavations. This is going back and forth between wandering through the arena and being in this deposition mode, where the interesting thing is how do they decide whether to join in a site where others are digging, and the important factors, how many of them are currently digging at that time. So here are two flowcharts. We're computer scientists, so we can code these up in a simulation. And really just as a sanity check, let a simulated arena of termites dig according to either the deposition-focused flowchart or the excavation-focused one and look at how patterns of deposition and excavation wind up accumulating. And as you'd expect, the deposition-focused one, as I said, is a sanity check. So deposition-focused one, you get accumulations of material, but excavations are fairly randomly scattered, more of a preponderance around the edges because termites are spending more time around the edges, but there's still a scattering of excavation sites. In the excavation-focused model, you've got a limited number of shared excavations with deposition sort of lining their edges. And again, that's closer to what we see in the actual dishes. You completely can't see this here, but if I highlight the deposition excavation sites, hopefully it's a little bit clearer. So that's a qualitative comparison. You can do a more quantitative comparison as well. You look at different metrics of what's going on during this early stage of construction, especially metrics focused on excavation. And the excavation-focused model seems to capture better what the termites are doing. And it doesn't capture it perfectly. This is not meant to be a complete model, but in terms of a simplified first pass, it seems to be better capturing some of what the termites are doing better than the traditional deposition-focused model. So that's some stuff about how termites are influenced in their excavation by this factor of aggregation. Another thing we really wanted to look at was how is their building activity influenced by the shape of what they're building? As they're creating piles, that's creating a shape that they might be responding to. And that's something that we've been interested in, in particular for a long time. This is a factor that goes back to some earlier studies. This is a study that actually really influenced me when I was getting to this area. The building agents in this system are looking at local configurations of material, and they're really just looking at where materials already been added to decide where to add more. That's the reason that that was the approach we took in the termite system. The robots are looking at local configurations of material to decide where to add more, but only physical configurations and not anything like a chemical trace. There's some evidence for that being an issue with termites. So there's a study with a different species, non-mowed building, but still termites, which found that the shape of something built up in an arena seemed to be a stronger cue for termites to go and dig around it than the smell of something in the arena. And this is something that's been taken into account also in studies of other species. The shape of what they build will sometimes influence where they add more material. So we wanted to ask, how does the curvature of the surface that the termites are standing on affect where they like to spend their time? So we printed this sort of weird structure, and the idea is that every point on this surface... So we will coat this with mud and record where termites are spending their time. And every point on the surface is associated with curvature. And of course it's also associated with other measures, each point has a slope, each point has a height above the ground, and any of those things could be affecting what termites like to do. Maybe they like to climb to the highest available point. So you can disambiguate those features by putting this surface in different orientations, and that changes the inclination of each point, it changes the height of each point, but it doesn't change the curvature. And so in any given orientation you can say you have a map of what each of these quantities is at each point, and if you look at where the termites are active, it matches pretty well with curvature and not with the other two quantities. And same idea if you tilt this in different ways, the inclination map changes, the height map changes, the curvature map does not, and consistently activity maps pretty well to curvature. So this, it's a reasonably high correlation and not with either of the other two quantities. So again, this isn't a perfect correlation with curvature, this is not capturing everything that termites do, but the idea is that this is a piece of the puzzle. And so what we're trying to do is to assemble these pieces and ultimately to be able to put them together and to then predict in the longer term on longer timescales and on larger spatial scales to understand what it is that, how it is that the termites are building, how it is they wind up producing these large scale complex structures, and it may also be that the insights that we learned from that wind up being useful in creating artificial systems to be building human relevant structures for our own use. So with that I would like to thank my collaborators in this work and their funding sources, and I guess I've got a few minutes for questions.