 The title of the paper is Finding Life in the Shadows. That's meant to be sort of a little stupid pun. And the idea is that how many people are familiar with the Packard-Bedot way of assessing evolution? Yeah. So the idea is, how do you know if evolution is really happening? Well, the way you could know is if you could do the exact same model that you were doing with evolution, but just turn off evolution and then see if there was somehow systematic differences that would appear between the ones with evolution. Turn it along as long as you want to. And tell you what it is. And the model with the evolution turned off is called the shadow. And in this particular work, oddly enough, for being speaking at the open-ended evolution workshop, there's going to be no evolution here at all. In fact, what we're doing is trying to get a running start toward evolution. And that's why the ideas we're focusing on, what is morally the shadow? That's the idea, in addition to the rainbow bridge looking kind of like a shadow with the overcast. I historically do not know when to shut up. So the strategy is start at the end and then work backwards. And however much time there is, I just stop talking when the time runs out. So I'm most interested in soft A-life, artificial life systems that are somehow implemented inside computers using mostly software type stuff. There's certainly the chemical stuff. I think it's super great. The hardware stuff, I think it's super great also. But I don't want to do either of those. I can hack. And I've got to go with the strength that I have. So the claim is, the conclusion, the takeaway is, if an OEE researcher, open-ended evolution in soft, artificial life, we should get over the habit of implementing creatures. We should get over the habit of saying, OK, what is a creature? Well, a creature is a class. It has a genome, which is also a class, which has data, which is an array, and so on. Because every time we do that, we make life that much harder for us down the road to try to see something open-ended in it. Because we privileged the representation that we're coming in on. Essentially, we've made laws of physics that know where an individual ends, where a genome starts and ends. Exactly how big it is, blah, blah, blah, blah, whatever it is. So from one point of view, this is just an argument for artificial chemistry. This is just an argument saying, we should begin one level below so that it's not even entirely clear where the creature is. And that's why it's an actual non-trivial problem to find the life in the shadow. So what I want to talk about in this talk is following up on some flaming I did in the first OEE workshop in York that kind of accidentally sort of delivered on that research program in a very small way, sort of without even really trying. So the plan for the talk is to get a running start, go back to 2014 in York. I'm going to skip the definite scale a little bit and try to immediately do a demo to show you where we were three or four years ago, where we are now, what the key differences are. And then go through this actual algorithm. I don't usually give talks anymore that have algorithms in them with steps and things, because I'm such a visionary. So I actually kind of enjoy having something incredibly concrete to say. It's even got intersections and union signs and everything. So I actually want to go through it enough to understand it. It's not that hard to understand. But again, it's an example of coming at it without assuming we know what the living creatures are, without assuming we know what the evolutionary creatures would be. And that's basically it. So that's the plan. I'm good at waving my hands. This was the talk that's on the web someplace on YouTube. The slide behind me is the one that I want to focus on. And here is a repainted version of it. The challenge I put before us was, if we didn't know, here is some volume of space and time, a million light years across, a billion years. And you can search anywhere in it. You have a lens that you can say, I want to look at a one meter resolution over here, and so forth. How could you know? How could you do some computations, some statistical process that would end up saying, oh, oh, life, life. And for the more evolution. And the idea was that you go through this, you take whatever a thing you like, you gather images and do autocorrelations between them, and look for stuff that's sort of almost perfectly repeated. And those would be candidates for living creatures. And then you could move through time and trace those candidate living creatures to see how they moved in space. And perhaps some of them would disappear, some of them who knows. The idea was come up with an automatic method, a statistical method for doing something like that. Okay? If we then did that, then we would have some justification for saying, no, no, no, it's not that I programmed this in there. There's some dynamics happening for whatever reason dynamics happen. And now the statistics say, I have good reason to say that's life, that's a species, that's evolution, whatever. Okay? So in particular today, I want to focus on step two. So we're going to cheat on step one, and step one is a big one, which is actually deciding candidate creatures. And we're going to take advantage of special knowledge of the problem that we know to finesse that. And we're going to focus on, okay, but even if we have a reason for saying, I think this thing is kind of a creature, actually figuring out what happens over time, turns out to be non-trivial. And that makes me excited. I think that's cool. All right? So the idea is to figure out how to do step two in one specific case. All right. So now I want to do show a little video from the past and the future, and then we'll come back around. In 2015, this is all part of a long-term project that we've been doing to build new, indefinitely scalable hardware that you can just, It looks so young. Ah! Uh-oh. Okay, whoops. Hang on. Give me one sec. No. No. No. No. No. This will be fun. So what you see here is a mob. This is a thing that grew from a single starting atom that blows up into a cluster, and it's following a rule where they're moving sort of randomly, but the admission says, I can move to an empty spot. If the empty spot is closer to another one of my guys than I am. And it works pretty well. It works pretty well, but you saw what happened. They had additional background saying, I want to head south, I want to head west, whatever it is. And then they gossiped among each other, saying, I think we're supposed to head south. Do you think we're supposed to head south? And so on. And as long as they stayed close enough to each other that they could gossip about which direction we should go, it worked all right. But just because of randomness, little satellite fissioners could spew off. And once they got far enough away, they would all follow each other and say, oh, you know, we're the greatest mob, even though there's only three of them. And the rest of the mob was heading in the other direction. And once they fell out of communication range, the mob would disintegrate. So really what we wanted was some way to keep them corral closer in, number one, so they'd stay in communication range. And number two, so that other mobs couldn't come in and mix with them. Okay. So let me see if I can get the second one to go in here. So what we have here, the green stuff in the middle is a mob. It's a three years newer evolved mob, so it's cooler. But fundamentally it's the same thing. But what we have that's new around it is a cell membrane, a two layer, inner layer, outer layer. That's obeying simple geometric rules. I say simple, meaning it only took me 25 years to figure them out. And I only had to invent one new programming language in order to do it, actually tune programming languages. And this works really well. This works surprisingly well. And it's all a little backwards because I'm gonna be presenting this cell at the lightning talk on Friday for those six people who are still here for the last talk of the conference. But the point of the talk today is we're just gonna use this, all right? So I have a video of a longer version of this. In the stuff that we're, I'm talking about on Friday, it's about the cell membrane itself, how it works, what it does, how it actually heals in certain cases, the language that we built to do it, and so forth. Again, once you actually get to it, it looks really obvious. And I love that. Well, again, the amount of suffering behind these things one never really appreciates. All right, so this is a longer run of four cells that have been modified to try to reproduce. Every once in a while, they decide to double the amount of content they have, half the content go one way, the other half of the content go the other way, and then you figure, hey, reproduction. And it even kind of works. It doesn't work at all like I imagined it would work, the stately procession of paramecium, bearing, but it works kind of, okay. So now the problem is, the problem is, once these things have been reproducing for a while, they fill up space. That's what life does. And once it does that, it's very non-trivial to look from the outside and be absolutely sure where this guy ends and the next guy begins. It's, we can take advantage of the fact that there are these invariants that the membrane has as long as things aren't damaged. But even then, the membrane can split. Fission and fusion are both possible with this membrane, and it just happens automatically under the correct geometric conditions. It's not like you ask to have a kid. It's just, if you're in that position, that's what happens, or vice versa, okay. So how are we gonna know the life is? How are we gonna know who's the parent? Who's the kid? Where's the offspring? And that's what we figured out. So here's how it works. It's a five-step process. What we're gonna do is, we're gonna start from completely outside the system, essentially knowing nothing, except we're cheating, observing the system exactly the way we were just observing it in this demo and gather data to infer where the creatures are and what's happening, all right. So step one is gathering observations. And here are some observations. They're now familiar to you because you've seen the interface for the simulator. And the simulator will record screenshots automatically at whatever rate you want it. That is our ground truth, a series of ping files. And we analyze the ping files with God's honest pearl script. And we go through them, and on the basis of color, we say, oh, that's an outer membrane. Oh, that's an inner membrane. Oh, that's an empty spot. This is how hands-off we are. And it's the point. If we were actually coming upon some alien universe and trying to study it, this is how we would study it. It's a big data problem. It's not a programming problem from the inside. We'll see in just a sec. Yes, and that's where we're cheating, that we know some rules about how the membrane wants to work. So we observe the system as ping files. Step one is on the basis of color to identify the things that we care about. Outer membrane, inner membrane, empty squares, content, anything else. And we get a picture like that. It's got O's, I's, stars, and so forth. And once we've done that, we can do a raster scan of the whole thing and use our topological knowledge that an outer membrane should always have an inner membrane next to it. Anything that's next to an inner membrane must be inside the cell. We scan through it. Whenever we find an inner membrane, we do a flood search to fill out that particular cell and we label it with an arbitrary tag and we keep on going until everything's tagged. So topological inference. We went from the atomic level and now we have found inside and outside for each of the cells that are in this image, tagged with some arbitrary tag based on the raster scan order where it found it, okay? Now, the next step is compare this topological recovery to the previous one because we're trying to infer lifelines through time. So we have two of these images representing then and now. And in fact, in the particular images that we picked, something happened. We have a different number of cells and so on, okay? So what we do is, and this is where it actually almost turns into computer science, we figure out the volume of each cell and then we take all pairwise intersections of the cells to see how much they overlap. Say again, in both directions, we say, for example, how much does NC overlap with TD? How much does TC overlap with NC, okay? And we get a percentage relative to the guy we're comparing from. We sort them in order from biggest overlap to littlest and we build a graph saying this guy probably came from that guy because they have a huge overlap between then and now. And it turns out, so in this particular example, here it is, there's eight edges that have a variety of overlaps. You see some of them are from then to now, T to N, some of them are from now to then. They can go in either direction. Even though the set intersection is symmetric, the overlap is relative to the source guy. So they're not necessarily the same score and we build this graph. We put in the first three graphs, the first three edges, we still have things that are undetermined. Eventually we keep filling in other edges and we get to a quiescent condition. Everybody has an incoming edge, at least one, an outgoing edge, or we ran out of edges, which can happen when somebody dies or in the other rare case, when God shows up and creates somebody out of thin air, then yes, we have a guy that has no overlap. So we can apply these rules. If the in degree and if a T guy and an N guy are pointing at each other and the in degree is this and so on and so forth, then we declare that a live event has occurred. This guy in previous time is associated with this guy in the current time and so on. Five possible events. Live, split, merge, that's what we saw here. It was a merge event. Genesis and death, okay? And then as the last step, we take our evolutionary steps and we build a graph, build a evolutionary tree. This guy merged with that guy and so on. And from there, we have these giant plots, I'm gonna be out of time so I'm not gonna go find it for you, but it's in the paper, it's the last page of the paper. And it turns out that you can actually tell quite a bit about what is going on in this kind of situation by looking exactly at the history of these things. The stuff that would have been directly programmed, split it, selection, all of these things happen as a byproduct of the lower level stuff. I have to stop there because I'm out of time. Thank you for your attention. Thank you, David. Questions? Somebody else who just likes a lot of weird screenshots of things and tries to figure out what's going on, I think this is really interesting. Do you think you would learn anything interesting by looking at image impressions statistics or not really? In this case, we kind of knew what we were going for. I mean, these were ping files, but the main point is the simulator uses stylistic colors for inter-outer and so forth. So we were just looking for literal pixel matches. I absolutely have thought about that, that stuff that is compressible in various 2D kind of ways should say something, but we haven't needed it. Cool, thank you. Iraqi, yeah. Thank you, David. I'm so encouraged to see this research. Yeah, so you are now reconstructing the images, but do you also plan to detect interactions between, you know, organisms? You mean sort of day in the life interactions? We're getting fusions and fusions. Those come out. Or maybe like, well, maybe not happening in this system, but technically, if you have cells chasing each other or something, that kind of the interaction could also be detectable. In fact, someone I know, you, are doing some of that in the same approach with analyzing pictures and I thought it was totally cool. Yeah, at the moment, these guys not only have no evolution, they have no senses, nothing like that. So we're absolutely just at the beginning. Alastair, could you get the set off? We can take another question. OK, yeah. David, I wanted to ask, you compared the pain files that were right next to each other in time, but if you get further apart, then you're not able to have that overlap. Yeah, absolutely. I'm sorry for cutting off. The slide that I just had up there on the end, they really weren't right next to each other in time. And we tried it at several different granularities of time because it's an expensive analysis if you actually do every adjacent thing. And the trick is ground truth changes. Sometimes this now looks like this guy had twins, where in fact they came out of slightly different times and so forth. And that's as it should be. Nice. I also wanted to ask if you think about doing the same thing in other kinds of spaces, 3D or a strange graphical state of graphs and connectivity. Yeah, I mean, we haven't done any of it like that. We feel like the door is open. I mean, it's easy. It was difficult to avoid having evolution happen. I mean, even in the most degenerate sense of how many of the content go into one side and how many of the content go into the other side, that can have differential consequences to how fast the cells move. Whether it was exactly heritable in the next generation was unclear. But that was the beauty. We're excited to go looking for this stuff now. Is it a quick one? Yeah. OK, one more quick. Where's the guy running? Yes, I was in clear whether you were looking at a cellular automata model or something else like squirm 3. Yeah, this is the movable feast machine, which to a first approximation is a asynchronous cellular automata. But it's entire stuff. It's stuff that we can turn into hardware in indefinitely scalable size. Thank you very much. Thank you.