 My name is Jordy. I'm going to talk to you about something called change set evolution in Mercurial. Mercurial, as you probably know, is like one of the alternatives to Git. The idea of a change set evolution is that you can edit commits together as a team. If you want to follow my talk, by the way, there is a URL there you can clone, htclone, he.active.org, evolve, dash talk. That's the repo for this talk, where the source code is, just in case you're curious. OK, so in order to tell you what Mercurial is doing with evolution, with change set evolution, I want to speak about the problems it's trying to solve. So there are a few problems, we think, with the way people collaborate right now. They're not very big, but they're problems we like to fix anyways. So the first problem is recovering from an upstream rewrite or a rebase. Now, in the Git man pages, it's written like this. Rebasing or any other form of writing a branch that others have based work on is a bad idea. Anyone downstream of it is forced to manually fix their history. This is from the Git rebase man page. So let's just quickly see what this problem is here. Suppose you wrote the two blue commits, someone else wrote the two red commits. Three red commits, I'm sorry. You haven't actually pushed your blue commits. You're working locally. And that other red person on the public servers decides those red commits are no longer good and turns them into the orange commits. And then what happens here is you have a bit of an annoyance here. You have like an, oh crap, moment. So now you have to go and your blue commits are no longer good either, and you have to rebase them on top of the orange commits. This case is pretty simple, but in general it can get pretty complicated. And if you go look at that very same Git man page, it's actually a pretty complicated thing, and it goes on and on and on. There's a bit more things here to go. So in general, it's a complicated problem. Even though that example was simple, in general, it's pretty complicated. So that's problem number two. Problem number two is that some commits just plain should never be rewritten because they're a historical record. And you want your historical record to be good for several reasons. You want to get blame. You want it to work correctly or HG blame. You want to be able to know who did what. You want to be able to buy sex bugs. So you want a clean historical record for serious reasons. If you rewrite history, well, then you're as bad as Stalin. If you're rewriting history, you're bad as Stalin. You should not edit people out of history. A third problem is how do you safely share your commits with other people? Now, the problem here is you want to be able to undo, if you rewrote something, you still want to be able to undo. And you also want to be able to not cause problems for other people. So there is a story that I like. Jenkins accidentally did Git push force to 150 repos on GitHub. This happened about three months ago. And they had a lot of trouble fixing this. They had to contact GitHub support. They had to contact each other, like, hey, who has the last commit? It went around for a while. Eventually, that got almost all of it correct. There were a few small repos they didn't care very much about, and they were never quite sure where the commits were on those last repos. But they got almost everything correct. And when this happens, well, Reddit makes fun of you. So again, the three problems is recovering from an upstream rewrite. Some commits should not be rewritten. And you want to safely share commits with each other. Now, the goal is that to provide a clean, consistent UI to solve these problems. In Mercurial, we are very proud of providing a clean UI. We want to make things easy for users to do. We want to make it just easy and clean. So evolve is what allows us to do this. So when you enable evolve, you get a bunch of commands. You get a bunch of more commands for editing your history. The first one is just amend, which just amends a single commit. Next we have fold, also known as a squash. It puts the commits together. We also have reorder. Actually, we don't have it yet. I'm working on this one, but it's almost there. This one just reorders commits. Previous and next are for moving on the DAG, the directed acyclic graph. So you go to the previous commit or the next commit. You just move around to make more editing. Pruning is for getting rid of commits. Uncommit is for getting rid of a commit, but keeping the changes in your working directory. Restore is, if you already edited a commit, and it's part of already it's gone, it's for getting it from the garbage. Now none of these commands are actually all that new, because they're already pretty much implemented, already mercurial, without Evolve. What Evolve does is it provides these commands, and they work in slightly different ways, which I will get to in a bit. But they're already there. Sorry. So commit is already, ht commit dash dash amend. It already exists. Fold and reorder can already be done with ht hist edit. Previous and next can already be done by updating to the parent or to the children. By the way, that notation on the right with quoting in the parents and brackets, those are refsets. Pruning can also be done with ht strip. Uncommit can be doing with ht strip dash dash keeping your changes in the working directory. And restore can be done by unbundling the strips, the strip backups. Just for the get users in the audience for reference. Oh, sorry, one more thing. And these are very scriptable. The nice thing about these, I mean, one thing we don't like about hist edit is that you have to create a plan file and the plan files have to say, order, pick this commit, reorder this one, delete that one, and then edit that one. We don't like the plan file very much. We like these commits because we don't want to be able to script these things very easily. People actually build tools on top of Mercurial and they like our command line because it's very easy to script it. And for the get people in the audience, I'm just going to give a quick translation for get. A mend is also called get command dash dash amend. Fold and reorder can be done with get rebase dash i. Previous is a get checkout head hat, which means the one before the one you're standing on. As far as I know and correct me if I'm wrong, there's nothing simple for next in get. You have to go digging through the ref log or the log to figure out which commit you want and then go to it. You check it out. Pruning is a get reset hard and an uncommit is a get reset soft. If I get this right, right. Is it mixed? I think mixed is the default. There's soft, mixed, keep, merge. I don't get them. I think merge is the default. OK, and for restore, there's no single command that restores. You have to go dig through the ref log, figure out which one you want, and then move your refs around until they get to where you want to be. And correct me if I'm wrong. I don't think there's a simple way to just say this one without actually looking at the ref log. I guess you can parse it. Now there's also a shiny new command in hd evolve that does magic. It's called hd evolve. Now what this command does is it basically fixes any problems. If someone rebased underneath you, it actually knows where to go and puts you in the right location. And this is what solves problem number one. Now what about some things that are set in stone? That's pep8, pep8 send stone. So for those things that are set in stone, how does evolve handle these problems? We have something called phases. So certain commits are public and immutable. So once a commit goes to a public server, it can no longer be edited. Some commits are draft. When you're sharing it between friends, these are drafts. And there's an understanding that these commits can still change around. So let's see how this works. How does that look? OK, so first I'm going to clone the code of conduct from the official Python repository. I'm going to go into the code of conduct. Let's see what we have here. There's only two commits, very simple repository. There are three files, the code of conduct, the license, and the read mean. Let's set up two bookmarks, one for upstream and one for me. The dirty bookmark is going to follow me along. Now the code of conduct is not word wrapped, so I'm going to word wrap it. This fold is a GNU fold command unrelated to HG fold. This is just word wrapping, the code of conduct. I'm going to get rid of the trailing white space. And I'm going to commit this word wrapping. And now the code of conduct is for Python, but we're going to edit it, and we're going to call it Cobra. And we're going to change Python to something cooler, which is Cobra. So there we go. And now we just added two commits. You'll see that my bookmark is up there. Majority, there's a bookmark upstream down here. So these are the commits originally that were there. I added two more. The actual changes that I made are the Cobra communities made up of members from around the globe with a diverse blah, blah, blah. That's just the change you can see it. Now, but the Cobra is not really a community. It's a command. The Cobra command is not a community. I just added another commit there. Just to show you, now we have three commits. And there's a two there at the bottom still. But I made a mistake. Command should be capitalized. So I'm going to make that. And I'm not actually going to make a new commit here. I'm just going to amend the last one I made. That's all. I made a change, and I amended. And now that commit, I still have the three commits. But now notice something interesting here. If you look at the revision numbers, which are just local, remember, I have zero, one, two, three, and then it skips to six after that amend. Four and five are missing. And we'll see in a bit where those commits went. Now I'm going to go to the previous commit. So just to show you, this little at shows me where I'm standing. I'm standing right now on commit number three, because I went to the previous one from the six. And I'm going to make another change here. I actually forgot to edit the readme here. So instead of the Python Software Foundation, it's the co-recommender HQ that wrote this document. And there's also the abbreviation, which is now the CCHQ. Just to show you, this code of conduct was approved by members of the co-recommender HQ during the April 2013 vote. And they must be approved by the CCHQ's board of directors. And now I'm going to amend this one. I'm amending here a commit that was back in history. And this told me that I have a new unstable change set. Now because I'm amending a commit in history, you'll see that this commit, three, got amended into commit number eight over here. And it's being marked here on the graph with a little x, like as it's dead. That commit is no longer good. But now this gives me an unstable change set over here, change it number six. So in order to solve this problem, all I do is hg-evolve. Boom, it fixes it. It knows that six and eight have to be fixed around. It moves things around. Everything's magic. And everything gets settled out like that. Yeah, yeah, yeah. Yes, you can still update everything this next. Now let's see what happens if we try to prune upstream. Now prune upstream, that was a public thing. I grabbed it from the public repository. What happens if I try to get rid of a commit? I don't want whatever Brian Curtin did. It says, oops, I cannot do it. Cannot abort immutable change set. It's immutable because it came from a public repository. OK, let's go back to my slides here. OK, so just I showed you how this handles problems one and two. Problem one is about recovering from upstream rebase. That just hg-evolve. It's magic. Actually, there's a lot of logic behind it, but for now it's magic. And then there is problem number two about immutable commits. But what about problem number three about sharing safely? This has all been local so far, but once you start pushing and pulling, then it gets more messy, right? And what was going on with those revision numbers skipping around? So what happens here is that what evolve is doing is it's creating a meta history on top of the history. It's creating another history that says it's only adding ever-adding things to it. It's never actually deleting anything at all. It's just adding more stuff to the history. So what happens is that the pre-edited commits, before you rewrote them, they still stick around. But they're only marked as obsolete, and they're only shared when they have to be shared. And commits may be obsolete by another command that overwrites them, like a commit knows, OK, I'm obsolete, but that commit over there is what replaces me. And fold and squash, for example, and amend are commands that do this overwriting. Or commits may just be plain obsolete and have nothing to replace them. They just say, I'm obsolete, but I have no successors. And evolve those how to propagate all this across clones. So I'm going to try to show you now with another little cute diagram that I made. So suppose that there are two users, lefty and righty, and they're having a private server. This server is marked as private, so it's fine to add stuff in it. And lefty here, first clones, gets the exact same thing as what's in the server, of course. And lefty doesn't like the green commit. If anyone's colorblind, green is that one. Lefty doesn't like the green commit and turns it into the yellow commit with an amend. And then doesn't want to actually keep the blue commit around, doesn't actually want to rebase anything, says the yellow commit is good enough, I just get rid of blue. And then lefty pushes these changes back into the central repo. Now, the central repo gets an information. It gets information that green and blue are now obsolete. And yellow is a successor of green. Now, when righty, over here, clones, righty only gets the non-obsolete commits. If righty really wanted to, there's also a poll, everything including the rewritten ones. They're still in the server. But most of the time you don't care about that, you just care about the final result as you're rewriting. So obsolete commits gradually fade away from history as they're being shared around, because they don't actually get pushed or pulled unless you really want them. The final result looks like this. So let's see how Evolve does this. Okay, so back here, remember that I cloned as commit from, I mean this repository from the Python repositories. I'm gonna clone it again, but this time I'm gonna put it in a code of conduct other. I hope everyone can see that. I'm gonna put it back up there. So a new clone from upstream. I'm gonna push, right now I'm standing, I'm standing right now on the original one that I cloned. I'm gonna push my two bookmarks, Jordy and upstream, into the other one. That added the three change sets only, and my two bookmarks that I just created. And now here is that other option that Evolve adds to mercurial, dash dash hidden. So this is where all those commits went. So now you see that we have them all here. From zero to one, and then all those Xs there on the right are where I was overriding things. There was even a temporary amend commit that wasn't even made explicit. But this is the commit that edits this commit. So it's all saved there. But in the end, I only get these that are not absolutely. And now let's look at the other repository. And when we look at the other repository, there are actually no hidden commits here. When I pushed from the original repository into the second repository, the hidden commits did not get pushed. Now, notice that here on the second repository I was standing on the upstream bookmark right here. Because when I push to it, it doesn't change where I'm standing. It doesn't change the working directory when I'm standing, it just adds more history to it. So I'm just gonna move to the top. And what I'm gonna do now, just as an example, I'm gonna fold the last two edits where I redefined the Python code of conduct to the co-recommander code of conduct. The syntax should look familiar from Git. This is, I'm folding the Geordi bookmark and the predecessor of the Geordi bookmark into a single commit. There it is, it's folded. Now there's only one commit. So you see that, now I only have two commits ahead of upstream. Again, we see that there is a skipping here under revision numbers. There's more hidden commits that we just don't care about at the moment. And I'm gonna actually show you what the actual change is. The P is for showing the patch, the V is for verbose. And co-recommand is made up of members from around the globe with a diverse set of skills, et cetera, et cetera, et cetera. And this actually includes the changes to the code of conduct.rst and the changes to the readme that I made as well, which I folded together. Now let's go back to the original one and let's pull. I'm pulling from the one that I just folded them together. Now after I pull, I mean it says the usual things, but the interesting part that I wanna show you is that the bottom says here, you're working directory parent as obsolete. You pulled something that changed history around you. Well, we get out of this mess the same as before. Well, let me show you first. I'm standing right here. Now nine and eight just got folded into 10. So it's telling me you're standing on an obsolete commit. I just do HT Evolve and Evolve figures out that all it has to do is just update. You just have to update to the correct one. If I look again, I'm exactly like the other one. And you see that again, I'm here at two, here's at 10, and there's a bunch of hidden rewritten history underneath me. Okay, so just final summary again. The three problems are recovering from upstream rewrite. This is solved by the HG Evolve command. Some commits are restored record and should not be rewritten. That's solved by the phases. Evolve knows which phases are public and which are draft. You wanna be able to safely share commits across repos and that's done with obsolete since markers. That's all.