 So, quick about me, I'm Mohit Hathay on Twitter, Pastafari on GitHub, ask me what that means later. I work at HelpShift where we use closure in production. If you're interested in writing some closure, come talk to us. We have a booth back there. And when I'm not working, I like to spend time out in the great outdoors doing fun stuff. All right, so that's enough about me. Brief outline of what we're going to be talking about is, we're going to be building from basics, right? So, first I'm going to talk about the motivations. Why am I here? Why am I talking about this? Some definitions in what. And then we'll take some data structures that we'll study a little bit in detail. And finally, some takeaways, learnings. All right. So, let's start with the why. So, I went and googled for a day in the life of a programmer. And this is on Wikipedia, right? So, what do we do every day? Most of us here are programmers, I'm assuming. And we get into work. We set up our fancy monitor systems, MacBooks, everything. But what is it that we actually do? We develop features. And then we fix bugs. And then we develop features. And then we fix bugs. If those of you who attended Venkat's keynote will know that this is pretty much an endless cycle, right? So, we are basically sitting at a layer that is built on a tower of abstraction, right? So, we are essentially, if you look at that tower, we're writing programs that make use of certain data structures, right? And there is this abstraction layer, which is the abstract data type on which we depend. But we don't really care or know about the implementations of these data structures, right? How does the language implement these data structures? I, for example, am curious about these things, but I've never actually bothered to read the code or to understand it. So, I, for me, at least, it was magic. It really was. Like, how does Clojure do a quick lookup from a map? How does Clojure give me a quick random access vector? No idea. But it does. I know that, right? So, to steal from Arthur Clark, any sufficiently advanced data structure is indistinguishable from magic. At least it was for me. So, my primary motivation for doing this talk was to understand this a little better myself. And share some of my learning, some of my aha moments with you guys. I'm by no means an expert on this. Debashish here has written some amazing blog posts on this topic, so I'm probably going to deflect all the hard questions to him if you ask me any. And there's a lot of people who've written amazing stuff out there. But, anyway, I'm going to share some of what I learned along the way. So, that was the why. That was the motivation, right? And I like to paraphrase it by saying that it's like driving a car, but when it breaks down, not knowing how to repair it, because you have no idea how the engine works, right? And it's kind of similar. Okay, next up, what? So, data structures is a concretion. It's an implementation, right? But it's hidden from us by data abstractions, right? In that tower of abstraction that we looked at, we depend on interfaces, we depend on signatures, but we don't really depend on implementations, right? So, how do we talk about data abstraction, right? So, there is this formal way to talk about it, what is an abstract data type, right? So, it's a name, it's a bunch of operations, and it's a bunch of invariants that data type has to adhere to. For example, a stack has the operations push and pop, and the invariant is the thing that goes in last, is the thing that comes out first, right? Fair enough. As an A side, I realized that most of the languages that we use have very good ways to specify the contract at the operation level, but not at the invariant level, right? So, if you write an interface in Java or a signature in ML, you can say that anything that claims to be a stack must implement a push and a pop, but I don't know how to claim that it must implement last in first out. That's just something that I learned, that was Naha moment for me. In this talk, I want you to change perspective. This is a picture of Earthrise. This is taken by the astronauts of Apollo 8. This is what Earth looks like from the moon. So, we're changing perspectives, and we're thinking like a data structure designer today, right? We are going one level below in that tower of abstraction. We are no longer data structure consumers, we are data structure designers, right? Okay. So, let's look at what our challenge is, and I call it the ADT challenge. So, the most basic level is that given an ADT, I need to give a correct implementation of all operations. Fair enough? That's the first challenge. And you could come right back at me and say that any good algorithms or data structures textbook will give me the answer to that, right? And I'll just go and implement it. The problem to quote Okasaki is that all these books give you pseudo code that is imperative in nature. And we're going to talk about the difference between imperative and functional. But this is like Henry Ford had said way back when that customers can buy a Ford of any color as long as it's black, right? And this is exactly what your data structures textbooks are telling you can code in any language as long as it is imperative, right? And that's a problem. So, we don't want to be imperative. We want to be functional. Why do we want to be functional, right? I guess this audience is self-selected to answer that question. So, won't spend too much time there. But basically these two things, right? We care about immutability and we care about persistence. And why do we care about them? Because it makes it easier to reason about code, right? So, I'll spend a couple of minutes and talk about what this really implies as an A side. So, here's a snippet. Who wants to volunteer and tell me what it's going to spit out? It's going to spit out five, right? And that's because the value five is immutable. You can add things to it and it'll give you new numbers back. The number five itself is not going to change. Here's another snippet. Exactly the same shape. Exactly the same shape, right? It's doing step one, assign or declare something and then do something to it and then print it. So, what's the output going to be now? One, two, three, right? So, one, two, three is not immutable, right? And why this distinction? Because that's just how it's been. There's no good reason to say that one, two cannot be treated as a value, right? Why is that a problem? Look at this piece of code and tell me what X is going to be now. If I printed X, can you say with any confidence what's going to come out? You cannot. An equivalent piece of code in closure, the let just creates a local variable. Think of it like that. X is a vector. You pass it down to some function and then you yield it back. I can guarantee you that it's always going to be one, two. So it is clearly easier to reason about things in the presence of immutability. Especially if you throw in things like concurrency, multithreading, what have you, then you have to get into details of I'm going to take a log, only I can write, only I can read So we're all agreed that immutability and persistence is good. Functional is good. So our ADT challenge level two. Give a correct and functional implementation of all the operations. But that's not enough for us, right? We care about performance. We want our functional awesome code to actually be deployed in production and we want people to be using it. So it's not good enough for us to have slow implementation, right? So we want performance. So what's the sweet spot? You want it to be correct. You want it to be functional and you want it to be performant. And X marks the spot, right? That's ADT challenge level three, like that's the epic level ADT challenge that we're facing. Along comes someone called Chris Okasaki, right? So this is pretty much paraphrasing the first chapter of his book and this really came from his PhD thesis at the time. So he was a little frustrated by the lack of literature on how to do things in a functional way in the world of data structures, right? He basically talks about how all these data structure books give you imperative pseudocode. And so his answer was that I'll just write my thesis about it, which he later converted into a book, which is really, really awesome but extremely dense read. So challenge accepted. All right? So now we've talked about what. What are functional data structures? They are data... They are implementations of these ADTs in a functional style, right? Which means persistence immutability is required. You are not allowed to do destructive updates, right? That's a constraint on you. So how do you achieve this? Let's start with the simplest, like most humble little data structure, which is the pair. Okay? What is a pair? A pair is just two things. What can you do? You can construct a pair with cons. You can get the first thing out with head and you can get the second thing out with tail. Here I've defined X to be the cons of three on minus two, which yields the thing that... I'm gonna try this light thing. Yeah. So cons of three on minus two yields that pair. If I do head of X, what am I gonna get out? Three, right? Fairly straightforward. If I do tail of X, what am I gonna get out? Minus two. All right, pop quiz. If I need to implement this thing, how do I do it? Sure. That's one way, but we'll get to that later. There's a really simple answer, right? So everyone here is talking about some kind of memory allocation, arrays, linked lists, so on and so forth. You can actually implement a pair using pure functions. You do not need to worry about the thing called a computer. That's purely incidental. Cons the magnificent with a hat-tip to the little schema. This is a definition of cons. It takes X and Y, and it returns a function that takes another function and applies it to X and Y. Okay? If this is the definition of cons, this is a definition of head, and this is a definition of tail. This is a direct translation of some lambda calculus. I'm not going to go into details of why it works. I'll leave it as an exercise to the reader. Again, inspired by Okazaki because he leaves really tough things to the reader. But it's fairly simple to do pure substitution, like just say cons of three minus two, the things that I showed you on the previous slide, and you put them into this, and you'll see that head gives you three and tail gives you minus two, right? This is from SICP, yes. This is chapter two of SICP, and it teaches you how to write cons using pure functions. So this was a mind-blowing aha moment for me. Why? Because I just created a data abstraction out of functions, right? So code is data. Data is code. It all makes sense suddenly. Here I have represented a data structure with pure functions, right? So it was a mind-blowing moment. All right. Now, if I recursively apply cons, what do I get? I get this, right? I've just recursively applied cons. So the last thing is cons of one on nil, which is just a simple consel. Then you cons two onto that, then you cons three onto that, and you're building this recursive data structure, and you've just gone from a pair to a list, right? And now, if you have a list, could you implement a set using a list as the primitive, right? You could just put the elements one by one into the list, and then when you want to do contains, you can just go through the list and see whether it contains it or not, and you get a set. You could do map using a list by putting key-value pairs in the list. And so it goes. And so we've built this tower of abstraction out of nothing but pure functions. And this, again, is a mind-blowing thing for me. Like, the computer is purely incidental. Give me a language with first-class functions, and I will give you data structures. That's what they are saying, right? Which is pretty amazing. So what does a set do? A set basically does membership, or we're talking about finite sets here. So you can put things into a set, and then you can ask the set whether it's there or not, and then you can take things out in union and defend all those things. But at its very root, when you're putting something in, you can just cons it into the list. When you're checking to see whether it's there, you can just iterate over the list and see whether it's there or not. And in the cons, you can first do a check to look up, then do a cons if it's not there. So you avoid duplicates, basically. No, so I showed you that you can do list by doing recursive cons, right? So if I gave you a list as a structure, then consing on to a list would be consing on to head and tail, right? So it would just work in the same way that. Mm-hmm. So one will also be, so one will be duplicated in that list. Yeah, so I am not talking about performance right now. I did not say that this has passed the ADT challenge. That's a very valid point. I did not say that this is performance. I just said that it blew my mind because I just need functions and I can go all the way up. That was the point of this section. And that's why it's called cons the magnificent. Here's a functional update, right? So we talked briefly about immutability and persistence. But the cons example that I showed you, right? How do we basically get immutability and persistence using just these primitives? So here's a real working closure code. It's called myUpdate. The first argument is a collection. Second argument is a index. Third argument is a value. So what I'm saying is, given a list and an index n, I want to put the value v at that index. Fair enough. This is an update like a of n equal to v. That's what this is. But I cannot modify the original list given to me. Fair. I cannot modify the original list given to me. All right, so what does the implementation look like? So the first if is if n is 0, means I have reached the place where I want to do the update. Then I'm saying I'll cons the value v onto the rest of the list. All right, this is the recursion breaking case, right? The second else in closure, basically the first thing following the if is the then part and the next thing is the else part, all right? So if there is something at the head of the list and at this point we know that n is not 0, right? We know that n is not 0. So we want to move one position down and put this value at a later point, right? So this is a recursion step. So we are checking is there something in the list or is it empty? If it's empty and n is not 0, then the index is out of bounds. Then I cannot update that position. Fair, that's the else part. If it is not, then what I'm saying is keep the first one as it is and cons it onto my update called with the rest of the list and n decremented, right? So we are now solving a sub problem. We are solving a smaller problem. So basically inserting something in a list at the nth position is equivalent to inserting something in the tail in the n minus 1th position. Fair enough, because I took one thing, I constate onto the result of that. So this is a recursive implementation and what this is going to yield is something like this, right? So if I did update with the list on top, position 1 and value 4, in the first call I'm going to recurse. So I'm going to const 3 onto update of the rest. Then in the second call n is going to become 0 because I've reached the place where I want to update. So I'm going to const v onto the tail and the tail is the same as the tail of the original list. Is this clear? Everyone with me so far? So the point here is I don't have to copy the whole list. Like a Java implementation of immutable list would perhaps have said make a deep copy and then go and do a mutable update. That would have been perfectly valid. It would be immutable, but it would copy a lot of stuff. I don't need to do that here. I can do something called structural sharing. Fair? All right. So let's move on to something that is sequential but has a different invariant, which is a queue. I guess everyone is aware of the operations that a queue supports, right? So I've just given them more common names. Okasaki calls this cons or snock head and tail because snock is the opposite of cons. And in a queue, you always insert at the end and take out from the front. So Okasaki calls it snock. I decided to call it nq peak and dq, right? This is the ADT. The invariant is that first thing that goes in should be the first thing that comes out. Everyone with me so far? So now we're moving to a different language. This is SML, standard ML. And this is a signature. For those of you who have done a little bit of Java, it's like an interface. You're saying that anything that claims to be a queue must implement these things. So it must have this abstract data type called alpha queue. It must have an empty which returns a queue. It must have a isMT which takes a queue and returns a boolean. nq takes a queue and an alpha and returns another queue. Peak takes a queue and returns an alpha. dq takes a queue and returns a queue. What is alpha here? Alpha is a type parameter. So if you attended some of the talks on Haskell and so forth, this queue basically takes homogeneous types. Everything in the queue has to be the same type. They have to be a general type called alpha. So the compiler will only check after you put one thing in that the next thing you put in is not going to be different. How do you do this? Ideas. It's similar to what I said before. If you already have a certain data structure, can you build a new data structure using it? So let's assume that we already have lists and this idea is called structural decomposition that you use an existing data structure and you create newer and bigger data structures with it. So if you already have lists, you can use a list to implement a queue. How do you do that? So here's some code. I'll quickly walk through it. I don't want to spend too much time. So first of all, I declared the type alpha queue to be a list of alpha. And list is a built-in type in ml, which just gives me back a list. Then empty just returns the empty list. The two square brackets are empty list because an empty list is an empty queue. Is empty just asks, this is a function that returns true if the thing is empty, otherwise it's false. An enqueue here would be a list. Okay, so this is the first one that we need to really think about. What does enqueue mean? So this is a feature in ml called pattern matching. And again, if those of you attended any of the Haskell talks yesterday, you'll know what I'm talking about. Enqueue is defined in terms of patterns of input. So what are the inputs to enqueue? The signature said that give me a queue and an element, and I should give you a queue back. So what are the possible queues that I can get in? The first possibility is that it's empty. So this pattern is matching the empty queue and an element E, and I'm returning the single element list E back. This is valid. Otherwise, I already have a list in there which contains a head and const on to a tail. So this double colon is the const operator in ml. If anyone has written Scala, they will know what I'm talking about. This is how you can write cons in Scala as well. So what you're doing is you're taking the X out and you're enqueuing the element into the tail, which is a recursive call. Why? Because you want to put it at the end. It is first in, first out. So anything new that comes in should go to the end of the list. Does this make sense? All right. Peek is fairly simple. Peek on an empty list will raise empty. There's nothing in there. Peek on X, const on to X is just X. Right? Dequeue equally simple. Dequeue of an empty list will raise empty. Dequeue of X on X gives X is back. Now, I found this mind-blowing when I first saw this. Yeah. So that depends on how you define the signature. That's what you want to use to get the entity. And Dequeue, that's the default, but that's not what my signature said. So I'm trying to say that you have a peek. Just use the peek when you want to get the first thing. So what's the signature of a immutable Dequeue? That is what peek is, right? That's what I'm trying to say. Peek is that. If you think in terms of signatures, if you think in terms of types, your peek is giving you access to the head and you want to, like, remove that thing and change the queue, you want to dequeue, and then you get a new queue that you can pass around. Yeah, I mean, the point... But the point I was trying to make is the signature you gave me for dequeue is exactly the signature for peek, right? So it's already there. That operation is there. Okay. Is this correct? Yes. Is this functional? Yes. Is it performant? No. Every insert is going all the way to the back. When the queue grows, your enqueue is going to get slower and slower. Fine? So how do we do better? We be clever and we use two lists, right? And, again, people who remember their engineering data structures class will remember that you can use two lists, one at the front and one at the rear. And every time you get an enqueue, you actually append it to the head of the rear list. Every time you dequeue, you take it off the front list. If the front list becomes empty, you take the rear list and put it in front. So think of it like when you're standing in a queue for a bus ticket or checking into your flight, how they have, like, those dividers and the queue snakes around like that. That's the idea here, right? And you're breaking it into two lists. That's all it is. So let's look at an implementation. So what we're doing at the top there is we're declaring our queue to be a couple of two lists. Again, homogeneous, right? Alpha is the same in all three places. So I'm saying it's now two lists instead of one. Empty just returns two empty lists. Is empty just checks whether the front list is empty because the invariant is that the front list will only be empty if the queue is empty. And I have to ensure that I maintain this internal invariant. This is an invariant for my implementation, not for the signature. What does head look like? So like I just said, if the front list is empty, so I'm pattern matching on only the front list, I don't care what's in the rare. Then I'm going to raise empty. Otherwise, the front list is x constant to something and I'm going to return x, right? Simple enough. So basically I'm saying head takes from the front. Then what does enqueue do? So again, look at this pattern. This pattern is saying when you give me a couple of two lists, f and r and an element x, I cons the element onto the rare list and then I check and make sure that the front list is not empty. And why is this necessary? Because the first time, the first time when I do enqueue, my front list will be empty, right? So when I do an enqueue and I push something onto the rare list, I don't want to leave the front list empty because otherwise my queue will think that it is itself empty. Fair enough. Okay, what does this checkf do? So checkf is checking if the front list is empty and I have a rare list, reverse the rare and swap them, basically. Otherwise, just return the queue as it is, right? Simple enough, yeah. But reverse is order n, no. So we'll see why this is not the same as that in just a minute. That is a good point, but we'll see why. Tail is again doing destructuring on the empty list raising empty. Otherwise it's returning checkf on f and r, right? So you want to raise empty before you do the second pattern match. Yeah, that's a good point. Sorry, this is the old slide. I think I forgot to do it with the older names. All right, so correct, yes. Functional yes. Performance, kind of sort of, right? Why kind of sort of? Because cons on the head of something is constant time. When you put something on the rare list, it is constant time, yes? So what we're doing here is we're doing an amortized analysis. So what does amortized analysis say? Amortized analysis says that I don't really care about worst case. I care about a sequence of operations and what's going to happen in that case, right? So if I do nq, nq, nq, nq, nq, and then I do a dq, and then I do a dq, dq, dq, something like that, how many times am I going to reverse? I'm not going to reverse all the time, right? So basically what it's saying is that each time I don't do a reverse, I earn a credit. It's kind of an accounting system. Each time I don't do a reverse, I earn myself a credit. When I do do a reverse, I have to spend those credits. Then I don't have those credits anymore, right? So given a sequence of operations, we could very well have for n operations, it could be on. Not for any one particular operation. That's what amortized analysis says. So it's kind of sort of. But if you do a reverse twice on the same list, you're taking the hit over and over again. And because we have persistence, that list that originally got created is never going away, right? So if you come up to a place where on the next call to tail, you are going to have to reverse, then all further calls to tail you are going to have to reverse. If that makes sense. And so you can only spend your credits once in this amortized analysis. You can only spend the credits that you have saved up once. I have called reverse once. I've spent my credits. Now that Q which I called reverse on is still around and somebody else could force a reverse on it again. So that's why I said kind of sort of. But it's better than before. And it's a clever idea. Fair? Okay. Let's move on to an associative data structure. So far we've seen sequential data structures where the order of insertion mattered. Associative data structures are things that you can use as values basically. So sets or maps. And map edity is fairly straightforward, right? You put something into the map, you get something out, you delete something, so on and so forth. What are the invariants? Again, I feel very uncomfortable with invariants because I don't have good tools to express them in languages. So I have to do with English. Maybe there are formal ways to express them in math notation. I don't know how. Other than maybe generative testing. That would be a good way to specify invariants. But basically I've just said what you put is what you get. So if I put a key and a value and then I get that key, I should get the value back. That's an invariant for a map. Everyone agrees? Okay. Here's a signature in ML again. So it's saying there is a type key and there is a type alpha map where alpha is the value. Alpha map and gives another alpha map. Get takes a key alpha map and gives another alpha, right? Why is key a separate thing? Because Okasaki said so. No, it's because we want the keys to be ordered for certain map implementations. And we'll quickly see why that is. But before that, ideas, how do I implement maps? Sorry? Yes, perfect. So you could just have a list of key value, key value, key value. Because we already have lists. So it could look very similar to the Q implementation except instead of putting a value, you put a key value. And then when you're doing a lookup, you go through that list until you find the key that you want. Intuitively obvious that you can do this. Obvious downside also. Not only lookup, even your insert. But yeah, you could have cons on the head. So lookup is going to be terribly slow as your map grows bigger and bigger. And I'm guessing that most of us don't know more about lookups because you kind of create maps one time and then you pass it around and look things out of it. So we care about lookups. We don't want lookups to be slow. So what other ideas do you have? Absolutely. So you don't have to back it with a list. You can back it with a tree. And again, we're seeing an example of structural decomposition. So we're using existing data structures and we're implementing something bigger. So how do you use a tree? It's a code. But I wanted to show you this piece of code because I think SML is awesome. So this is something called a functor in SML. Can everyone see the code? Okay. So this is a functor in SML. What is a functor in SML? A functor is something that takes one structure and gives back another structure. So it's a way of combining structures together and building bigger higher-order structures, right? So what this guy is saying is give me an ordered element as my input, and I will give you a map as the output. That's the top declaration. The functor is saying give me an element and ordered over there itself is a signature, right? So element needs to be some structure that satisfies the ordered signature. And ordered means what? Greater than, less than, equal to three operations on that type. So what I'm saying inside my map implementation, which has to remember adhere to the finite map signature. We saw the finite map signature a couple of slides back. Finite map declared a type called key, and this is why it's important, because now I can enforce that my key has to be ordered, right? How do I enforce that? Because I say my key is the abstract type from element, and element is an ordered element, right? So now my key is ordered. That is enforced. Then I have declared a data type called tree, which is a alpha tree, which is either empty, or it is a T of key, alpha, alpha tree, alpha tree, right? So this is just a way of saying couple, and this is a type constructor. So E is the empty leaf node. There's nothing there. Or T, which contains the key value, which is key star alpha, and left subtree and right subtree. Fair enough. It's a recursive data structure because it references alpha tree in the definition of alpha tree, right? Okay. How do you do put? So put KV in an empty tree, we'll yield a new T node with KV empty, empty, right? So you have one thing which has KV as its values, and its two children are empty, right? This is the zero case of a recursion. This is a recursion ending case. The second part is saying put KV, this is an alias, small t, and here I'm destructuring what I'm currently seeing, right? So the node where I'm currently at has to be constructed with a T which I've already taken care of the empty case, right? So it's not an empty node. So it's a T node. I have said K prime as the name of the key. I don't care what the value is because I'll overwrite it. And then left and right are the subterries. Now look at my checks. My checks are using element dot less than. My checks are not using the less than operator, right? So it's basically enforcing that anything that is an element can be used as a key. Or anything that is an ordered can be used as a key, right? So if element less than KK prime, then you're going to go down the left side. Otherwise you're going to go down the right side. Otherwise, you're already there. You're already at the node where you want to do the put. And so you can just return T of KV left, right? Fair? Same for get if, oh, one second. So get of K and empty will raise not found because you've reached an empty node. Your recursion has ended, right? You've already reached an empty node. Your K has not been found. Otherwise, again, we're destructuring on the T node and we're saying if less than KK prime then go down left. Otherwise go down right. Otherwise return V prime, which is this guy. So K and K prime are equal. So we've already reached the right place. Make sense? All right. Again, structural sharing. What that code is going to yield is something like this. When I put something in X's when I do a put, right? The first check I do is whether I should go down on the right or the left. If I choose to go down on the right then my left pointer remains unchanged. Right? In the result, my left pointer and I'm calling it pointer, but it's it's just a name in the ML code. It's pointers are incidental. So my left subtree of D is shared between the two trees completely. You see that I don't have to make a copy of the tree and then go and update that copy. I can freely share the left side because any further invocation is going to keep sharing. There's not going to be any mutation anyway. Right? And this is a big D. Structural sharing again. Okay. So all is well until your tree is balanced. But what if you keep inserting keys in increasing order? Correct? You'll keep going down the right side and your tree will degrade into a list and you're back to ON. How do you get around this? So one of my favorite movies, these two guys are awesome. They say For the guys who don't speak Hindi it means balance is essential. Right? So unbalanced binary trees are not going to cut it. We want balanced binary trees. Family. There's AVL trees. There's A8 trees I think. There's 2-3 trees. There's red-black trees. Splay trees, yeah. 2-3 finger trees and what have you. I'm going to focus on this one because I like it and I like the name. Gibas and Sejvik described this back in 78. So what they said is let's add a color property on every node. So node can be either red or black. Second constraint node can have a red side. Third constraint, most interesting one, every path from root to empty contains the same number of black nodes. Okay, first time I saw this and I was like, what? This leads to a balanced tree, how? I did not get it at all. It was extremely confusing. Then I searched on Google and I found a course by Professor Ruffgarden of Stanford on Coursera and he has done an amazing job of explaining this. There is also a tutorial by Julian Walker on eternally confused which explains this. But intuitively the idea is like this that suppose I have to insert a new thing and when does unbalancing happen when you insert things on one side and not on the other right? So suppose I have to insert a new thing, then let's say I inserted it on one side then intuitively, okay, before that let's understand what path from root to empty means. What is a path from root to empty? It's an unsuccessful search. You are searching for something, you went left, right, left, right and you came to the bottom of the tree and you are unsuccessful. You did not find something. That's an unsuccessful search. So imagine if the tree was unbalanced. Imagine if the tree was unbalanced. There was a lot more stuff on this side than on this side. Then an unsuccessful search would obviously contain more nodes than a search that went down on this side, right? That much everyone agrees with. Now, if you wanted both sides to have the same number of black nodes then on this side if there were some X black nodes, on this side also you could only have X blacks and then between them you would have to keep putting reds. You can't put any more black nodes, right? But if you keep putting reds you are going to reach the first or the second invariant which is no red node can have a red child and that breaks. So you have to rebalance. So you have to use tricks, but intuitively why do these invariants lead to balance? It's because of this. Does that make sense? Okay, cool. So here's how you insert something into the list, right? So into the tree. So if I had to put seven, first what will I take? I'll take the left, then again I'll take the left, then I'll take the right, then again I'll take the right and here I am, right? But you can clearly see that this thing is violating my invariant because red has a red child, right? So Okasaki's strategy is basically let's always try to insert a red node. To begin with let's always try to insert a new red node. If it violates the constraint let's balance. Fair enough, that's his strategy. So this is our problem. So basically Okasaki identifies the various zigzag patterns that can lead to violation of the red-red constraint. And he says that you can rotate the tree. Can you see the names? I don't know if you can. But basically the idea is these are nodes and these are sub-trees, okay? These are entire sub-trees. The triangles are not nodes. They are the full thing that's below. So what he's saying is all these shapes are violating the red-red invariant, correct? Zigzag, zigzag. Because it doesn't violate. And we have two blacks at the same level on both sides. So that's not going to add anything to our first constraint. So that's the idea, that's the big idea. Here's the code for balance. Four lines. Mind blown, again. And to me actually this maps exactly to the geometry of this, right? Because you're using type constructors to do pattern matching. So if you see these things like x, y, z, a, b, c, d, those are all the variables. And you can literally make patterns out of this. And for each of the four shapes that we saw, rotate it and bring it back into what you want, right? Isn't pattern matching awesome? All right. Do we have time? Four, sorry? Yes. That's for Debashis to answer. So delete is left. Actually this is interesting. You can use delete as an exercise to the reader because Okazaki is like that. And Matt Might has written a blog recently where he implements delete and Scala. So it will be similar. I'm not even going to try to answer your question. I see. I see, okay. Yeah. Right. So if you look at Corman and Lyserson, which is CLR, they identify eight cases. But Okazaki has actually reduced them in the form of information. So it's pretty cool. Finally, I have just two or three minutes left. I'll quickly run through this. So this is another super magic thing, right? If somebody has asked me how Clojure is implemented, no chance in hell that I would say persistent bit partitioned vector try. No chance. And that is why I said any sufficiently advanced data structure is indistinguishable from magic. So what is a try? A try is something where it's like a state machine, basically. So if you're searching for a string in this, say you're searching for 10, TEN. First you look for T, then you go down this side, then you look for E, then you look for N, and you reach that node, right, at the bottom. Am I pointing? So TEN10, right? That's how tries work, very basic. What is a bit partitioned vector try? Okay. So let's understand what vectors are. Vectors are functions of their index. Vectors are associative structures because you want to look up the thing in the n-th position in double-quick time, basically, right? So it's like a map, except the key is always the index. Does that make sense? Right? So now if I want to look up the 887th thing in this vector, and I'm really sorry about the bad image quality, what you do is you divide it into digits. So let's say we were doing this in base of 10, right? Just bear with me here. 8, 8, and 7. Suppose I built a try, which had 10 branches at each level. What would I take on the first level? I would take 8, then I would take 8 again, then I would take 7, and I would reach what I wanted to reach, right? So exactly like this, what Rich Hickey did is he used binary to speed this up, and he used basically a 5-way branch. So if you use a 5-way branch, or sorry, a 2-way branch, to use a 32-way branch, what you need is 5 binary digits. 5 binary digits produce 32 numbers, right? So the idea is that take a number, convert it to binary, then take the first 5 bits and see which branch to go down, then take the next 5 bits and see which branch to go down, and so on and so forth until you reach where you want to reach. That is broadly the idea. Other than that, it's the same as the previous tree I showed you, structural sharing. Roads can reference each other and so on and so forth. Big idea is partitioning using bit operations because you can basically do shift by 5 and what is it, and or with 31. Just basically 1, 1, 1, 1, 1. You and it, you'll get the 5 digits on this side. Or you shift this side, you'll get the 5 digits on this side, right? So basically it's just shifting, anding, anding, and that is super fast. So in terms of actual performance, plus, BG told me recently that 32 is the cache line size in the JVM, so 32 things are also local. Very quick lookups. You won't have very many cache misses. So super efficient implementation of vectors. All right, so quick summary of what we learned. First thing is abstraction is vital. It is vital for me to not have to care about this in my day-to-day life. Imagine if I had to invent a vector each time I needed it. I would not be able to get anything done. We need to study implementations to understand trade-offs. And Richiki says that engineers understand the value of everything, but the trade-offs of nothing, right? So trade-offs are vital. We need to know which data structure to use in which scenario. It's necessary to go one level down to get that. Structural sharing, structural decomposition help us build bigger data structures from smaller ones, and also help us avoid just copy and mutate, right? That's amortized analysis and laziness. So laziness is something I skipped, but there is this concept called which cons gives you. Where you only compute the things that you need and you don't compute what you don't need. And that is also a very nice way of doing infinite streams and what have you. Are pretty cool techniques. DJ Spivak has given a talk at one of the closure conges called extreme cleverness. And I love this name because I think that at these levels of abstraction there is a lot of extreme cleverness going on in order to give us languages that we will then go on and use to build whatever website that we want to build. So extreme cleverness is there. Finally, further study I would highly recommend that one second. So the persistent vectors I took from a blog by a person called John Nicholas. His blog is called Hyperion, H-Y-P-I-R-I-O-N. He has written a series on persistent vectors in closure. Really, really nice. He has written a lot about this. So go look up Debashish's blog. Julian Walker, who I mentioned earlier. Her blog is called eternally confused and she describes all the data structures, not just red-black trees, AVL trees, play trees. Everything is there. Another really nice resource. Finally, Coursera is brilliant. So Rough Gardens course is nice. Like someone said, Sejvik has a course. You know, we can't complain. Everything is at our fingertips. So take your time, go and study. I don't think I'll be able to answer them. Programming languages by Dan Grossman. It's coming up on October 15th. Awesome course. Oh, it started, is it? Okay, great. Yeah, so that's basically it. Final note in closing, I don't know if you guys have read Matt Might, but he has this thing about what it means to get a PhD, where he draws a big circle of human knowledge and then he says after high school, you know this much. After bachelors, you know this much. Then you start specializing, and specializing, and specializing, and after PhD, you add a little dot in the sea of human knowledge, and that's what Okazaki has done. But damn, man, that is a dense book. That little dot that Okazaki has researched is really a treasure. And it's thanks to people like him that functional programming I think is probably going to become mainstream. Thank you.