 All right, guys, we can jump into this. Let me make sure this is on. It's on, all right. All right, so today we're going to talk about in-memory databases. So just before we get into the outline, real quick, everyone should have submitted the synopsis for the first assignment. Remember, this is something you go online to the Google forum to turn it in. Everybody have problems with this? OK, awesome. So for people that are possibly still on the wait list, I'll address the project number one at the end. So currently, as of this morning, or actually as of just now, we had five people drop off the course. So now there's nine open slots. So it's very likely that if you want to get in and you finished project one in a reasonable amount of time, you'll be able to get into the course. So I strongly encourage you to keep up the date with the readings. And then if anybody else wants to audit the course, there is an official form that I fill out for CMU. You don't get credit. You just get something on your forms when you graduate and said, yes, you audited this course. Your transcripts is what I was looking for. So again, if you want to do that, just let me know. We can sign it up. Now has everyone been able to sign up for Piazza? They're going to have an issue with that. And then for AutoLab, we will enable the VM later today after class so that everyone can then submit project number one. OK? All right, so another thing I'll say too is, and I should have said this last time, but I forgot. I get excited when I talk about databases and I start to talk fast. And that if you're not a native English speaker, that can be confusing. So if I say something and you're like, I don't understand what he's saying, please stop me and I'll slow down. And also too, if you have questions about something we're talking about, please raise your hand and interrupt me because if you have a question about something that somebody else might have the same question. So I'd rather, we can discuss it here than you come up to me after the class. OK? All right, so for today's lecture, we're going to talk about the background, some background information about database systems. And then we're going to spend time talking about the reading, about in-memory database systems. So we'll talk about why they actually are different than the discord in the guys and what makes them special. And then we'll finish up talking about Peloton, the system we're building here at CMU. That's an in-memory database system. And then we'll also talk about project number one. So in the last lecture on Tuesday, I said it was the history of databases, but it was really the history of data models. I walked through all these historical systems leading up to the current time right in the front, go for it. And I wasn't really talking about how these things were actually implemented or not. I talked about how they actually organized data. We talked about IMS, which is the hierarchical model. We talked about Codercil, which is the network model. Then we talked about the relational model. And so the overarching theme that was going on while people were building these systems is that everyone was building what I'll call disk-oriented database systems, right? And the issue with disk-oriented systems is that you're dealing with data on disk and it's really slow. So all of these systems, a lot of what actually goes on inside of them are designed to mask the slowness of disk. And so the other thing, and so we'll see how we actually handle that as we go along. And then the other thing to be mindful of is that the first systems like the Ingress and System R, when they were built in the 1970s, these were designed for the hardware that they had available at the time. Right, so that means that they were designed to work on a massive mainframe machine that maybe just had a single CPU core, a single unit processor. The amount of RAM that they had was really limited compared to what we have today, right? We're talking maybe like kilobytes, if you were lucky. So that means that the database had to be stored on disk. But disks back then were even slower than the disks we have now. So they had to spend a lot of time trying to figure out how to overcome this problem. So, but now we flash forward into 2017 and DRAM capacities are such that you can get, you know, for not a lot of money, you can get a machine that has enough RAM to probably store your entire database. Most databases aren't that big. I don't have exact numbers, but every single time you talk to people, maybe a couple gigabytes, some of the larger ones could be in the hundreds of gigabytes. You certainly have the outliers, the terabytes and the petabytes and things like that, but those are, again, the Googles and the Facebooks of the world. Majority of people don't have that really large database. So now you may think, all right, well look, if I can get a machine on Amazon that has a lot of DRAM, can I just take one of these, so we'll all say traditional disk-oriented database systems, the ones that are based on the same design assumptions that System R and Ingress had back in the 1970s? Can I just take one of these existing systems and just give them a ton of DRAM or give them like a memory map file that's mapped to a RAM disk? And is that gonna be good enough? Can I still get really good performance? And as you'll see, the answer is no and this is sort of motivating why this course matters because we'll show how there's all these design assumptions that in the late 1970s that we can throw away when we assume the database is residing entirely in main memory and this is gonna allow us to get better performance. So before, again, before we talk about the in-memory stuff, again, I'll do a refresher to talk about how a disk-oriented database system works and I'll go through the three major components and you'll see why they are slow if you assume everything could be on disk and then we'll talk about how we can make better versions for in-memory. And this again, this is sort of a precursor for everything that we're gonna talk about in the course. So I would define a disk-oriented database system is one where the architecture is designed such that it assumes that the primary location of the database is on disk, right? Some kind of non-volatile storage. So like a SSD or a spinning hard drive and the database is gonna be organized on disk in a set of fixed-length blocks organized as slotted pages. I'll remind you what slotted pages are in the next couple of slides, right? So it's basically gonna say, my database is here, I'm gonna break it up into blocks and each block will be a slotted page format. So now what's gonna happen is the data is gonna use an in-memory buffer pool, right? Which again, it's based on DRAM, so it's volatile. It's gonna use this buffer pool as a cache for blocks from the database that it fetches in and out from disk, right? And so there's all these components in the system that are designed to manage this movement of data back and forth between DRAM and disk, right? Because if you modify something in the database, you always have to modify it in memory and then you don't wanna lose any data, so at some point you have to write that dirty page in memory out the disk, right? And so this is what basically the data system is gonna be doing in these disk-oriented models. So let's focus on the buffer pool because this is sort of the main thing that you have in a disk-oriented system that you're not gonna have it in memory system. So with the buffer pool what's gonna happen is it's gonna keep track of what pages that are in memory that it got from disk, right? So what happens is say you have a query comes along and wants to read a record, right? So what happens is you say you follow an index, it tells you what page the record is on, and then so now the system will go into the buffer pool and check to see whether that page is already in memory. If it is, then you just access it and you're done, but if not, then you gotta go retrieve it. So what you need to do is then go figure out where it is on disk, go fetch it, and then copy it into a frame in your buffer pool. Your buffer pool has a fixed number of frames. So if there's no free frames though, then you gotta evict one of them, one of the existing ones, right? So that means that you're running some kind of eviction policy to say, well, which one should I get rid of, okay? And these are things like, simple things like LRU or clock, but in the more sophisticated commercial systems, they're running complex algorithms to try to keep track of things like, you know, what's the statistics about, what's the likelihood that a page is gonna get, you know, read or written to again in the future based on what it's seen in the past. All right, so you need to find a frame that has a page that you can evict, all right? And if that page you wanna evict is dirty, meaning some query in the past had modified it, then we need to write it back out the disk, right? If it's clean, meaning it hasn't been modified, then you just throw it away without doing anything, right? So then once you do all this, once you just sort of find a frame you wanna want, evict whatever's in there before, possibly write it out the disk, then you're allowed to go copy the new page you want into that frame. And at that point, you then you have to go back and update any internal data structures you have to say, for this page, it's no longer in disk, it's now here in memory, and here's where it is, right? So to give you sort of a high level overview of what this looks like, let's say that we have a really simple database that has an index, an in-memory buffer pool, and then we have our collection of slotted pages out on disk, right? Now, for this example, I'm gonna assume that the index is entirely in main memory, right? And so we're not gonna worry about pages getting swapped in and out of this, but in reality, the index nodes are actually treated as pages as well, so they follow on the same sort of buffer pool policy as data pages, right? So to say our query does a lookup in the index and finds the record that it wants, and for that given key, the value will be its record ID, which is a composite composition of the page ID and the slot number for that individual tuple. So now what you need to do is go look up in the page table and say, for this page ID, tell me where it is, right? Now in this example here, we need page one and that's out on disk, so we need to go fetch that in. So now we're gonna do all the steps that I talked about before, we gotta find a free frame, throw something out and put our new guy in. So now when we recognize that we have to fetch the thing from disk and put it into memory, we actually have to set a latch here in the page table to prevent any other thread from coming along and trying to do the same thing, because otherwise we'd end up with two copies of the same page in memory and we don't need that. So I'm clear here, does everyone understand the distinction between a lock and a latch in the context of databases? All right, she's taking her head now, anybody else? No, okay, so if you come from an OS background, a latch is really called a lock in OSs. In databases, we call it a latch, right? So in databases, a lock is a higher level primitive that you take on a logical aspect or logical component of the database. So you take a lock on a database, take a lock on a table, take a lock on a record, right? A latch is the low level mutex or spin lock, whatever it is, that you take on a data structure. So you would take a latch on an entry in a page table, you take a latch on an index page, right? So a lock is like, take a lock on a table, a latch is like an internal thing, okay? So for example, you can write SQL that takes a lock on a record, you can't write SQL that takes a latch on a record because that happens internally, does that make sense? I can, this always trips up people, if they don't come from a database background, I say a lock and a latch, I'm like, what's the difference? Okay, so we're gonna take a latch, again, it could be a spin lock, it could be a mutex, it doesn't matter. We'll take a latch on our entry in the page table and then we'll run now our eviction policy to decide which frame we wanna free up. So let's say for whatever reason, we wanna take this page two, it has been used in a while, we think it's safe for us to go and get rid of it, but it's dirty, so we need to go write it out to the disk, right? We'll just overwrite the one that was already there. So now we can free this up and then we can copy our page one into this location here, right? Are we done? See, he says release the latch before that. Change the page table. Exactly, now you gotta go back into the page table, update that to now point to the new location, right? This is the swizzling part, this is saying like, all right, before I was out on disk, but now I'm in memory, so if anybody comes along and wants to look for me, here's where I am, right? And then you go ahead and release the latches, right? And then now you're allowed to do this. So now think about this, right? See all these steps that we had to do to go fetch the one thing we needed from disk. That's expensive. Now if you imagine if you just assume that everything is in memory, then you're doing all these steps, you're not fetching anything from disk, but everything's already gonna be in memory, you're doing these steps over and over again, even though the thing that you're looking for is never gonna be out on disk, it's always gonna be in memory. And the reason why the disk ordinance systems do this, because they don't know that you're never gonna have a database that exceeds the amount of DRAM that you have, right? It doesn't know that I have a 10 gigabyte database now and I have 100 gigs of RAM. My SQL's not gonna know that you're not gonna come back a week later and load up 200 gigs of data, right? So it has to do these steps over and over again. So just to remind you what a slot of page looks like, because this will help you understand how the in-memory database organizes data differently later on. A slot of page is basically like the sort of how a database system organizes tuples together into a page or a block, right? And typically these pages could be 4K and my SQL is like 16K, usually on that order. All right, so at the top we have our header and the header just says for this page, here's how many tuples I have, here's the version of the database system that wrote this page, here's the check sum and things like that. And then at the end of the page, we have the fixed length data slots. So these are all for every tuple that's in our page, it's the contiguous values, it's all the values that are contiguous that are fixed length. So anything that will not be a different size no matter what value you have. So a thing like an integer, a float, a timestamp, these things are always 32 bits, 64 bits, no matter what value you have. So you can have all of these at the end and it's gonna go in this direction to add them. And at the top or at the beginning of the page, after the header, you're gonna have the variable length data. So these are the things that have different sizes based on different values. So like a var char, a var binary, a text field, things like that. And then what you'll have is down here on the fixed length guys, you'll have pointers up to the variable length fields for each tuple, right? So tuple three has some variable length field that's pointed here in block two. So the way now what happens is as you add items to this page, the fixed length guys grow this way and the variable length guys grow that way and then at some point you meet in the middle and when you can't put any more of the fixed length ones, the variable length ones, then you consider the pages full. So this is pretty efficient. This is probably the best way to do this as far as I know. This is what pretty much everyone does. The only downside is that you're always gonna have like a little bit of free space that you can't use at all because you can't fit anything else in, right? So you waste just a little bit of space but in practice it's probably not too bad. So this is what a slot of page looks like. All right, so I sort of said this before but the database system is gonna have to go through all those steps of the buffer pool that I showed before no matter whether the data is always to be in memory or not. You're always gonna have to translate the record ID to its memory location by doing the look up in the page table and then when you go start accessing the page you need in the buffer pool, the worker thread has to set a latch or pin that page in memory to avoid anybody else from possibly swapping it out while it's trying to read or write to it. All right, again, it doesn't know that it's always gonna have enough space for the current data that it has, right? So this is sort of a good example why you just take an existing disk-ordian database system, give it a ton of RAM, right? It's never gonna actually work that fast because it still has the architectural components that are designed from an era when your database could be larger than the amount of DRim that you have, okay? All right, so the next thing we gotta deal with is concurrency control. So remember concurrency control is the protocol that the database uses internally to allow multiple transactions to run at the same time without interfering with each other, right? Because we wanna provide the asset guarantee that our transactions appear as if they're running with exclusive access to the database, right? When in practice that doesn't happen. So in a disk-oriented system they add a concurrency control back in the days because the problem with these systems is that any time a transaction could try to access a page that's not in the buffer pool and therefore would have to stall while the data system goes and fetches the thing that it needs and puts it into DRAM, right? And because this was super slow you don't wanna have this long stall for your query any single time you access something that's not in DRAM and had the system appear to be completely unresponsive. So with concurrency control for these older systems what you could do is you could have multiple transactions running at the same time and that way if one worker thread stalls because it has to go get something from disk all the worker threads can keep on running and still make forward progress and then from you as the application or the end user it appears that the database system is like super responsive all the time even though it's swapping things in and out from disk. This is sort of what I say at the beginning it's sort of masking the slowness of disk by allowing multiple things to occur at the same time. So now in order to do this in order to deal with these transaction stalls you need to set locks and latches to make sure that transactions don't interfere with each other, right? You don't want the cap in the case where like my first query runs and it updates something but then I go do my next query and I have to stall because some other, the thing I need is not in DRAM you don't want some other thread coming along and updating something that I'm about to read and violate the serializable order of transactions. So they're gonna use locks and latches to do this. But now the way now you're, so this is another extra data structure like the page table that we're maintaining because we have to, because transactions can stall at any time because we're getting things from disks. So in these systems, they're gonna store the locks in a separate data structure that's an always pinned in memory that's gonna be separate from the actual data itself, right? And this is sort of obvious why you wanna keep this in a separate data structure that never gets written out to disk. Why does this matter? Because disks are used to mask the slowness of the disk. Yes. It is stored on the disk, it is used in disk. Exactly, so he said you're using locks as part of the Concertico protocol to mask the slowness of disk. But now if you put the lock information for the Concertico protocol out on disk, then any single time a query needs to access a page, you may have to go out to disk and get that page first to figure out whether you need to touch something, right? So by keeping these things in memory, you avoid this problem. But again, this is sort of now this, you have to go check this every single time you wanna access something. So to sort of, even if it was in memory, it's one sort of possible catch miss to go to the lock table then another catch miss to actually go get the tuple, right? This is a good point, okay. So the next thing we gotta deal with in a database system is the logging protocol and we need this for recovery. So most database systems use the steal no force buffer pool policy. And so that means that when a transaction commits, all the changes that it made to the database has to be flushed out to the right of head log before it's allowed to commit. Right again, so steal means that you're allowed to flush dirty pages out the disk before the transaction finishes. No force means that you don't require all the dirty pages to be flushed. And so the only way to do steal no force is that you have to have a right of head log, right? So the right of head log is always gonna contain the before and after image of anything that was modified because if we crash and come back, we need to figure out whether the changes we have on disk for our database and our slot of pages should actually be there or not. So the before image allows us to undo things, the after image allows us to redo things. So we talked about three things. We talked about the buffer pool policies and the pool manager. We talked about the concurrency control scheme and we talked about the logging recovery stuff. So these are the three things that need to have in a database system if you're gonna assume things are on disk. So now there was a study done, I guess man, it's almost 10 years ago, 2008. And for the project group that I was involved in when I was in grad school, where they took a sort of state of the art disk-oriented database system, this thing called Shor out of Wisconsin. And at the time of 2008, Shor came out of maybe like the 1990s, but it had all sort of the standard things that we talked about here that you expect to have in a modern database system. So it had a buffer pool, it had a current actual lock manager and it had a right head log for logging recovery. So they took that system and they loaded it entirely, they put it, they loaded a database that fit entirely in DRAM. And they got rid of the disk entirely and they wanted to do, they wanted to measure the amount of time that the CPU spends in these three different components to figure out where all the overhead was going. So you can do the breakdown between the buffer pool, the lock manager and the recovery protocol. And then this last metric they have here is the remaining time after you do all these three other things of where the database system was actually doing what we'll call real work or useful work. So this is actually, the part that actually executes the query does lookups in the indexes and modifies the database. So for the buffer pool, they found it took 30% of the time of the total system. So that means 30% of the CPU cycles when you actually run sort of a standard transaction benchmark on this thing was spent doing all the things we talked about before. Maintaining the information that you need for the eviction policy, pinning and unpinning pages, doing the translation lookup in the page table. So that's 30% of the time. The lock manager is spending another 30% of the time. So this is the acquiring the locks, possibly waiting for locks that you need, updating the wait for a graph, running the deadlock detection policy, things like that. 28% of the time is spent in the recovery manager. So this is preparing the log records, updating the information about dependencies between different transactions in order to write them out to the log. This has nothing to do with actually writing things out the disk. They didn't measure that at all. It's just the CPU cycles. So this leaves you a poultry 12% left over where the data system is actually doing useful work. So this one graph here motivates the entire course. It's showing you that all this 88% of the time is spent doing stuff that's possibly unnecessary if your database is gonna fit entirely in DRAM. All right. So now this, again, so the study was done in 2008. Since then, and what this course is really gonna focus on is there's been a lot of sort of modern variants for in-memory databases for all these three things. So actually, I think it'd be very interesting to go back and do the study again to see like say it doesn't have to be our database system and take another in-memory database system and re-measure all these things and see whether how bad things are actually now too as well, but that's for later. Okay. All right, so again, this one graph here motivates the entire course. All right, so now we can switch over to in-memory databases. So in an in-memory database, I've defined as one where the system architecture assumes that the database, the primary storage location of the database is entirely in DRAM. We'll relax that later on when we talk about larger than memory databases and I'll say something about that at the end. But again, we just assume that everything we ever, any tuple that a query could ever want to look at will already be in DRAM for it. So now this idea is not new. There was a lot of early work done in 1980s, especially at the University of Wisconsin, but they sort of proposed these prototype systems and thought about some of the things that we're gonna care about in this course, but it was never actually used in production because at the time DRAM capacities were quite limited and it was super expensive. And the survey you guys read was from 1992, which I realized is probably before most of you were born, but you can sort of see that they've been thinking about this idea for a long time and a lot of the key ideas of what we're gonna have in a modern system, they sort of laid out the foundation for back then. It's just now we have actually the Harvard actually to do this. So you may be thinking, if when we talk about in-memory databases, given that I just showed how the buffer pool managers are really expensive, the commercial stuff is really expensive, what if we just didn't have a buffer pool at all and we just let the OS manage this? So what if we just use M-MAP or memory map files? So real quick, who here knows what M-MAP is, the syscall? Come on the back, okay. So what M-MAP allows you to do is the application or sort of user level process can tell the OS, take this file from disk and map it into this region of memory, sort of like, instead of like, you know, mallocating a bunch of space and then copying out from the file and putting it into memory, you can just tell the OS, take this thing and map it into this region of memory. And then what happens is the OS takes control of deciding when things are brought in and out from disk into DRAM, right? So no longer you have to have a buffer pool do this for you, the OS just does it for you. So it maintains its own sort of information about what page was read last, what page is dirty and how to move things back and forth. And then what you can do is you can use the syscalls M-Advise to tell the OS, hey, make sure you only, you know, these regions of my memory map file should always be in DRAM, don't swap them out. And then if you do some writes, you wanna make sure that they're actually flushed out, you can use the msync command to tell the OS to do that. Because otherwise the OS decides on its own, you know, when it has free cycles, when to write out dirty pages on its own. So, M-Map is not a new idea, it's been around for a while, but there's not actually many database systems that use this. The three that I only know about are these here. The first one's probably the most famous, right, MongoDB. When they first started they used the M-Map storage engine. MonetDB is an analytical system out of CWI in Europe. It's one of the first column store systems about from a decade ago. This is sort of a good example. MonetDB is a good example of a system that came out of academia that sort of got into the real world and people actually use. And the last one is LMDB, the LightningMap database. So think of this as like an in-memory embedded database, sort of like SQLite. And we actually had the developer of this, or the inventor of this come to CMU about two years ago to give a talk. So there's not many, this is not very many, right? You don't see Postgres, you don't see MySQL, you don't see Oracle, DB2, and all the big database systems you know about, right? And if you think about this also too, like MongoDB, they started off using M-Map, but then about two years ago now, they ditched that and switched to using WireTiger, the WireTiger storage engine. And WireTiger doesn't use M-Map, it uses the buffer pool that we talked about. So given that like MongoDB had raised whatever, a quarter billion dollars, they could have done anything in their database system. They could have made the M-Map engine be super, super awesome. But they ditched it and switched to WireTiger, right? So if anybody here knows M-Map, why would you not want to build a database that isn't using M-Map? Yes. Because in databases, we usually want to have a control over what pages we flush in and out of memory. Yes. Which the OS can't really determine. Exactly, so she said with M-Map, you relinquish control over the database process for linkages control, what it decides, how to write things in and out from the disk, given over to the OS. And the OS doesn't know anything about queries, it doesn't know anything about transactions, it just knows that these pages are being accessed. So there's certain aspects of certain workloads you can imagine where yes, this page was just less accessed by some query, but I probably want to not, I probably don't want to keep it around, I'll probably want to flush it out because I'm not going to access it in the future. And the OS doesn't know this, the OS just knows that I access this thing less. So she's absolutely right. So if you use M-Map, you give up the fine-grained control of how you move everything out of memory. The other big issue is that you cannot perform non-blocking memory access. So in the buffer pull example I showed before, the database system knows, oh, the thing I'm looking for, the page I need is not in memory, let me have another thread go and fetch it and pull it in and my original thread can then go off and do whatever it wants, still try to do useful work. With M-Map, what happens is you try to access the memory region that the OS hasn't swapped in, then you get a page fault, it blocks your thread, while it goes and fetches the thing that you need and once it's available then it wakes up your thread. So that sucks in our work in the database world because that's what we were trying to avoid by having a buffer pull and concurrence show in the first place. So again, the key thing is that the OS has no way of knowing exactly what's going on with the application and its workload and therefore you don't want to let the OS manage these things. If you don't remember anything from this course, don't build your database using M-Map, because the data system is always gonna know best. Now, you may say, what if I just take M-Map, I map in a file and then I just use M-Advise to pin that entire region so that it's always in memory, I'll run M-Sync or flush it myself and I don't worry about the OS really doing anything. In that case, they're essentially just doing the same thing as malucking a big chunk of data and leaving it in memory. The only difference is that the on-disk representation of how things are, of the contents of the data is in memory has to match, the on-disk representation and the in-memory representation have to be exactly the same. So when you M-Map a file in, however you modify that memory in your process is how it's actually gonna get written out the disk. And we'll see when we talk about compression and other things, there may be cases where you wanna store the database compressed on-disk but then when you wanna bring it in memory, you wanna uncompress it and you can't do that very easily with M-Map. So again, never use M-Map, people will see the video, they will email and complain but whatever, they're wrong. Okay, so now if we get rid of disk and we assume everything's in memory, disk is no longer the bottleneck for us, right? So now all these other problems start to crop up in our in-memory database. Sort of like the slowness of disk was the high pole in the tent that was calling us to be slow. We get rid of that and we now find a bunch of other bottlenecks. So the things we're gonna have to deal with and we'll talk about in the course is now we gotta deal with locking, latching, we gotta worry about cache line misses, we have to worry about pointer chasing or if you're using C++, virtual function table lookups, we have to do predicate evaluation or traversal, right? So if you have like a where clause and you wanna evaluate that where clause on a per tuple basis, right? That gets expensive. Moving data around and copying it, especially when you have a multi-socket system and you have to deal with numeral regions, that gets expensive. And then lastly we have to deal with networking. So when I say networking, I don't mean like a distributed database where you're sort of running a transaction or a query that touches multiple machines. What I mean here instead is like, say you have a website and you have an application server and that's running on one machine and then you have your database system that's running on another machine. Now when you start a transaction or execute a query and you go back and forth between the application side and the server side, that actually becomes a bottleneck now because disk is no longer the main issue. The way database systems can avoid this problem or applications can avoid this problem is by using stored procedures, which we might talk about, yeah, we'll talk about next class, but in practice people don't always use stored procedures because they're kind of awkward to write and then now you have logic of your application split between the database system and the application server, which is usually a bad thing, right? So again, the key thing here is disk is gone, but now all these other things crop up. All right, so then the last thing I'll say is to be mindful, so numbers to be mindful when we start talking about getting sort of low level optimizations, everyone should be aware of what the relative speeds are of different storage mediums in our system. So on one end you have your CPU caches like L3, this is running on SRAM, so this would be, when you do a read and write, this would be about 20 nanoseconds. With DRAM, a read and write is roughly about 60 nanoseconds and it depends on whether you're writing to DIMs that are local to your socket. If you have to go over the QPI bus to another socket, that can change, but in general it's around 60 nanoseconds. So this is about three X difference here between going L3 from DRAM, which is not insignificant, but it's a lot, but it's not as bad as going from DRAM to these other solid state storage devices and then spinning disk hard drives. To do a read on an SSD is 25,000 nanoseconds, to do a write is 300,000 nanoseconds, that's several orders of magnitude difference, and then certainly going out to a spinning disk hard drive, now you're in the 10 million nanosecond range or 10 milliseconds. I think newer disks can do roughly five milliseconds, but it's still pretty significantly different. So now you can see why in the last slide I showed all these different bottlenecks that we didn't have in a disk-oriented system, because now we're dealing with these kind of latencies rather than these latencies. Jim Gray has another good metaphor, the way to think about this is like, these latency differences, like reading something from L3 cache is equivalent to picking up a book that's right on the desk in front of you, right? Whereas reading something from a spinning disk hard drive is equivalent to flying to Pluto to go get the book, it's massive and we don't really think about these things because 10 milliseconds doesn't seem like a lot, but actually is really slow when you're actually trying to do a lot of transactions at the same time, right? So this is part of the reason why the in-memory databases are gonna get much better performance, much better throughput and latency than the disk-oriented guys, because you can try to maximize the amount of data that's gonna be in L3, and then if you have to go to DRAM, it's not as bad as going to an SSD. All right, so now I wanna sort of go through sort of go through some high-level points about why in-memory databases are different than the disk-oriented guys, and again, this is sort of a preview of all the things that we're gonna talk about in the semester coming up. So the first thing is how the database is gonna now be organized if we assume everything's in main memory. So remember when we talked about this with the disk system, we had these slotted pages, and we had these buffer pools that we would copy things into, but now when everything's in memory, there's some optimizations we can do. The first is that we don't need to maintain like these sort of logical record IDs that say like we had before, like here's the page ID and here's the slot number. Now in some cases, we can have direct pointers to the tuple at where it exists in memory. And then rather than having a slot of pages that can combine both the fixed length guys and the variable length information, we'll have separate memory pools for the fixed length records and the fixed length attributes and the variable length attributes. And then we'll also want to maintain check sums in memory to make sure that if, you know, because the data can fit entirely in main memory, some thread may just start writing to an address space that it shouldn't. And that would essentially now start trashing our data, which is bad, right? So we have to maintain some check sums to make sure to avoid this problem. All right, so let's go back to that same example that I had before where we're doing a lookup at index to find the record that we want. So now when we do our traversal on the index and we find the key that we need, the value will end up being a memory address, again, rather than a record ID. So no longer do we have to do a page table lookup because this is going to point, you know, it's a 64-bit value, the 48-bit pointer on Intel machines. This will point exactly to the record that we want. And in the case of the variable length guys, what you'll have is there'll be a pointer, a 64-bit pointer, you know, as part of the attribute list for this tuple, and that'll point to some location in the variable length pool, right? And so typically what happens is, since these are all fixed length, you can organize these in like, you know, exactly, you know, you know, the fixed number of tuples per block. In our system, we use 10 megabytes. I don't think it makes that big of a difference. And then for the variable length guys, you usually use like a slab allocator where you'll have a bunch of arenas and pools for things that are less than one megabyte, 10 megabytes, or 100 megabytes, and things like that. Right, and that's, so the key thing here is that you're probably getting much better efficiency than you would get in a slotted page approach. Like as you're packing in as much information as possible with not a lot of wasted space. So we'll talk more about this policy of how you decide whether you to have inline variable length objects versus putting them out to a separate thing. Right, so in this case here, if like the value was less than 64 bits, let's say you have like a var char like two or three, then you just want to put it inline. But anything bigger than that, you want to put out here in the separate pool. All right, and different David systems do different things. All right, so now we go back to current control and the key observation that we made before and you saw in the reading is that if everything's in main memory and your locks are in memory and your tuples in memory, then the cost of going reading a tuple or going reading a lock is the same thing as just go reading the tuple. So in some cases, it may be better just to go read the tuple, right? And so in the survey, they talked about how instead of having locking on a per tuple basis as systems normally do with two phase locking, you may want to have locking that's a bit more coarse grained, right? So fine grained locking is nice because it allows for better concurrency but it requires you to acquire more locks, right? If I have to scan 100 tuples, I have to scan 100 locks. In coarse grained locking, the idea is that you have fewer locks. That means you have to go, you know, fewer times you have to go set things in memory but it's gonna limit the amount of parallelism you have because now you have, you know, one lock could, you know, even though one transaction only needs one tuple that's covered by a lock, that lock may cover 100 tuples and may prevent other transactions from trying to read those things even though it actually really is in a conflict. So we'll see more about this in the next reading. Most systems still use fine grained locking. The system that I help work on and help build H-Store or BoltDB, you'll see this in the paper, they actually use a locking scheme based on partitions. So you take a table, you split up horizontally to disjoint subsets and assign each partition to a core and so each partition will have a lock. So if you have any transaction that wants to come along and read data at a CPU core, it has to acquire that lock and then it's allowed to start running and then no other transaction can run at the same core at the same time. It allows you to be super fast if your transactions only have to touch a single core, a single partition. But if you have to touch multiple partitions as you'll see in the paper, then things get slow because you limit the amount of parallelism you have. The other difference is that now we're actually going to store any locking information that we have about tuples alongside with the actual data. So again, before in the disk-oriented system they had a separate lock table and that was different from where the data actually was. Now we're going to put everything together because we don't have to worry about the data getting swapped out. And what's nice about this is that you're going to get better CPU cache locality because when you say I do need to acquire the lock on a tuple, I go fetch the cache line that has that tuple with that lock and I'm also going to also get all the values for that tuple as well. So now I don't have to do two separate lookups. All right, we'll talk more about this when we talk about indexes, but they talk I think in the paper they talk about using mutexes for these things, but back then that was fine. Now they're actually mutexes are super slow and we want to use compare and swap and spin locks and other things to do this more efficiently. So now what we have is the contention we're going to have is not from the main bottleneck we're going to have in currency control is not because a transaction got stalled because it fetched something from disk. It's actually because we're going to have a lot of transactions running at the same time trying to access all the same thing, right? So it's very typical in database workloads that the access patterns of queries and transactions are very skewed, right? Like Donald Trump's Twitter account gets a lot of people reading it and responding to him for better or worse. And so that means you would have a lot of transactions trying to all access the records that correspond to him. So that'd be a lot of contention. Whereas like my Twitter account doesn't have nearly as many followers and I don't make as controversial statements. So in my case, the access pattern on my records is not very highly contended, right? So in the old world, they only had a single, they had a single uniprocessor so you really only had one core available but what happened is when a transaction tried to read something that wasn't on disk it would get stalled and then sort of swapped out and then the processor would then pick up the next transaction and run that, right? In our case, we're doing these multi-core environments and we could have 64 threads running on the same box so we could have some number of transactions running all at the same time trying to access all the data at the same time and they're not gonna have long stalls, right? And instead we're gonna have stalls because of disk and have stalls because you're going access to the same block, right? So it's a different challenge but some of the ideas are the same. All right, so now for indexes. In the survey they talk about these sort of specialized main memory indexes that sort of were developed around this time, right? So the MRI said Wisconsin did a lot of early work on main memory databases so they had this thing called T-trees that they mentioned in the paper that saying that these things are better than B-plus trees if you have a main memory resident database. What I'll say though is that nobody actually uses T-trees anymore. We'll discuss them when we talk about indexes in a few weeks. The key thing that happened was when T-trees first got invented the speed difference between CPU caches and DRAM was not as significant as we have now, right? They were sort of roughly the same. But then what happened in the 1990s, SRAM got a lot faster and therefore having to go do memory lookups became more expensive. So it was better to have your data all right there in your CPU cache, right? So nobody actually uses T-trees anymore. Everyone uses B-plus trees. And this is because the B-plus trees actually tend up to being more friendly to your CPU caches than the T-trees guys. I'll show when we talk about indexes I'll show what a T-trees looks like. It should be obvious about why nobody actually still uses them. The other key thing that's different in a main memory database than a disk ordinate database is that you don't actually maintain the index on disk. You don't do any logging for the index. So what I mean by that is in a disk ordinate system remember I said the index nodes actually will be pages themselves and they can be swapped in and out just like a data page. In a memory database we actually don't, for the most systems don't actually store anything about the index on disk because if you crash you just come back up and rebuild the index. And that's probably a better choice because now you're not slowing down the runtime of the system because you're logging the index information. It's just only in recovery you take a little bit more time because you have to rebuild the index. But in these modern indexes you can build them pretty quickly. The other thing that happens too also is if you're running a memory database and you're probably running sort of a high profile application where you care about availability. So that means that you'll have a replica of your main server running on the side as well. So that way if the master server goes down you can just flip over and elect the old replica to be the new master and keep serving your request that way. So in that case the replica will be maintaining the index of memory as well. So you don't have to stop the world to rebuild everything. It just sort of immediately pick up what the other guy left off. So this is why in as far as I know for all in-memory databases they don't maintain the indexes on disk you just rebuild them when you restart and you can have replicas to avoid having to have an expensive restart process. For query processing some of the key things that change in a memory database is that it's no longer the case that we had in like a spinning-giz hard drive that doing a sequential read and write was much cheaper than a random read and write, right? In the spinning-giz hard drive memory you have this little arm that needs to plop down on the platter and then it does a bunch of reads or writes. But if you have to do a bunch of random writes and the arm has to pick up and jump around over and over again and that's really slow because that's actually a mechanical thing moving. But in memory we don't have that problem so the benefit you got from doing a sequential write or sequential read is no longer that significant. So some of the algorithms we'll see when we talk about doing query processing don't assume that sequential reads and writes are faster and therefore they do different things. The other key difference is that the traditional two-plot-of-time approach for doing iterating over data when you process a query actually ends up being slow because of the function calls. Remember I said that again doing pointer chasing and virtual function table lookups becomes expensive because now you're blowing away your CPU's pipeline and jumping to another location. This is gonna be more significant when we talk about doing analytical queries but I'll just give a quick preview of what this looks like here and help you understand what I'm talking about. So to say we have a simple SQL statement here we're doing join on table A and B and then this is the query plan for it and the two-plot-of-time approach and this is what pretty much every single database system uses, Discordian system. This is also called the volcano model. We I think we covered this last semester but what happens is you sort of start from the top and go down and call next from one operator to the next to start moving data up from the bottom. So you would start here with the projection operator it would call next on the join the join would call next on the filter the filter calls next on the access method on B it gets one tuple and then moves that up up the tree and does all the processing on it. So two-plot-of-time you're doing sort of a top-down approach iterating over a single tuple at each step but now we said if everything's in main memory this is really slow because now you're doing for every single next call that's another function lookup. Yeah, you can try to inline everything but that's not easy to do. So other approaches you wanna possibly use is the operator time approach and this is the bottom up approach where you start at the bottom and each operator generates or materializes its entire output and then pushes that to the next guy here. So there's no longer you have this sort of this forward calling next, next and next over the bottom guy this getting to get some folk and just takes everything and shoves it up. Again, you avoid the amount of function calls you have to do. So this is how we actually implement this in h-store and BoltDB. Again, this is okay for OLTP because in OLTP workloads you're only grabbing possibly one or two tuples at a time. So you invoke this thing and it only shoves up one record, right? If you have an OLAP query that may wanna scan like a billion records then that can be problematic because now you're taking a billion things and trying to move it up whereas the next approach is better. Or so the tuple at a time approach would be better. The sort of the best of both worlds and what's used in more modern systems is the vector at a time approach. This is still going from top down when you're calling next, next, next but the difference is when you call next rather than getting one tuple you get a chunk of tuples, right? Sort of seems obvious but not everyone actually can do this in their system. So this is the approach we use in ours and our system which will come up later on when we talk about query processing. But again, the key thing to point out is with this approach before when we were going to disk that next call was not a big deal. And actually to the tuple at a time approach is easier for humans to sort of reason about and debug. Whereas these other approaches only really make sense or you need to do if you're not bottlenecked by disk and you're bottlenecked by other things. All right, sort of keep going for logging recovery because we're in memory when people don't like to lose their data and therefore we still maintain a log so that if we crash we can come back and load everything in. So for this we're going to use the group commit protocol or technique where we'll back to a lot of log entries together and do it a single F sync to write them out all at the same time. This is to amortize the disk call or the disk write across multiple transactions. So this is not actually not specific to an in memory database, disk-oriented database to do this too. They bring it up in the paper because the survey because it was written in 1992 and I think the group commit paper came out in like 87 so it sort of seems obvious to us now but back then this might have been groundbreaking I guess. And we'll see this later on as well that because if we use different kinds of concurrency code policies other than sort of straight to phase locking it may be the case that we don't have to maintain all the same information we have in right ahead log for a disk-oriented system for an in memory system. So what I mean by that is maybe the case we only need to keep track of the undo information or sorry the redo information for transactions and never the undo. So this reduces the amount of data we have to write because again the idea is that when a transaction commits we would never, when we know transaction commits and we write out the log entry safely to disk we will never have to go and undo it because there aren't any dirty pages for us to reverse if we have to come back after a crash because the checkpoint will always be consistent. So in some cases we'll have a more light-waking logging scheme because we're in memory and because we get to choose other coerchical models. For checkpoints we're still gonna maintain those as well and again the idea here is that that you need to maintain a checkpoint or you take checkpoints every so often so that if you crash and come back you don't have to replay the entire log to restore the database. So the checkpoint allows you to sort of say here's a snapshot of the database and only recover the log after that point. So within memory databases there are some techniques that we can use that allows us to do this more efficiently and we'll see some ideas from Facebook and other things later on but one sort of simple idea is that because I'm in memory all my data is in my processes address space I can just take a, I can just call fork in the OS and that'll spawn a new process that has the exact copy of the contents of memory of the parent process. So now that fork process can just be in charge of writing out the snapshot while the original process can continue processing transactions. So there's some systems we'll talk about that actually use this which I think is actually kind of clever. So basically there's things we can do that are easier for a memory system that you couldn't quite do with a disk based system for checkpoints. I sort of mentioned this before but we eventually want to relax the constraint that our database has to fit entirely in memory and this will happen later in the semester. The basic way to think about this is that if you want to support hybrid workloads meaning you want to do fast transactions and analytics all in single database instance usually what happens is there's always a portion of the database that's considered hot and that's where you're getting all the updates and all the modifications to, right? And then there's be a larger portion of the database that's considered cold that's not likely to be modified by a transaction going forward but you still want to maybe run analytical queries on it, right? You see this all the time. Think about like if you use Reddit or Hacker News or whatever. Most people only look at the posts or comments from the last one or two days. It's very rare that you actually go back and look at posts from 60 days ago, right? But Reddit probably runs analytical queries to figure out what people were doing across the last 60 days so they may want to run analytical queries that look at these older things. So in a larger than memory, if you want to add larger support for larger than memory databases, typically what you'll do is you'll take the hot data that you're still gonna want to modify in transactions and that'll be in DRAM and then you want to move some portion of the database that's cold out to either non-volatile memory, SSD or a spinning this hard drive. So we'll talk about how to do this later on and it's gonna allow us to have again larger databases without bringing back all the slow things that we just talked about we could avoid from before. So you may be thinking like, isn't this just the same as a disk-oriented database? And I would say no, the difference here is that in a disk-oriented database, the database starts out on disk and you move hot things into DRAM. What we're talking about here is that the database starts out in DRAM and you move cold things out the disk. This may seem like I'm just sort of like, I'm splitting hairs or just a semantic difference but there are key things that you can do differently if you follow the second approach rather than sort of a disk-oriented approach. And again, we'll discuss this later in the semester as we go along. We'll also talk about non-volatile memory. So this is a sort of newer storage devices that are coming along that they're gonna have the same read and write speed of DRAM or nearly the same read and write speed as DRAM and they'll be byte addressable. So we don't have to worry about addressing things on a block base on a block basis as you would in SSD. But they're gonna have all the persistence guarantees of an SSD. So think of this, you would have this magic storage device that you can put into your DRAM BIM slot and any time you read and write to it, it's considered durable. You pull the plug and everything comes back, right? So there's a couple, these are sometimes called storage class memories. There's a couple of different technologies that people have been posing for a while. Phase change memory, memorysters, Intel calls their new device 3D crosspoint which is whatever a terrible name but that's okay. And so it's been for a long time people have been saying these things are coming out, right? And every time they say it's like, we're pretty close. They always say, yeah, it's pretty close but it's two years away, right? So when I first got interested in these kind of things I was in grad school, it was 2008 and HP announced the memorysters were coming out and they said like, oh yeah, in two years everyone, every machine is gonna have this, right? 2010 rolls around, they say, oh, we're still working on it two more years, two years after that it's two more years, right? So it keeps going on and on and on but I would say when these things actually come out I think this is actually gonna be a big game changer for database systems because now you can sort of get all the benefits you want but you have an in-memory system but not may have to do all the extra things you have to do to make sure that your database is durable if you lose power, right? And so we'll talk about, we've been doing a lot of work here at CMU, we'll talk about some of the papers as we go along that are designed or systems that are designed for when these things actually finally come out. So supposedly Intel is gonna have the first release on the 3D crosspoint this year but the first instance will be just like a PCI Express SSD card, right? The thing that's really interesting that I said will make a big difference is the ones where you can actually fit it to the DRAM, the DIMM slot that's probably next year, actually I'm gonna guarantee it's next year but after years and years and then saying, oh, it's coming, it's coming, don't worry about it's coming and it never comes I think we're actually very, very close for two reasons, one there's now an industry consortium that's standardized what these things are actually called and what the form factors actually are and then the other key thing is that both Linux and Windows have added support for NVM into their kernels as of this year, right? So that's a big deal and I guess actually the last thing is when Intel updates the Xeon instruction set next year, actually probably this year, they're gonna include the explicit instructions you need to do cache line flushes out to NVM, right? So we're getting pretty, pretty close but it's probably gonna be very expensive at first. But so we'll talk about how you can build a database system to take advantage of this and I'll say too is also that I can't prove this but I believe that if you have an in-memory database system that it's not gonna be that much work to convert it over to NVM, I mean it's gonna be work but not as bad as it's gonna have to be if you wanna convert a disk-oriented system, right? Because in the case of the disk-oriented system even if your DRAM is persistent, you're sure gonna have that buffer pool that was super slow that we talked about before. So I would say when these things finally come along, the in-memory guys will be better positioned than the traditional disk guys. Okay, so any questions about this? All right, so again, this is sort of a crash course high-level overview of in-memory databases and again it'll set, it's sort of setting the agenda for what we're gonna talk about this entire semester. Okay? All right, so for project number one or actually all the projects in this class are gonna be based on Peloton which is this new in-memory hybrid database system we've been building here at CMU. So a lot of the papers that you've been assigned in this course are actually papers that we wrote based on this system here and I'm not just doing this because I think my papers are awesome, even though they are. We sort of, it'll cover the key things that I think are gonna matter that you need to have in a modern in-memory system and hopefully we'll be able to read the paper and then look at the code and say oh, this is the thing they talk about in the paper and here it is in the code which I think is actually kind of cool. So the key architectural components that you should be aware of now but you'll learn more about this as we go along is that we use a latch-free implementation of multi-version concurrency control and the paper you'll read in two weeks or actually maybe next week is our sort of survey paper on all possible variants of in-memory multi-version concurrency control and the idea was in this paper was there's a bunch of different ways to implement this. We didn't know which one was the best so we implemented all of them and then whatever one turned out to be the fastest that's what we actually put in the system, right? So we use NBCC, we have a lock-free BW tree index as internal data structure and this is a big deal because this is actually, the BW tree was actually invented by Microsoft. They released a paper about describe what it was but they never actually released the code and so we have actually a full implementation of the BW tree which I think is really good. We have a tile-based store manager that allows it to have a row store and a column store in a single system. It's multi-threaded so you can have multiple transactions or queries running at the same time. We don't support intro query parallelism so we can't take a single query and break that across multiple threads. This is sort of a remnant of when we started off using Postgres because the Postgres does this although the newer version they changed it but the idea is that when the query comes in it's assigned to a thread and the one thread executes it from beginning to end. So hopefully we'll fix that later. And then we support right ahead logging and checkpoints. I'm not sure whether this actually still works but for your project this won't be a problem, at least we'll get go. So we currently support some of SQL 92. Last year we had more because we had Postgres at the top. We ripped all that out and we're in the process of adding these things back. So we can support it in the actual execution engine. It's just that we don't have support for it in the parser to generate a query plan that actually do what you wanted to do. So maybe the case when you try the system out and you try some random queries in it'll come back and say an error like it can't process SQL, can't parse the SQL or something like that. And that's just because we can't handle it just yet. All right, so for project number one what you guys are assigned to do is implement the extract function. Extract SQL function. Does anybody know what that is? It extracts the b. Exactly, yeah. So it's pretty simple, you're given a timestamp and you say I want to extract the day, the week, the year from that timestamp and returns back integer giving you that information. So it's not meant to be super challenging, right? It's really meant to be just for you to set up the environment, get the thing running on your laptop and understand what's going on in the source code and build and test it, okay? So I can even give a quick demo of everything. So I can start the system up, it's running here and then the great thing about what we can do is, can I read that or make it that bigger? Is that good? So the great thing about what our system can do is that you can actually connect to it with the Postgres command line client, right? So we speak the Psequel wire protocol, we speak the Postgres JDBC protocol. So you don't have to worry about some hacky client to run this thing. You can just use the Psequel one. So the instructions on the website how to actually do this, how to connect to it. But just do a quick example of what we're talking about when we say extract. So let's say that we load in a simple database, right? Now, so in this case here I created a single table, that single table has one attribute, it's a value that's a timestamp and then I load it in a simple record. So now I do select star from the table and I see my timestamp, right? So now with the extract function, the basic idea is that you say extract day from the attribute that we're looking at from our table, right? And it returns back that it's day two, right? So I go back to the original value, oh, every time. I complained, they'll take care of this. All right, what you would see is that the query is just running, awesome. And it's almost like exactly like an hour too, like I turn it on, it's like an hour and five minutes. All right, so let's look on back and I'll talk more. So the, let me say then, so we're, I'll post the slides up for this right after class, but we're gonna provide you with a basic C++ unit test that tests some of the date part types you have to support. So the test will say, you know, extracting hour, extracting minute, year and things like that, but you'll see in the instructions the full list of all the date parts you have to support. And if you don't know what they actually mean, there's a link in the project specification that'll take you to the Postgres website that'll say exactly what they all are. We took out any of the ones that can get weird when you have time zone issues, right? So it should be pretty straightforward things, right? If it's January 1st and you ask it what month it is, it'll tell you January, right? Doesn't matter what the timestamp actually is. So we're also gonna have a SQL batch script that will do a bit more testing, but it'll show you how to run these things from the command line if you wanna open up the terminal. And so you can load that in and then do a diff with the expected output to see whether your implementation actually matches. All right, almost there. What sucks is I know much of you guys pay for tuition and it's like, oh, come on. All right. So the tests we're gonna give you guys, it's not exhaustive, it's not thorough. So we're not gonna be checking all possible corner cases and all possible date part types. So you're gonna have to write your own test cases to make sure that you actually do this correctly. And then when you submit it on Autolab, we'll run the full test that checks everything. We'll also run a speed test to see how fast your implementation actually is. And then I think there's a way to rank you with a leaderboard to say whether you do the fastest or not. We're not gonna grade you differently based on if you have the slowest, but from personal pride, you don't wanna be the slowest, right? And she said real quick, oh, because I have to hit this? All right, yeah, so that's the example I was showing before, and again, I say extract day from value and the day is the, oh, that's actually wrong too. All right, that sucks. Let's try it. It might be hard code for something else. Let's try hour. 12, all right, got that one right, all right. All right, so, all right. The other thing, so I'm gonna show you real quickly. This is, so when you build it, I think this smaller thing actually, that didn't work. This is the date function's test. When you build the whole thing and run make check, it'll automatically make this executable for you and then you just invoke that and it'll tell you at the bottom that you passed all the tests. And you're seeing here that when you look actually at the code, there's a log debug statement that it'll tell you that you're passing in what date part type and then what date timestamp you're looking at. It tells you what line of the code you're doing that on. So, that'll help you again to debug and try to understand what's actually going on. Okay, right, so again, it's strongly, I mean, you have to do this because we only give you some of the tests. You're gonna have to write your own tests to check to see whether your thing is actually working correctly. All right, so the way again we're gonna grade you is that we'll run tests beyond what we just gave you but we're also gonna run Valgrind on your code to make sure that you don't have memory leaks. So, in the instructions on the project specification, there'll be a one line command that you can put in your bash terminal, whatever, that'll invoke Valgrind and then it'll Valgrind will tell you whether you're leaking memory or not. So, we're giving you a clean copy of the repository that doesn't leak memory. So, if there's any issues that come up, it's your code, not ours, okay? All the source code also must pass the Clang Format Syndex Checker. So, again, there's instructions on how to do this. Basically, this is a script you can set up so that as you write code, it prevents you from writing ugly code. So, make sure all the code looks the same. So, just as much as I hate M-Map, I also hate tabs, right? So, this thing will make sure that you can't write code with tabs, okay? So, as I said, on Tuesday, as far as we know, Peloton only builds on 64-bit Linux. So, we're gonna give you guys a vagrant config file that will automatically download Ubuntu and install it on your laptop to do all your development. Some of you have tried to get it to build on OSX. We're looking into it. It may not happen right away. So, you probably just wanna end up doing, you know, making sure that it runs on Linux. You can run like C-Line outside of the VM and have it use like a shared directory and point into inside of it and just do all the compilation there. So, if you wanna use a better IDE to do your development, you can do that. There's instructions on how to do this in Eclipse. Eclipse is terrible. I don't recommend it. C-Line is much better, but I haven't had the time to actually set it up. So, hopefully we maybe have some instructions for you how to do this as well. Again, I'm assuming that everyone has access to a machine or a laptop they can use, do this development on. This is CMU, so I'd be surprised if you can't find one here, but if you don't have a machine to do this assignment, please come ask me. Just talk to me, okay? All right, so the due date is January 31st. All the projects will be turned on Autolab. This will be the URL for the project. It's currently not online, but as soon as class is over, I'll push everything up. Autolab should also be turned on soon after this as well. And again, there's I think nine free slots on the wait list and we will pick people off that in the order that they submit the assignment and pass all the tests, right? And you'll be added to the course, right? And then probably what I'll do is, I mean, to be, how should we do this, be fair. I will announce to everyone when the all the nine slots are used up, right? Rather than say this person got in, this person got in, I'll just say at the end, it's done, okay? Any questions? So my parting thoughts for this, for this class, disk ordinance systems are relic of the past, but they're not going away anytime soon, right? Certainly my SQL Postgres and Oracle are more widely deployed than any in-memory database that I know of, but I think that slowly over time, they'll go away or they'll have to change their architecture to be more main memory oriented. When I first started grad school in like 2007, people were actually really uncomfortable with the idea of having an in-memory database, right? In the early, I remember going on this like sort of, I guess sales call, whatever you want to call it with, for both the B with like Stonebrick or we went to go visit a Silicon Valley company and they were like, well, we're not sure we want to be storing our database in memory. That sounds, you know, terrible, right? But now I'll start in the case that this is not an issue anymore. People sort of are more comfortable with this and it's just a matter of whether you can afford to have a machine that has enough DRAM. All right, and again, if you don't remember anything of this course, remember that you never want to use ML for your database. When I die and I have a tombstone, this is going to be at the top, right? So don't use ML. All right, so next class. Next class, the paper you'll be reading is a survey paper over running currency control on a large multi-core or mini-core machine. This is not a realistic scenario, like there's no machine with 1,000 cores, but the idea of the paper is that it's going to sort of really stress test these different protocols and you start to see what the bottlenecks actually are, right? And then from going then we'll talk about more specific algorithms after that. All right, guys, thanks. And I'll post project one right after class, all right? See you Tuesday.