 Afternoon, hopefully everyone's still awake after lunch As he said my name is Graham McAllister. I'm an actual engineer. I try to do stuff at least, you know, two or three minutes a week At least that's what my team thinks I do But I work on the RDS service. I helped founded a few years ago. I think it's now like seven or eight and Very happy to be here talking about Postgres. It's kind of my favorite engine I spent a lot of my time on it and This talk will kind of cover things that we have announced in the last year including our Amazon Aurora which I'll get to in a second as well as some of the lessons We've learned helping customers run their databases on RDS. So let's jump right in so as I said one of the big announcements That we made in the last year was our own Version of Postgres called Amazon Aurora with Postgres SQL compatibility. That's the official name I don't call it that I call it Aurora Postgres because I don't like to talk that much But essentially what it is is it's a Postgres 9.6 compatible engine So on a wire from a functionality perspective, it's going to behave exactly the same as RDS Postgres same extensions You know everything will work the same, but but the difference is that it's cloud optimized and it's log based So a little picture over on the side is essentially showing you Well that we've taken sort of the bottom half of the engine where normally Postgres writes to files and change that into a log Structured kind of storage mechanism, and I'll jump into a lot more details later on in the talk One of the key benefits is that we do six copies across three availability zones So a higher level of durability than we've been able to provide before Up to 15 read replicas with very low latency on those faster failover down to less than 30 seconds enhanced scaling Show you some performance numbers and the other one of the cool ones is that it'll auto scale your storage up to 64 terabytes without you having to go and add more storage or do any of that So like I said, I'll cover that as we get we get further into the talk On the RDS Postgres front we released support for 9.6 Shortly after the community did we're just prepping right now to get the new minor releases 9.62 9.56 and some of the older ones out which includes some patches But the other thing that we do when we release these is we actually typically add some new functionality In this case we added some extension support on both of the last minor releases So bloom filter and PG visibility, which were just core 9.6 Extensions we added support for those in 9.62. We added this new one called log FDW And I'll get to that in a second because you probably never heard of that one before PG hint plan which we had a lot of requests for especially out of our Japanese customers and PG free space map Which again was a standard one that we just didn't have in the original one So when we started we had 32 Extensions with RDS and then we added a few more on 9.3 and then 39 on 9.4 9.5 had 44 9.6 now has 49 now in the future We're gonna have a lot more and how do we decide on what a lot more is well when people talk to us at conferences They send it to this email Mailing list we basically look at those and then we talk to folks and we try to figure out what the priority is So we've had lots of requests for things like PG repack PG audit just to name a few So if those are things that are interesting to you, please let us know like you're gonna come talk to us at the booths and me emails and you know Because that's what we do We're trying to focus to make sure that we're trying to provide the extensions that we can now we don't do on safe extensions like PL Pearl you Because of the management structure that we put around RDS, but anything that's not like that There's no reason why we you know if we've got enough customer demand that we'll go execute on it So the new one that we added which we actually developed and I say developed it's kind of in quotes because we took basically the file FDW and cut it down to make this thing called log FDW And what it allows you to do is have database access to your Postgres error log So if you set your log destination to CSV, right? And you go create this extension just like you would any other ones you create a server Basically, you can select from list Postgres log files You get that nice list and then you select the one you want and you do the create foreign table for log file and Once you've done that you can see you can select stuff, right? So you can go now Real-time look at your logs and say hey, you know who was can it like I turned on logging for connection So I could see who is connecting to my database for example, right? So this is just a different way than using our API method to get the Postgres error logs out Which you know for folks that have been using my sequel this has kind of been a feature that they've had for a long time And so we had a lot of requests to do something like that. So one of our developers basically just knocked this out I actually had a question about this feature even though we hadn't released it by someone who said hey I heard you're doing this. What's the story? Do I have to use CSV log? I don't necessarily you know turn that on all the time. I said no no you can do it without You just you know specify one of the log names. It's not a CSV log The only thing is of course you get one really long long line, right? You don't get any kind of parsing, but you can see in my select statement. I can still do likes where I can say hey Show me something like connections. It's just gonna you know take a little more to get that out The other one we added like I said that the Japanese customers really liked was PG hand plan a lot of them are moving from Oracle today They're moving to Postgres, you know, they've got an application where they're trying to get things to run And they're having a little bit of difficulty So they wanted this extension I'll show you sort of an example in a second once you load this you have to add it to your shared preload libraries Because it's that kind of extension and then once you do you can you'll see these extra guck parameters And I won't go through all of them that you can set to basically control the behavior of this This extension. What does it look like in practice? Well, let's say you're doing an explain on you know PG bench which everybody does and this is just a join between the two common tables Normally, you're gonna get this kind of explain, you know hash join with this is sequential scan So nothing too complicated. Let's say you're like I hate hash joins For some reason you're like give me make me make it a nest of the loop because that's better not really but but What you see is you can just put this hint in and say I want to nest the loop on a and B and now The planner is changing to do something else. So, you know, obviously you got to you know You know pick the right thing here, but it does allow you to override some behavior also allows you to set some different behaviors as far as Gucks inside of one query. So it's actually kind of a very powerful little extension So that's going to be available in the latest miners for us That's just highlighted what I just said One of the things of course our customers depend on us is to allow them to move up to new major versions So we have a major version upgrade that you can do showing sort of going from nine five here We take a backup when you ask us to do this We run PG upgrade for those of you familiar with it We take another backup and then you get a production nine six instance The reason why we take those two backups for anyone that's not familiar with PG upgrade. It's not transactional So there's no point in time during it, right? It basically is moving around data files and doing some really kind of squishy stuff that you know You want to be able to say hey if it worked great if it didn't I want to be able to go back to where I was right? And this works on a lot of instances, but there's some really funny stuff that you can do like defining views on Catalog schema that then changed during a major version upgrade and then PG upgrade kind of goes I don't like that I'm gonna stop so there are a few things So that's why we recommend not doing this But actually of course building a test instance, which is really easy because you just make a you know A copy of your RDS instance running through the whole thing again Running your application testing making sure that works and then going to do production, right? And this is one of the cool things about RDS is you know doing this extra step Well, maybe you know cost you 20 or 30 minutes of your time to go verify I mean outside of the application testing that it's not going to break when you're gonna do it to your production system So obviously a big thing for our customers, especially you folks that you know are in sort of the financial industry Security is a big deal if you're coming from places like Oracle, you know, this has been a bit of a richer system At least at the database level we've tried to sort of add pieces that allow you to do this at a lower cost So we got the typical application of us database instance snapshots of your data backups basically plus your logs, right? We started by adding first we had the security group that allowed you to control who could access the network was okay Of course we support SSL And then we added the virtual private container cloud kind of thing, which is BPC Which really allows you to have things like ACLs and full network capabilities It actually is better than what most people run their own networks, which are typically very flat To really control access Then we added encryption at rest with a default key or our own key through our KMS system So this is for your snapshots your data files your log backups. That's all protected So at this point you kind of it looks good except for that then customers came back and said well When my people say SSL, you know mode disabled on their clients It does this and now the encryption story is kind of gone, right? So we added an extra piece called New guck called rdsforce SSL when you turn that on by sending it to one it essentially You know causes now if someone sets this they won't be able to connect to your database So you can say to your auditors look i'm fully encrypted no one's going to be able to talk to my database No one's going to be able to look at my data. It's it's all good So this was nice from a running database perspective But we have cool features like snapshot sharing and when we first launched this it was for unencrypted stuff allows you to basically You know share a snapshot with an account And that person can make a snapshot of that they can make an instance They can even make a public one, right? It's all great unless you of course need to encrypt your data And then none of this is applicable So what we did was we rolled out encrypted snapshot sharing Which works similarly, but slightly different because now you have a key involved Now if you go to rds, and this is one of the things, you know We always kind of mess up a few things one is that to make it easy We made it so that you can do encryption arrest with a default key. You don't have to go create a key The problem with that is when you talk about sharing things Since that default key may be shared may be covering many many items We don't actually allow you when you say i would like to share It says uh, no Because you may unintentionally be sharing a lot more things than you meant to by sharing that key out So you actually have to go create a custom key And that's what we recommend when you do encryption arrest with us is to actually go ahead and spend the time to Do your own custom key through kms So once you do that there's basically a two factor here One is that you need to Share that custom key you add an external account to it And then you actually need to share this snapshot So you can actually divide the privileges up in your company and say bob has the ability to share snapshots Steve has the ability to you know, share the keys and that way, you know You don't have someone making a mistake of being like, oh, we just shared all our snapshots to you know our competitors. Whoops Um And once you've done that it works exactly the same way that you can create snapshots or instances off of it So this was good and well received The next thing our customers wanted was they wanted this to work for our cross region replication So for dr purposes, so we're showing here kind of our classic configuration with our multi az high availability You can now with an encrypted instance with a key Basically create a replica in another region all fully encrypted both, you know on the wire and storage again, right? And if you go and promote that guy on the new region, it'll actually have its own independent key So they are actually separated that way The other big news that we announced in november was uh rds postgres becoming a HIPAA eligible service So we've had a lot of people in the health care industry really interested in you know, using rds that they couldn't before this And the other one is that we have fed ramp in our aws cloud cloud region for folks that need that as well So those are you know, sort of long time coming Things that we need, you know to get through the compliance stuff Data movements another one that a lot of customers are talking to us about Whether it's getting off of a current Vendor that they're not happy with or just moving data around between databases Which is kind of common now. We like to do that a lot um So a while back we announced the data migration service Or dms as I like to call it because I actually don't like the m there because I really think of it as a replication service that does migration but You know, I don't win all the naming battles. So uh The you can see all the engines that we sport up top And so these are you can basically go from let's say postgres to postgres So you can go oracle to postgres or postgres to oracle, and I'll talk about the details there But it's a really powerful tool and then it does change data capture And so how does that work? Well, if you were doing a migration for example, and you were on premise You know or even in ec2 you could set this up And you basically go and configure a database migration service instance much like an rds instance You tell it what tables or what connection points you want What's your source and destination and then what tables you're going to use And then you basically go do a full load and this does a consistent select of all your data Sticks it in the new one And but the key thing that's next is that it does change data capture Then to pull from the logs and catch up so that once you're caught up All you got to do is stop writing on the customer premise side and switch over right? So you can't it's not zero downtime, but it's very short, right? We've had people do it in less than a minute So How does this work? Well, you can be on premise you can be an ec2 You can be an rds for those versions that i've specified Now it's nine four and above because we do use the logical replication capabilities of postgres the cdc there So, um, you know, if you're not if you're on premise and not at nine four Then probably the thing you do is upgrade to nine four and then move in right? um As I said about the bulk copy is a consistent select And then we're just using the logical replication and that's the url for that team and actually we have One of their team members is here at the booth. So if you have other questions about this Or some of the other stuff I talk about he'd be happy to answer those The other part that we heard from customers is they actually wanted to migrate from one engine to another So it came up with this tool called the sct the schema conversion tool So it's a downloadable tool that you can use to go basically look at what it takes to convert from one engine to another And What it can do first of all one of the cool things It can do is it can tell you how off you are in doing this like how much work So the green there for example is the stuff that's just going to Transparently move over the red stuff is on the far end where you're probably going to have to do some work with your application Or change some major stuff to make it happen. So some people, you know Asked me well, should I go to my sequel or postgres? So I'm coming from oracle. I said well one of the things you can do is you can go run the sct tool For both of those and see how much work it is for either engine most cases postgres being the richer engine You know has less red on it right So that that's kind of nice in detail like here. I don't know how well you can see this But we have an oracle procedure that we're moving to a postgres function And it would tell us, you know, it'll actually do the conversion for you once you get to that stage The other thing is as part of getting dms to do logical replication support We made generic logical replication support for rds possible. So same versions that I talked about before All you have to do is set this rds logical replication parameter one Which basically changes the wall level to Logical and a few other things and then you got to create a slot so that you start capturing the data And that's what that command and then with pg to receive logical You can just start pulling the data now. There's getting to be all kinds of different ways to connect to this Like the jdbc driver does it now. I think Dave was saying maybe the python one two So this this is kind of a new upcoming area One of the things that that is kind of odd today is that we only support the test decoding plugin I'd be really happy to get any feedback now or later on about what other plugins people are interested in if they're thinking about using logical decoding Now, how many people are familiar with logical decoding in postgres? How many people know when you make a very very large transaction? That it has to be kind of assembled out on disk Before it gets pulled to the other nodes Yeah Couple so yeah, so one of the things that we did was we added a metric so that you could actually track this because You might run out of disk space and you might not know why and it's because you're basically You know materializing this thing from a physical wall to the logical And and uh, and so you can now monitor that as part of the rds solution And of course we have the normal replication leg metrics and then the other one we added was The even though you're creating a slot for logical It's just the same as a physical slot like we use for cross region replication And if you're not pulling you're going to be using up additional this space here So we made another metric so that you could alarm on that in case someone's like Creates a slot and forgets about it and you wonder why you run out of you know space on your database, right? Which is never a good thing So one of the cool things is really talking about what you can do with this right So let's say you have an rds postgres instance You can set up dms and you can replicate to another rds postgres instance You could replicate to redshift. They actually have a very optimized loader for redshift where it goes and does it in chunks Which is what redshift likes as opposed to a single line insert Um, you could go back to on premise Uh, you can go to ec2 You could go to s3. We just announced this functionality for dms You can now do some portion of what you capture coming out of the change data capture into s3 directly Um, you could go to ec2 oracle We have some customers who are in the process of moving from oracle to postgres But still have other oracle databases that they need to fade data back to so this is a way to do that And of course as i showed you could have your own custom logical hender using pg receive log or the jdbc driver Driving let's say a no sequel database right and putting data in there so One of the big lessons that we've seen working with customers is Around vacuum right and this is one where How many people have ever had transaction id wrap around on one of their databases? A few right It's not one not something you're gonna ever forget Because it's painful. Uh, what it forces you to do is to go into single user mode until you vacuum that table Um, and until then guess what you're down, right? This is not a good time for you. Your boss is probably going to be in your office at that point Asking very difficult questions about why this happened. So there's a lot of parameters that you can set and I won't go through this I know um As we just did a talk on vacuum and you know, it's probably much more in-depth So, you know for the details on that but For us for rds one of the things that we noticed was we weren't giving enough information back to the customers So they could see all the vacuum. So we made some changes The first one here was to allow uh logging of the vacuum commands So you can now set the force auto vacuum logging level and this will now spit out things Like this where it shows. Well, we had to cancel an auto vacuuming task because of a lock not being available, right? So if this was the table that you know kept your transaction id going up You need to go kind of figure out why you have this locking problem, right? Uh, the other was seeing what auto vacuum was doing Uh before you get this rds admin is running that command you get insufficient privilege Now you actually see what auto vacuum it's running. So that's kind of helpful. Just if you're like, what's auto vacuum doing at the moment, right? But the big one that I really was actually really happy that the team came up with was this metric around maximum use transaction id And so this is out of the space that is the transaction id space, right that you have It's you can see with now that we have this metric how fast it's growing by kind of the slope of the line And then how close you are to running out and you can set alarms on this Now we monitor this ourselves So if a customer gets way too close we'll probably reach out to them and be like There's a problem because we don't want them to go into a single user mode situation, right? But From a general safety perspective, this has been really helpful because customer said, you know Yes, I could have monitored this but now it's much easier. I like one click set an alarm on this And if I hit a billion, you know Someone in my ops team is going to get a page about, you know, figuring out why we're not vacuuming keeping up. So So that was a big one around vacuum So scale and availability is always a hot topic for a lot of our customers. I like to spend some time on that One of the ones that we do on a regular basis is we introduce new instance classes And this is just showing the m4 and I just picked the m4 large is kind of one of the smaller ones And around a pg bench read only all in memory just to kind of show the cpu power that you're getting Transactions per second on the vertical axis Larger is better, right Blue is the old m3 large, green's the new one And as you can see we get better performance With the new one and that you know, that's good, right? It's the 37 percent You know increase But the cool thing is it's actually a cheaper instance class So your you know price performance is actually increasing So this is kind of the regular cycle that you see with us is that we you know Keep doing new instances and it's funny because you know, my my goal is to have you on the latest instance I actually don't want to support all the old ones. I'd like everybody on the new one So I'm always happy that we're doing this stuff, right? and With with a roar we'll be starting with the r4s and you'll see those roll out as well across the fleet over time One of the other things that we had around sort of performance was customers saying, you know I like the idea of rds, but man. I really like being able to touch the metal. I like being able to run top I like being able to run sar and we said well from we can't really allow you to ssh into the instance That's going to be problematic. So we came up with this idea which is enhanced operating metrics, right? So we basically give you all these metrics and you can get down to a one second granularity on them, right? And you can even pull this stuff via api because it's in our cloud watch logs Service so Any of these can be used to look at the system And then the other cool one is that we actually do a process list like you have in top, right? And again because this is api driven you can actually look at the like this on a one second basis Like back in time for like days if you wanted to so that that's kind of interesting Like here you can see i'm running a select query for those of you probably in the front of the room And it's using three percent of the overall box cpu, right? So, you know if you have those kind of weird runaway ones where you're like, what's using the cpu and you're like Oh, look, you know bob's running his select again. Oh great Sorry if there's a bob in the room And then of course the nice thing is of course, you know people like graphs a little easier to see Let's say you have you know your boss goes and says what happened there, man Why didn't our cpu, you know go off the wall and you're like well I think someone might have started up a whole bunch new processes accidentally, right? So it makes diagnostics a lot easier when you kind of see it in this kind of pain view So our customers are really happy about this, but they said well, that's all great But what about my database? What about the queries running it? And that's where this new feature that we're doing called performance insights or pi for short Is in beta along with the aurora or in preview along with the aurora a postgres And that's where we're the first engine we're starting this on But it's going to roll out across all our engines over time And what it does is it really allows you to look at by weights as you can see sequel hosts or user What's going on in your system has what statements are being run You know, who's running them where they're running them from So and we also look at things like capturing plan changes. So you'll be able to say well, look I had a spike Oh, there's a plan change at that same time. That's the problem or you know These new machines fired up and they're starting to you know abuse the system by doing something, right? So and this is just a different view of that where I'm just doing it by weights Um, so in this case like I was running pg bench on a very small, um Size scale one right which causes a lot of contention on the smaller tables And that's what you know, you can see with these a couple of weights showing right So we think this is going to be really helpful and you know, we're going to be developing a lot of features in in this area So aurora postgres well This is our multi az product before aurora. It was you know, you had a primary you have a multi az secondary It's not readable. It's cold, right? It's highly available because it's synchronous replication cross to availability zones We allowed you to have read replicas and in some cases people just use these if they don't have high availability requirements But you can kind of add them for extra read scaling Um, and it's asynchronous. I mean, that's just standard postgres streaming replication And if you can divide your database up and your application so that you can you know do these Eventually consistent reads you can build this nicely scalable system And we have a lot of customers that do this and it works great But customers ask us to basically improve on this and that's part of the thing that we've done with aurora postgres Because as much as this system works, you know, the primary fails And you can fail over to the secondary you notice there's only two copies, right? And some people say look my kind of industry I need more protection than that so You know this works it does lots of good stuff You can you know keep your read replicas up during upgrades you can modify them So you can build it to be highly available, but it's a little more complex than some of our customers liked So with aurora, it's a little different model Essentially what you have again, you have the application you have like a database host that you connect to But now we have storage that's distributed across three availability zones and we do six copies And I as I have there it's a four of six synchronous writes model and I'll get into the details on that But as you can see we just write to all of them at once Now this is an interesting problem because Typically, oh, I messed up my ordering. Okay, so for aurora what we do We send these off all at the same time And we start getting responses back And as soon as we have four of them, we're basically finished We don't have to wait for the other two to happen now they do happen But we get rid of a lot of jitter on this system And to look at like what that looks like if you're not doing this model This is an early test we did way back before starting aurora where we tried to look at What if we just made our multi z product write to more places? And so we compared sort of two nodes or two locations really from a network perspective with four disc copies to a Three location six disc copy Latency on the vertical axis so smaller is better in this case And you can see if you look at the percentiles of the query responses Or the these are actually disc responses You can see that we're like in the six to seven milliseconds at the 50th percentile So going to extra copies doesn't really hurt you there But if you look at the far side at the four nine so one in 10,000 basically executions Is getting a lot worse performance almost a forex degradation by adding this And for every one of those queries that you miss Right like so when you get one of those 123 It stops probably 20 of the little six millisecond ones from running right That's going to cause kind of problems with your application if you if you really need that kind of latency So that's why we did a lot of different changes for One of the other things that we did is we changed from something that had a log buffer to something that doesn't we don't do checkpointing so Here on the left side is sort of how postgres or traditional databases work as you're getting ready to commit work, right? You're just before you commit, right? You have all this queued work getting ready to go You hit commit and at that point you enter the log buffer What's cool about modern databases that does group commit allows multiple people into that buffer at once So that you can actually get some concurrency But if more work is getting ready to go As soon as that log buffer starts getting ready to send down to the disk you can't actually send more work in right So at that point You're waiting for this stuff to go down to storage and to get acknowledged before you can say that's committed And then the next piece of work can actually enter the log buffer So you can see how this is going to cause your system to sort of move a little bit herky jerky on the You know on its smoothness On a war it's a little different What we have here is as things come in work comes in to be committed It just flows down in whatever order whatever speed we don't care So we don't worry about the ordering on the top side. What we do is on the back side. We we track What's durable so in this case we have this durability tracking And basically there's the all the little transactions over on the side So when we get two of them in we know we've gotten two acknowledgements out of the six But the four we need So once we get somebody to four you can see I can send that commit back and say to the customer. That's good, right? But you'll notice that c I didn't say is acknowledged. Why is that? Because you still want your transactions to go in order, right? You don't want to be like that guy that committed after me is being seen But I'm not right even though I committed before him So since we need to wait for b to get to at least four before we say all of them are good Right and again, you can see in this case the last guy e is also not committed because d is not there But what this allows us to do is to not have to worry about Doing everything in an exact order, right? And this again changes the performance characteristics So from an actual structural perspective Multiple az's got the application you have your read write node and you're talking to this thing that I call a raw storage Right, so it's a layer that we talked to it's got a lot of different machines in it Spread across these three availability zones and then you have 10 gig chunks of data that we stripe things over And you end up with you know six copies, so that's trying to illustrate with the two colors Although I seem to some of the colors look remarkably similar But there are six there And you can think of this sort of as a distributed storage system that allows Us to repair and do all kinds of interesting stuff For example, if this one 10 gig chunk for example was a replacement because that host died Or because it was just behind because of the four of six rule We actually do gossiping between the nodes to do catch up, right? So there's multiple ways that we get you basically back to The correct state on your data now We never commit it before it's four of six, but to get you to six of six, which is where we want you to be Right, we do all this other work This allows you to have what we call replicas, but really are sort of read only nodes in this model, right? And they don't have to spin up and have extra storage like I showed you in the the current rds model So if you want 15 replicas today on rds, you have 15 copies of the data plus your primary In this you have one copy of your data. I mean it's six way copied, but one setup And you can have 15 of these But the interesting thing is we don't actually read from the storage to get to know where we are We use an asynchronous invalidation and update method where we push over to the replicas and say hey You know you that block you have in memory has changed. You need to either get rid of it or update it But the really neat feature is let's say has anyone ever done like a huge backfill on a replication system And had it go really horribly wrong So because if you go commit let's say a 10 gig change to your system that has to flow through Even if nobody cared about it from an application perspective, right? It has to flow to your replica In this model, we basically get to discard those because if they're not in memory on the read only node We just say oh look that's something we don't need to apply because we've stored it in the disk already, right? It's in our central storage system so This means we can have less lag And more performance because we're not doing unnecessary work on the replica that basically isn't benefiting the read only node The other thing that with Aurora that we do differently is we write less So again on the left side sort of classic Aurora on the right So you have this block in memory. You have your wall You go and let's say do an update you're going to end up touching a row that was the you know the original row You're going to basically delete an insert, right? So you end up with that But because let's say it's the first time you've done it since you've check pointed You're going to get a full copy of that log or that block into your log, right? So that's actually a lot of writing right for making two or some very small updates Now the nice thing is when you update that block again, you only get like one little chunk in the wall, right? So we've already done quite a bit of writing When the checkpoint comes around we have to checkpoint that block out, right? And of course we have to archive the wall So that's a lot of work On Aurora when we do this when we go do that update those two changes They basically get sent to Aurora storage as a log vector just like sort of wall And that's it And the same thing when we do another one. That's all we do. So there's no archiving that happens on your system There's no checkpointing There's no full block Logging to the wall stream because we don't need to do it because we don't have split block Problem So this means that you can get more throughput out of the system that and not having the log buffer as I Showed before So what does this look like in practice? Well, here's a pg bench doing, you know, it's sort of initialized Run, right? So you have the copy coming in then vacuum and index build and you can see that we have a Copy inward a bit faster, but vacuum and index build where we can do some different things Mostly on the vacuum. This is a lot faster One of the cool ones that I really liked And I actually did this slide. So of course, that's why I like it Is when you think about the tradeoff between recovery and performance, right? So on a checkpoint based system The more work you allow to happen between a checkpoint the less full wall blocks you're going to have to write But it's the longer the recovery you're going to have. So what I'm showing up here is Aurora with no checkpoints was able to hit 92,000 writes per second Now these I constrained Except for this one where I'm running as fast as I can. So I'm checkpointing at 12 and a half gig I'm only getting to 69,000 writes My recovery time was 102 seconds We don't have to do recovery because we're a log based structure. What comes back is already recovered, right? So there's no crash recovery in aurora. So it's 1.2 seconds And I have these two other examples to show you like if you want to get down to a reasonable recovery rate That's fine But now look what it did to your throughput by capping how much You know checkpointing size I had that then cost me a lot of performance So you used to have to make this decision between the two things of making do I want to go faster? Do I want to have you know more availability? So that's a that's a change This gets to gets to the throughput case where I was talking about not doing as much work and not having a log buffer We're able to sustain higher write rates than standard post-gres for longer and with more connections Uh, this is similar just a strictly the other one was a read write. This is just a write only And then again as I was saying about that jitter This is kind of an interesting one where this time it's response time and again lowers better the yellow as aurora So you can see the consistency we're getting Now this is partially because of checkpointing. It's partially because of the log buffer You can see that post-gres is bouncing around all over the place, right? So even though you might get some final number That says the average is x It's not what you'd be experiencing if you're running an application on it And this is what that looks like sort of drawn in a different way, right? You can see every one of those points is You know, I was going on post-gres from anywhere between 5000 and 30 000 depending on which second I'm looking at versus a very much more consistent band And then one of the other interesting ones that I didn't actually realize for quite a while about post-gres Is this whole thing about having to do full pages written to the wall after a checkpoint So if you can imagine if you have a very small working set the chance that you touch the same page over and over between one checkpoint The chance you're going to hit the same page a lot, right? But let's say you go and have 10 terabytes And you're doing updates all over the place the chance that you hit the same page becomes very small Which means every time you hit a page you're doing a full right To the wall this causes performance to decrease as you go larger with postgres So this one shows that you know with aurora at 10 gig. There's a meaningful difference, but it's not huge But when we start talking about you know 100 gig This becomes quite a large difference where you know we're three times faster and this actually just keeps sort of going up You know it slowly decreases where we don't get much faster, but it does you know as you go into the terabytes It sort of has that that problem And with that That's all I have on the Presentation happy to take questions on any of this or anything else we do Well, that's because that's because it's not being released till next week. Yeah, so That's why I had the word soon when I said when those were happening. Yeah It will be in our standard documentation. Um, it's essentially almost exactly what I did was what's in the documentation I like to copy as much as I can Yeah, so um all those things that I talked about they're going to be rolling out I think next week. Uh, so you should expect to see that then. Yeah, but I just thought hey, I'm here I'd rather tell you guys now about it instead of waiting another, you know X months till another conference to to announce it. So Other questions in the back When is aurora going official? Well, it's officially in preview We're working, you know towards a a date in the you know in the near future. Uh, I think we're Kevin, what do we say? We say q2. Yeah, sometime in q2 is is is the goal. Yeah Yeah He's reminding me since I'm on that development team. So yeah, who's it? Sorry, what's the It's before christmas. Yes, you won't have to wait for christmas to unwrap aurora postgres. Yes What will be the price difference? Uh, the price for aurora postgres is going to be the same as our aurora mysql offering so it's listed on the The page I don't actually look at the prices that much. So I can't tell you specifically in what region so But it'll be the same as like all the aurora ones will be the same from a pricing family perspective Other questions, okay Well, thank you very much