 Can you hear me now? Okay, good. So let's get going. I will try to explain something very complex in only 20 minutes. So, well, let's not waste time. So first of all, this talk is not going to be about the miraculous things that you can achieve by optimizing your server installation. It's not about extracting extra transactions or extra speed or anything like that. It's basically just going to pop up the hood and, well, look inside the engine that MySQL is. So this is why I chosen this background here. A little bit about me. So I have been, well, with this community forever, really grateful about it. It's been a life-changing event for me. I lead a team of server developers in the MySQL development team at Oracle. We specialized in security, client-server protocol, performance monitoring, and the component infrastructure that I'm going to talk about. I'm from Bulgaria and I like working from home as MySQL culture is. So that's about me. Okay, so our agenda. First of all, the reasonable question. Why do we need another infrastructure inside the server? We already have the plugins, so why? Then I'm going to go over the architecture. So basically how we are trying to address the why question. And then I'll do the inventory. Basically what services and what interfaces do we have added to that infrastructure. And then I'll try to explain how to write your own component and how to write your own service and some tips and tricks in doing that. And to honor the place that we are in right now, I'm also going to give you some homework. So whenever you see this picture on the end of the slide, this is for you to basically read through and understand. I won't really have... I wouldn't have the time to explain that in detail, but I think that the examples are pretty self-explanatory. So please, the slides are going to be on the FOSDEM site and also on my slide share. So please go over these more concrete examples. I don't have the time to go over the bits and bytes and the commas and semi-colons and all of that. Okay, so the big question here. Why? It's a historical question. So we have this great infrastructure called Plug-in Infrastructure. It sustained mass care through its initial phases. And it's really great and very convenient and very simple. The problem is that we've accumulated some scenes, some technical depth. Basically, plugins are not respecting the infrastructure and that was all done in the name of achieving goals faster and basically delivering a product. Also, I think that over the years the Plug-in Infrastructure became very complex because when you have a hammer, everything looks like a nail. So you want a hammer in and hammer out basically. And it doesn't really work like that with code. So the Plug-in Infrastructure over the years acquired some properties that are not really belonging to it, to the core infrastructure really. So I thought by just starting clean, I can avoid those mistakes and create a better layer and a better grouping and a better code division. So that's one of the important reasons, right? That same better code isolation and encapsulation, I know that's the holy grail of complex code development, but well, we should still strive to it. Another good thing is that right now plugins cannot really call each other and that's mostly because we don't have the notion of dependencies between individual plugins. We don't know if a Plug-in A depends on a Plug-in B and that infrastructure here is trying to fix it. Also, all components are equal. They can call each other and they can be called from all other components. So there is no central place of an arbiter that currently the MySQL server has for plugins. It's always plugins talk to the server and then the server talks back. Never plugins talking to each other directly, which is a limitation. Okay, so now how do we want to solve that? That's a tour of history of MySQL componentization code, by the way. So it all started with this big blob of code, which is the server binary, which has functionality in it. Later, people realized that they want to be adding code a bit simpler, so plugins appeared and server uses plugin APIs to call the plugins and then plugins use plugin services to talk back to the server. That's pretty much it until 5, 7, 8, yeah. Okay, so there were ideas on how to make this thing better, but we decided to start afresh, so in 8.0 we added this minimal chassis thing, which is basically a list of service implementations and also a dynamic loader that loads and unloads components from the server memory space, basically. Okay, obviously this is not enough because this dynamic loader is not a persistent thing. It does not track which components are loaded. It just loads and unloads components. So we added some glue code, which is basically the persistent dynamic loader. This is a layer on top of the dynamic loader that implements the persistency part using a MySQL server table, basically the MySQL components table. And we also implemented a file scheme handler, which allows the persistent loader to load components from files, executable files. And this is inside the server functionality. It's out of the minimal chassis, so that's important why we have these extra layers. And this all is called the server component currently. So we thought we should start with this basic component and close all of the current server functionality. Right, so now the server functionality can consult the registry if it needs certain functionalities. And use those. The problem is that this is part of the technical depth. The plugins can also do that directly because they have all the symbols in the server binary available to them. So in an attempt to prevent that extra acquisition of technical depth, I've created this plugin service, which allows plugins to access the data in the registry and the implementations in the registry. The idea being that one could use that too as a gradual migration device from plugins to components infrastructure. So if you have existing plugins, you can gradually start moving towards components like that. Okay, and obviously the whole deal is adding new components which are outside of the server component and they are independent. So unlike plugins, those do not have access to the server binary and all of its symbols. So they are forced into using the registry and whatever services are registered into it. And they can carry extra service implementations and register those in the registry. And because it's all in the registry, now pretty much everybody can call everybody. They all have access to the registry. They have access to all the service implementations defined in it. So basically that's how the equality of the components is achieved. They all just talk to the registry. That's the idea. So some terminology here. So I mentioned the dynamic loader. It loads and unloads components. So basically that part. So these yellow things are the actions and these other things are the object. It starts with the SQL command install and uninstall component which basically instructs the persistent dynamic loader which encapsulates the actual dynamic loader to load and unload components from the server into the server processor out of it. So what's a component? A component contains code. That's the defining characteristics of it. And this code is usually a part of it is service implementations which implement certain abstract APIs like services. We call services, component services. And these services can be a number of things. I'll show you the inventory later. But these are basically just abstract contracts between the consumer's functionality and the producer's functionality or provider's functionality. And these service implementations are registered into the registry. So the components can basically fetch references from the registry to these implementations either from their own component or from other components. So that's pretty much the terminology behind the component services. Okay, so now some homework slides. It's all written down here. You want to maybe check that later. Okay, it's basically explained in a lot of detail. Right, so the inventory. What is in that infrastructure right now? So we obviously have the registry and the dynamic loader which are services by themselves so you can access them like that. There's also the error logging subsystem. Currently MySQL uses that new infrastructure to produce its error logs. So you can basically have all of your components logging errors and so on. We support defining system variables, status variables, user defined functions which is quite handy because you don't, if you have multiple user defined functions you can have a single component and compass and call of these. And when you load the component it would automatically register all of the UDS for you. So kind of nice. Performance schema is so supported through the component infrastructure that's a great achievement so now you can also instrument your code and up your own tables that you want to instrument stuff in. Yep, security context is available as a component service. You can manipulate, well, who does your current process run as. There's password validation APIs, there's runtime error generation, there is working with collated strings inside MySQL and a lot more. It's 92 service headers so please check most of them. So how to write your own components and services? I'll give you the executive summary given the time that I have. So basically I have a bunch of utility macros if you ever written a plug in it's not very different. You can of course copy some of the existing components, plenty of examples, especially in the tests. We put tests for pretty much everything that we do. Also copy the existing service definitions because once you have your component you need to have it define certain services that it provides to other components basically. And there's plenty of examples for that, even a doxy gem section in the documentation so just go ahead and copy it all. And then also unlike plugins you need to compile it using the server source distribution. We are not yet at the point where you would take the binary and compile it. So be careful because they run in the system process, the server process and there's no boundaries, no checks and balances. So yeah, apply care basically. Okay, some more homework slides. I'm mentioning here the directories. I have a skeleton for various things basically. Yep, more. Okay, so some tips and tricks. I'll be quick here. Keep in mind that this is a binary interface. So you cannot really use the C++ way of doing things. You need to deal with handles. You need to call functions for these handles to create those, to initialize those and so on. So it's kind of a bit of a different way of thinking. You can achieve similar results. It's just that the tools are different. So keep that in mind. So here are some examples of related services. The very important concept is that unlike the plugin infrastructure where you were creating KPIs with 300 methods, storage engine API. Now you can add this atomic logically independent service APIs that stack on top of each other. So if you have a table to drive through APIs, you don't have to have a single interface driving all the aspects of a table. You can have multiple ones, like the one that just reads, the one that updates and so on. And you don't have to implement all of them for tables that do not support writing, for example. And that's a rough example. Okay, another important concept. This is again on the related services. It's important to be finding the right sets of services because if you have multiple services operating on a handle, then you want to make sure that they all understand how this handle does work internally. Because if you get two services from two different components, as is shown here, then handles from this service, produced by this service, will obviously not work in this implementation. So you need to be careful about that. So I have here an example of how you overload services. Basically, you use the service name. You don't use the implementation, concrete implementation. So whenever somebody basically defines a service, you get to use whatever implementation is here. But if later, a new component defines a new implementation of that service, your code will still look the same, but it will take this other new implementation instead of the original previously existing one. So that's a great concept. It allows you to overload an overwrite functionality that's even inside the server itself if you use components. So here are some random guidelines. Please make sure you read those. Try to use the component infrastructure macros. I know they are ugly, but I do have plans for it. So please help me out not to accrue more technical depth. Also, taking references to services is expensive. It involves taking a global lock currently and then, well, increasing some counters and so on. So try to keep references for as long as you can for as long as it makes sense for you. Okay, some more homework here. Now what's next? We are at a state where we decided internally that we shall have no new plugin APIs and no new service APIs, and the goal is to deprecate the plugins. And we also want to start dividing under the large blob of code that the Masquiao server component currently is. So that's all from me. I have 30 seconds for questions. Yes, Simon? Okay, so the question is whether we are considering versioning on the interfaces. No, the idea is that...