 Good afternoon. My name is Sergei Bermat. In case you were here in the morning, I am a software engineer at GameConcept. I work during the days, but during the night, I like to develop Hellenos. So, therefore, my goal for today is to tell you a little bit about capabilities in Hellenos. So, first of all, in case you didn't know, Hellenos is very portable. It runs on seven processor architectures, starting with ARM, going through Itanium, as was already mentioned by DZ, Spark, RPC, of course, MD64, maybe some others. And it was designed and implemented from scratch. It was mostly written... It was written by us, so it doesn't contain any third particle. It doesn't clone anything, so it does not look like Hellenos or Hellenos. And you wouldn't find someone else's code in it, basically. We like to write it in a way so that there are no huge monolithic components. So, for example, if you take a look at our networking stack, you will find several components that create it and that roughly respond to the layers of the networking stack. I would also like to deliver a short update from the last for them. So, last year, we managed to release this one in April, the second one in November. The November release is the one in which we switched from Bazar to GitHub. So, you can go to GitHub and work us there. Last year, we also had our regular hackathon event, which we call Hellenos Camp. For me personally, it was a very good experience because I saw new energy which came into our project. Some of the topics that I will be talking about today started there. And last but not least, we now also have a paid developer who is paid by CZepnik, which is the caretaker of the CZepmain. They have a stand where it goes to this room and also famous for making it into this Honea router. There's four things that are ahead of us that will be soon to reflect marching to Hellenos. There is a C++14 support, which should be ready sometime in February. And a USB 3.0 enhancement of our already existing USB stuff. What you can see on the picture is a laptop of one of the USB 3.0 team members, which is of course running Hellenos and has a flash disk attached via the XHCI controller and some details transferred from there. So that was for the update and for what's coming next. Now before we delve into capabilities in Hellenos, there's one more thing that I would like to mention that's a Google Summer of Code. As you might know, some of the projects assembled in this room participated last year in Google Summer of Code. By the way, Google Summer of Code is an internship program for university students. The students work on an open source project. There are some mentoring organizations that mentor the students and Google pays stipends to those projects, to those students. So last year we participated with very, very three students, one for Hort, one for Minix, and one for Redox. All of them were quite successful and this year we applied again. So you can keep your fingers crossed for us on February 12th, when the mentor organizations will be announced by Google and in case the microcon on the room is selected, then students can start applying, starting from March 12th. And in that case, you can search for this microcon on the room logo on Google Summer of Code pages. So that's for the agenda. And now capabilities. So this is how we are used to talk about capabilities in HINOS. So for us, capability is a task local reference to some kernel object. And kernel object is basically a wrapper which is reference counted and points to a real thing, to something which lives in the kernel and is accessible to user space only via capability, which is referred from user space using a handle. You can imagine the analogy of the FS files where you have a file handle, which is the integer reference, where you have in the kernel the table of open files, which is basically an array of capabilities and elements of this array then point to the underlying BFS object, which would be kernel object. What is the motivation for actually caring about capabilities in HINOS and trying to bring them to HINOS? So over the years, because HINOS started in 2001, so over the years we accumulated a certain amount of technical debt and we also haven't always known how to do things properly, so we need to fix those things. If we also want to do some more fancy stuff like virtualization, such as the one which was presented here last year forward, we need to get rid of all global identifiers or global names, and because we have, I think it's on this side, we have laptop stickers that say that HINOS is an advanced micro-canon system, I can't imagine going forward without having capabilities actually, so it's also a matter of modernization. So I would like to present some examples that illustrate the need for capabilities. So here is an example. Our goal is to pass an open file from a parent task to a child task. Both of them are clients of the VFS server and how do we do that? So one possibility would be to go like this. What we see here is the setup that I have just described. The VFS server contains state which logically belongs to the parent and to the child task. It's the table of open files and its per task. So it has a table for the beforeage task for the parent and it also has a table for the child. And the goal is to duplicate this, basically the object pointed by one entry in the open files table and duplicate it in the child's table. So now you are going to see an example of one of these broken interfaces that we implemented. So what I did a couple of years ago was something like this. You can see the whole process involves seven steps that result in the child's process receiving a capability. I will briefly walk you through it. I won't spend much time on it. Basically the parent starts by telling the child, hey child, now I'm going to send you a file. And immediately afterwards it says, and now I'm using a kernel recognized IPC call method, IPC message method, IPC and state change authorize. And the child says, okay. And then it says okay, both of them identify the common server. The kernel notices the positive reply to this message and sends an IPC notification to the VFS server. The VFS server then proceeds to actually duplicate the actual entry in the parent's open file table and creates a new entry in the child's table. Then the process continues by the child sending the VFS server for the new handle and the VFS server replies with the handle and affirmative reply. And the child concludes the whole handshake by replying to the original first message by the European. So is there among you someone who absolutely loves this approach? Yeah, I think so. So of course this is rather an example of how not to do things. Myself, I'm not 100% sure that this mechanism, even though it's a generic mechanism which allows two clients of the same server to negotiate a change of this state, I'm not myself convinced that it's actually safe and free of some race conditions. Also, there are two things to note about it. Those VFS objects that live inside VFS are not real kernel objects. And basically they all match to the VAL IPC entry point that exists in the VFS server. And second thing to note is the active role of the VFS server and also the kernel which leads to some unification. So the bottom line is we don't like this. It should have probably never been implemented. And instead, there is a different way how to do things which is capability. So instead, we now have something like a table of open files or if you will, a table of capabilities that is still associated with each of these tasks but lives in the kernel and references some kernel objects that also live in the kernel. And now the exchange is quite easy. Imagine that the parent task has a connection or has a way of talking to the child. It can be concluded in two steps. First of which is the parent basically telling the child please connect to me and identifying the object which to connect. And when the child positively responds the kernel duplicates basically the connection to the originally pointed object and it's done. So there is a mechanism which lies in an IPC message which is understood by the kernel called IPC-M connect either to me or me too depending on whether you want to connect to somebody or have somebody connected to you. So it's a mechanism for creating connections and go-back connections. And currently it does not accept the core number so it wouldn't work that way in the current Hellenos. There still needs to be something done in order for it to work like this. Note how the VFS server is not involved anyhow in this transaction. And we now have a different situation when each VFS file is represented by one kernel object. There is no more correspondence between one IPC endpoint in the VFS server and one kind of user space kept VFS object. So that was for passing files. And there is another example which actually there is no ethics for it in Hellenos. So imagine a device driver. Device drivers usually need to register some kind of interrupt handlers. And the question is how to properly identify the handler so that it can be referred to from user space and eventually unregistered. So before Hellenos 0.7.1 Hellenos supported three syscalls. The first syscall sysdevice assigned devno was a syscall used to assign a device number but it wasn't anything more. It was a syscall which would routinely assign a number from the global pointer and then increment the pointer. Then you could use this global identifier to subscribe your interrupt handler which in this case is referred to by the IRK argument. And you could also use it to unsubscribe the same. Now I wonder can some of you spot some problems with this interface? There are a couple of problems but I want you to think about it a little bit and come up with some potential problems that you might see. Yes, there are some problems for the collaboration so that's the only problem for the builders? Yes, that's definitely one of the issues. So Norman asks whether or he suggests there is no authorization for the components to do this. And I won't do this any longer. So there are a couple of problems. So first of all, micro-canon should probably not be assigning device numbers even in this stupid manner when a device number is basically a counter and doesn't really mean anything. But the more serious problems are that the API doesn't require the caller to actually use the returned identifier and instead the caller to use any number of its tools. So for registration it can make up something of its own and even more serious for the registration it can register intra-handlers for an arbitrary other task. So tasks in... G.Defno is the global? Sorry? G.Defno is the same number in two tasks? It's global. It's a really global lighting. So SCU probably will agree with me this is completely broken. And it was fixed in Hellenos 0.701 when we introduced capabilities into this. So we got rid of assigning device numbers in the first place and then modified the API. So now the subscribe Cisco returns capability which can be used by unsubscribers. So it's no longer possible for unsubscribe to unsubscribe something else and that belongs to someone else. Of course it can unsubscribe different hierarchy handler from the same task. So if you can agree that capabilities are great and everybody loves them how do we introduce them to Hellenos? And the question is that we don't have to do that because they are already there but that way is quite limited as we go through. So now here is a picture of the basic Hellenos IDC communication between two tasks. So in this kind of pink oval I put all the state which exists in the kernel. We have two tasks the orange one is the one which starts the communication it has a set of phones which are basically connections. Some of the phones are connected to other objects are called answer boxes. The answer box is basically a message queue and so where to begin. So the orange task can basically specify the connection by a handle user space handle in this case 2, 0, 1, 2 which to use then would specify some arguments of the message. So the handle identifies the phone the phone identifies the answer box then the receiving task can use Cisco to actually wait for messages on the answer box and gets the call basically receives the call which is identified by so called call ID we will get to call ID later. And then it has one of two options it can answer the call using the call ID and submit an argument in which case the call goes back to the answer box for the send a task from where it can be again waited for by the original sender which concludes though and the sender gets replied to its own message. But the recipient doesn't have to answer it can do a different thing it can also forward the call to someone else so now we have three tasks so here we see that as I've said the blue task is not directly refined but it's forwarding the message using some of its own IPC connections in this case number 5 so it's this one this one is connected to the answer box of the green task and again the green task can either decide to forward the message farther or reply what is important here is that the reply goes to the answer box of the original sender now how do you create new connections in HALAOS so you need to use a dedicated method which is understood by the kernel and basically if you if a task replies in an affirmative way to this call the kernel notices it and creates a connection between the two tasks so the sender was the orange task the recipient is the green task the kernel notices it and creates a new phone connected from the orange phone between the answer box it can also the direction can go the other way so instead of using IPCM connect me to I can say IPCM connect to me in which case the arrow would go like this so there will be a new connection created from the task which replies to the answer box of the original sender this was important to actually realize that HALAOS contains a coarse green capability system which is in its IPC my claim is that this IPC is roughly analogous to the Mac IPC even though there are some limitations so here is a little table which maps the analogies from Mac to HALAOS and vice versa so the unit of communication in Mac is a message and called in HALAOS an IPC communication endpoint is called HORT in Mac answer box in HALAOS the connection is represented by the sender right in Mac while it is co-op phone in HALAOS the receiving from an IPC endpoint can be done if you have a receive right in Mac and is implicitly allowed to be owning task to the task which wants to receive from its own answer box so it's implicitly allowed in order to answer or to reply to a message in Mac you need to have a sender right to some HORT which accepts a reply and in HALAOS you need to have the call ID of the accepted call and of course this is something that Samuel from the project was mentioning today in Mac there is a way to actually pass the sender right to a third party so if there is a name service, task and has connections has sender rights to some other clients then give such a connection to whoever inquires for it and the same effect is achieved in HALAOS by forwarding IPC connections as you see in the previous slides so the bottom line of this is that in HALAOS the capability corresponds to a phone and in order to pass capability you need to forward a connection now what are the limits I said it's analogous to Mac but it has limitations so the limitations of the state which existed like a year ago is that phones were the only types of capability there were no other objects that were somehow usable at least and it was only possible to pass one capability to take this connection there was only one answer box possible for a task which is different from Mac which you can have multiple ports owned by a task and there were some hard limits so there was for example a limit of 64 phones per task and there was one especially annoying thing which is the last one that those core IDs are the numbers that in user space identify a call to reply or forward where actually leaked kernel addresses because frankly can there be a better user identifier than a leaked kernel address and sure there can be and we will see what it is on this picture which I tried to draw we actually see a kind of granularity problem because of the existence of the single answer box so even though we have a couple of objects that exist in the user space context of the server we only have one answer box which somehow needs to be mapped on those user space objects and this was done by the libraries and Helen also so this is the past state what we have now the situation has changed slightly because we implemented a new capability for more and started with making IPC work with it so IPC is now implemented in top of this framework we also introduced two new kernel object types so one for calls we no longer refer to calls using kernel addresses but we have proper capabilities that identify a kernel object for each call and also as was apparent from one of the examples we now have capabilities for IR view objects unfortunately we still have only one answer box per task so we haven't fixed that yet but we will probably fix that very soon on the picture which is at the bottom here you can see the general structure of the capability framework so the orange symbols they represent the cup underbar T types which are the actual capabilities so if the capability is in a published state it is visible in the user space under a handle and it is 5.30 and also there is an owner of this capability which is a task called a capability objects they point to K object underbar T which contains a reference count and a type and points to the underlying raw object in this case which we call underbar T and apart from references from the object from the K underbar T structures there might also be references from the code so that's the reason why we have 3 and also 2 here related to having only one answer box we still in this kind of situation need to use the capability which is which points to the form and a global number which identifies the surveys or the object in the survey and so it's slightly better than it used to be but it's still not good enough and we would like to have something like that so on this picture you already see that we got rid of the limitation of one answer box per task and now we have as many as there are these library entities that were previously left to single answer box so and with that it's related to the fact that now it is sufficient to use only the capability handle in order to refer to the respective destination object and so there will be some correspondence between the C library async ports and the actual answer boxes there will be a new capability side for them if you do that you could actually get rid of the phone objects at all because now they will become somewhat redundant because depending on the capability right it will be either possible to make use of the answer box capability to send messages or to use an answer box capability to receive messages so I think we won't need phones when this is done and because even if we do this there will still be two more objects that use global IDs in LNOS and that's threads and task IDs currently these objects have global identifiers and whenever an API works with press or tasks it need to pass the identifier and the kernels to look up and make sure that such an object exists and this won't be necessary if we also if we also make it possible to reproduce objects by capabilities so the previous slide said what needs to be done in order for me to be satisfied with the situation but there is more to come and more what needs to be done so we will need to seriously think about resource management because along the course of making these adaptations I removed some hard limits so for example the limit on 64 capabilities per task is gone nowadays a hash table and it's possible to actually have a limited number of capabilities so now it's possible for tasks to actually consume so much memory that the system becomes unusable a similar situation is that I removed a limit which was there for the number of unanswered calls sent over one phone so in the old HANOS it wasn't possible to send more than four if you send the fifth the kernel would block such an attempt and because this was contributing to some hard to debug deadlocks I removed it and instead decided that we can somehow different problem to be resource allocated so if instead each task has some resource pool these limits can be basically enforced by the virtue of the task running out of its resource pool so it can it will be allowed for a task to have as many active calls or as many capabilities as its memory allows it to have so we will most likely introduce resource pools and I'm thinking of a resource trading scheme which would be probably very similar to the one which is present in Gnode so whenever a task request another task for a service such a request would have to be accompanied by some resources that the client pays the server actually with to the actual board then I have some minor things that we don't currently need or use but might seem to be useful or even needed in the future so for example we don't distinguish between different capability rights at this point all capabilities are equal that point to the same object but as I illustrated on the example of the answer box capability that could be both used for sending and receiving we might need to introduce something like capability rights currently there is also no way to revoke an existing capability so we might introduce that if that's indeed we are still able to pass only an IPC connection even though we have capabilities for other kinds of objects this might be actually this limitation might be actually removed when we have capabilities for tasks because now it's not even possible to specify a task using a capability but if it is we can specify a task by capability and another capability which should be duplicated in the new task and that could be a new or alternative mechanism for passing capabilities between tasks in how many years or we can do a similar thing for IPC and relax a little bit the requirement that the task object is an IPC connection and that IPC and connect to new head shape so in summary I wanted to say today that even though we hadn't known about it there was a capability system in Hellenos indeed which was basically by chance I asked the developer who created IPC for Hellenos whether he intended it or not and whether he was familiar with the MAC approach and he said that he basically declined except that he didn't intend to make a capability system and he wasn't even aware of the fact that it's a coarse grained limited capability system and anyway we took this foundation and extended it so that it's more generic and contains more capability capability types and kernel objects to the point in which we were able to fix some of our broken APIs fixes for some other cases like the one with passing VFS handles still needs to wait until we can have multiple answer boxes per task and all this stuff is still quite new it's a little bit still in the flux so there might be some time required for it to settle down so things still might change a little bit and with that I would like to thank you for your attention that's a question yes, so you mentioned that FVDTs have an orderly test, right? you have just said the rule I'm trying to determine the semantics of that what does that mean exactly that the orderly test stays yeah, so you have to you basically need to think about this as a magnified version of this and capabilities literally speaking are objects that exist in the context that belong, that are task local so it's basically a reference let's pick one for example number four is a task local name for this P for this kernel object and it exists here so when you say you want to test A test A is the local I'm trying to understand whether it's the issuer or the receiver well it so the question was who is the owner of the capability right? yes well every task has a set of its capabilities and they might be associated with some kernel object and it might be possible on these examples I don't think there was a case in which one actually on the last example if I switch here can I rephrase the question let me finish so here we have an instance of a kernel object which is referenced by two capabilities one of them is in the orange task and the other one is somewhere here in the blue task so there are two capabilities that point to the same kernel object and both of them are in different tasks yes and that's the task in your data structure well this was just an example so that's a few new tasks yeah there is because these entities are in a hash table and the task so they belong to a task and there is a hash table so they belong to the task by virtue of being in that task a hash table of capabilities because the opposite I mean there are at least two ways to do it yeah there are probably many ways to do this Norma do you have a question yes so this is by full dimensional change at least it looks like this and how much does it impact on your data so the last so there is a question about the impact on the existing code whether we had to rewrite loss of code and the answer is no because we did that slowly and gradually so we first converted the IDC framework to be using this new capability stuff and preserve the functionality which was there in fact we are still removing some weird kind of reminders of the whole system so we are still cleaning up and the other changes such as removing the kernel addresses for qualities that were quite straightforward and I just have to ask I have to change fix only a couple of places of course it takes some debugging and some development but we are now talking about the order of like a couple of evenings basically but this reminds me something that I didn't mention when we will be able to have multiple multiple answer boxes pair tasks it will naturally lead to a more comprehensive way of programming so we will lose this granularity problem and it might change actually the way how we write or how we look at our servers but so far the changes were contained to the libraries common libraries there was no major changes in the server yes so Martin said that these changes were confined to the kernel and the libraries and there were no other changes in the servers so that's true it's a follow up to the previous one you have the capability to you can have the capability to mention task A can send a message to task B yes and such a capability so the question is if I have a capability which allows a task A to task B and what does it look like and such a capability is so we can take this example so there will be in this orange task there will be capability under the handle 4 it points to this capability which has also label 4 in it and it points to this kernel object which is the phone IPC phone IPC connection space and this phone is connected to this answer box which is another kernel object blue one is B oh I see you are trying to you are trying to find the tasks that are so this was just I just made up the identifier here this picture existed before I drew this one so just disregard this A it's something that can be but it's not necessarily the same numbering as this here I'm actually trying to match this to the problem I encountered when I prototyped my capital DTA system which was I had difficulties with what basically the impossible potential race conditions with processes that are on both sides and what and things like that I could look at what she said that well I'm trying to understand if you have the same problem as me or if you actually so the question is maybe I should ask that maybe you can take it offline but let me deal with this so the question is there can be such a situation when I somehow lose track of what capability points to what so the user space part of the task can of course forget what its capability handles are, it's the same you can have a set of open files and you can completely forget which one is which so this can also happen to you but since kernel objects are reference counted and when the task is killed it cleans up its capabilities it will result in kind of decreasing the reference counter and then reference counter and making a kernel object which is not owned by a specific task drops down to zero such an object is automatically destroyed so you don't have to really care about freeing capabilities because there will be free time when the task is killed so and that's more of that so do you have any more questions because we have already done a bit question what happens when the server dies and you report the capability to that server's answer box so the question is what happens when the server dies and I hold the capability to the server's answer box and this is not actually specific to the new code because such a problem already existed before I think the phone got slammed and is marked as basically dangling dangling things so that when you attempt to actually send a call over it you will immediately get a message and there is another there is like a symmetric issue which is what happens when that kind of disconnects and in that case there will be hangout call which is sent to the answer box well again basically it could be related to closing TCP connection in the case there are no further questions I thank you once again for your attention