 Hi, my name is Sebastian and I'm making the archived web extension. And I'm going to talk about it's latest and how we're doing it. And I'm trying to give you more of you about the archived web itself. And I'm going over the next few and then I'm trying to give you a small summary of what we did and planning to do. I would hear not familiar with the archived web, she doesn't know what it does, please answer. Okay, so basically it's a pepsilic provide for central kernels. It's been around since the draft year series. The basic functionality what it does is it tries to create everything everywhere. It's not just everything, but it includes all sorts of interrupts, everything else. One thing a lot of people think is a patch that makes a kernel super fast, but it's always a little bit of a thing. It's actually super realistic. If you reach a patch that is 100 microsecond, this is the maximum. It never goes higher than this. But neither you can easily get multiple 10s, hundreds of milliseconds in delay. As I said, the project started around 2.4, which was the first product ready implementation. Later that the project became unfunded and we basically struggled for the one release and the other just to make it work before. Around October 2015, we got some funding from the Unions Foundation. The Earthly Pueblo project was initiated. With that money in place, we started to work on the project again, more or less full-time with multiple people. And it's just not the development and taking the current itself. It also started to move around the archived documentation from one place to the other. It's not just basically about moving things around from one place to the other. It's also about reading the old documentation that went back then, which was mostly old, since the bridging of current.org, which was, I know. But back then, the traffic on the Wiki on the current opportunities just stopped. There was nothing new that was added, nothing new. So right now we started to move it to a new place and trying to improve the Wiki and reading all the old content and add something new. Except for the documentation we have also tested in place, where each kernel, before I'm trying to release it, I upload it there and do some rough tests to see how it goes. So those things are new. Now I'm going over the page group we have. So a lot of people said that we can actually add the main address. And that involved it to always the same thing. So what I try to do is to make the rough estimation a bit. So this graph is for the 3.10 RT page set. The left side is the first release I did back then. The right side is the last RT release, which I did for the development. After that it gets picked up by another people, but I have nothing to do with that. So the red dotted line on the very top is the total amount of patches. So it will double the RT patch, go off the zero square and count all the patches we have in it. We get roughly about 2.35 with the red line. The blue line, the dotted line at the bottom, it's only the patches we intend to have for upstream. So from time to time we find bugs in the Valhalla curve as well. And we try to stage it at the front of the queue of the series file and get them out upstream so we don't have to bother them. So if you add the blue line and the purple one, you end up with the red at the bottom. So it's always a sum of those. And if you look right at the 3.10 cycle, you see there's almost no development at all. It's mostly a steady line. And back then the project was unfunded, so basically it tried to make it work. And after 3.10 we went through for 3.12, it's almost the same, because instead I got way more releases. The blue line is rising, which means I added more and more matches to the RT ordering clip up. Then we went up to the 3.14, only a few releases, nothing more. And back here in the 3.12 I managed to get a few matches around RT5, which then got us in the 3.14. But except for that there was much, nothing more. I think the depth of development. After that we had 3.18. There was a small spike here around RT5, which was most likely some part I found submitted upstream and then it got removed once we pulled in the standard tree. After 3.18, exactly, yes. And we had 3.14 and I had no time for 3.16. And then we went straight for 3.18. Then we had 4.8, 4.11. And there was a spike at the end of the 4.11 for one cycle. And this spike was due to the TI GPO rework. The problem back then was that the GPO driver didn't work perfectly and I had some duct tape to make it work. It just went somehow. I told the TI people that this is the work and I would like something like this and that. And they did it. So what I did then is once the air catches reached the main line, I pulled them back into RT and had it work. So once after that we went to 4.4 and you see the amount of catches were only 4, actually I'm surprised now. 4.4 is almost the part where we started finding the solution. And there were 2 spikes around RT 9, RT 11. And this was a major pile. We had back ported 8091 catches back to RT to get 8091 working. And we had simple rate cues, which we had back then in RT. And we tried to get it in the ditch. By now they are up to it. So those catches went to 4.6. And this one that he spiked ended up with too much lead work. So at that time the rework was done in the tip tree. It was more or less tested. And I pulled it back in into the RT and it's all just on top of it. So again if you look at the blue line, the purple line at the top, it slightly goes down. And this is mostly because the timer we were in for the week were able to remove some catches that were at least specific. And 4.6, 4.8, 4.8 was most required. 4.9, and 4.9 went to the future rework. The future scope and RT needed a base line, which was used by the future scope, was working mostly good. But we had a few hiccups here and there. And then Peter sat around at the rework part of the plot because it also affected its kept in mind scheduler. So he first came up with something. And after I eventually worked more or less in RT, we pulled it in into the tip tree. This is actually the first pile that got released. And after more tests we went out. There were still things broke. And this is the second pile which ran into an absolute cut and pulled it back into RT. And after 4.9, we got even more activity for level. And this is actually one part of this intervention which is done at top. We had something similar in the RT tree which we had one huge patch back then. And what top was doing is something that is perhaps still at top are discussing patches and they are targeting something around 4.15, 4.16. So even if they went into RTQ and the total amount rises more than the last patches, in the end, once the patches enter upstream, I don't have to carry them around. The second spike is EC200. We started with that in 4.4. Around 4.4, yes. And there was one small spike here but however the total amount of work started almost two years ago. And most of the patches were at this latest trade upstream so they never hit the peak of first place. So around 4.9, I tried the first time to get to RTQ work. I did the work back then so I just pushed it aside. And we tried again 4.11. And this around RTQ 9, 4.11 was actually the first release when we had STiP workbook, perfectly working. No more deadlifts at all. And at the end of 4.11 we have the HR camera we work with. These patches are right now around the first version still posted upstream and we aim to get those patches upstream. Again, the blue line is getting down. This is what we actually aim for. Getting more and more interest into their patch set but more and more are targeted for upstream SS. And this is for the team which I released last week and I didn't have any time to do anything else because it's the same version twice. But you look at comparison, we are roughly 180 patches which are targeted upstream and we went back to less than 100. So there is progress if you look at the left side. This is, on the left side we start with 3.10 the next one is the last one is 3.10 then go for 3.11, 3.14 and up to 4.11. And as you see, the blue line is crashing down around 4.01. And this only reason that that happened is because I went ahead and cleaned up the queue. We have places where I had like fix for fix for fix for fix. Then I sit down and collected all those fixes to build into one patch and clean it up. This is the only one reason why it went down in the ground. So this makes you look at the patch to see how progressive is their product. That's very visible. Just because I squashed patches doesn't mean we're going forward. It looks better than it did before, but this doesn't mean anything. Except for that I tried to look at the diff stuff. The diffs looked different than this. It was mostly the same queue, you didn't see there. It was mostly the same thing. Then I went ahead and tried the file size of the patch. It was mostly the same as it was that way. However, one way you look at another is to get into different graphics, but mostly what you see is that basically at the end of the releases we started to be more progressive because more and more patches ended up in the queue which were designated for up speed. Now, the summary, what happened. 4.049 was mostly quiet which we've seen in graphics, but back in the background we were targeting to do hard collage. This was rock which we spent over a year. We found it's not as simple. We had the few designs we wanted to go, but the implementation we had mainline was roughly difficult. We focused on it and we tried to make it work and we had to clean up the whole queue. We had the CPU notifier which had a priority. This is what we started to look at in the first place, but then we figured out that all of the vague files in which the files were organized were used as if you would like was also important. So this was one of the pieces that were not documented nowhere, but it makes sure that the code explodes during the CPU hardware. Once the pieces ended, I was too much trying to get it working for now but it didn't work. So we tried to remove the dark layer and get it working in some places else. We finally made it 4.11 and we need to focus differently on how we intended to get the CPU to work in the thicker queue. One of the things we do since the funding of the RTL project is starting with 4.04 we release every other cover which works mostly and we also agreed to go for the SNTS which is an after-tact work as well. So usually we do every other version but as we did with 4.8 and 4.9 we switched after 4.8 and 4.9 because 4.9 was the SNTS cover. This is what we continued since then except for 4.13 and 4.10 and we did the same thing. So the next 4.14 code was going to be SNTS because we had to attach it on top of it. So most of the versions out there are maintained with Steam. Julia Partridge agreed to maintain the 4.13 which she's using and one of the reasons she did it was because Steam didn't scale because the trees are LTS and stable and she needs to test them build them and maintain them somehow. So we ended the point where we tried either to avoid the trees or to keep you into the mechanics more. We talked about this yesterday last week in the summer and it looks mostly that we try to make what Stable fixes to the other currents we don't intend to make what features. We try to make what we want with the software we have all the things we had we don't intend to backport them with all the currents. One of the reasons is since the children's backport it carries a few headaches and becomes one of the ways other people can find the stabilizers and people are afraid using the stabilizers something like that may happen to them. So we decided to stick those features and just go for the real fixes that were obvious, the things that were broken before. What did we get merged? One thing that went in was the way we had to make for the stabilizer now to be built. In our bandilla back then it was pretty stable hand-to-hand for make for the stabilizer something we could not use for our team. So we tried to sell it upstream but we didn't have the use base for it. So we had something that worked but we couldn't sell it. A few years later the S290 people came along and they had a specific problem and we were looking at it and we were hey, this or patched it our team would solve it. So what the S290 people had is they took the patch set refurbished it and merged it upstream including all the things that we had involved for it. So this was I think the reduction between. The time when we were one of the things that happened was that no hands full was working. The requirement for no hands full was we were able to take a slip and spin-up from the stabilizer. So we tried it a few times and we didn't get it to work. It always blew up in one way. After a while we decided to not just spend time to make it work for the audience because it was a temporary. Once this went in it made it work for working. The next piece is as I mentioned the HR timeline is still pending. The HR timeline rework is one of the advantages we get out of it is that we can get rid of all the custom clues we have to get different timings and this is something we cannot have. So we had code to the data to the software to project which can be created. This was a lot of code we had but it was not too specific. But upstream we have something that is we have a hand full drive also tried to have the timers running to be able to enable it. And with this rework we managed to have one facility to have it working. And this is what we take for RT. So it's basically rework for upstream which also makes it simpler not to forget the tracing pages. Now we had a few days rework. So it wasn't basically RT specific but it takes a few bucks along with it. So RT we have a very small race control where one process boosted the other for priority inheritance and then it was de-boosted again. And there was a small race control where we could de-boost the process too fast and through. And this doesn't happen through the rework. And another thing that got fixed was the program automation was the schedule line. Because before that it was possible that two tasks were boosted to schedule line to finish the schedule line. So again, it's more rework and it takes multiple problems and more dough. We had a rework for the Paragami semaphore. So in vanilla there is the thing where the semaphore can be taken by multiple readers. It's an important writer. But in RT there is again one thing that we cannot have. Because having multiple readers won't mean we have to boost and it doesn't work. It's the same reason for for what we had in RT was only one reader can know or one writer. So it was basically a new text. But this was working in most cases. We broke, incidentally, the regular driver didn't know about it. So we got reports that doing graphics and the regular driver system would freeze for a while and after 20 minutes itself. So once we reworked the RW semaphore we managed to have multiple readers. That changed the regular drivers as it worked again. This is not only good for the regular driver but also performance setup which went away because now we are able to have multiple readers in the reader section. Same change resulted in RW in Europe. So if you look back then at the page that we had back then we had only one large page that we added all for RT. We included the RW semaphore and the spin-offs. All of them shared the same implementation to ground except for all the tiny details. For those in Europe to make other things work we ended up with more code in the engine. So even that we got more code in the RG page that makes progress because we got more things working to those changes. Another thing we get around was, I think it was exploited by GDB who started the process and started to forward in GDB. This part was around since the first release of RT and nobody noticed that. So this was, again, a series of changes around the trend drivers and such as the one in GDB. So, what's that? Most of them are RG specific. The terms works with the spin-offs. So, once we produced the spin-offs most of the problems arise with the spin-offs. So, once we have a bunch of events where we have in Pandilla, North Korea and the Netherlands is enabled and for RT, we try to shift the local regions and make it run with shorter areas for the SLB memory allocator. It's left in the beginning but the SLB becomes way better so we focus on the left one and the biorecator is the other one. So, for that we need to just look at the log sections and we try to get those parts because they are not really RG specific but those things happen a lot. With only one thing that is called backward disarray the problem is once to this end you want to take a spin-off you can not be preemptive. So, if you use the CPU by the way you always add the same CPU with this CPU and with slid-in spin-off you can get preemptive and you can get shifted to another CPU and at this point this distribute pointer is embedded so we have to work around and see if we bend it by greatest angle. So, every time you can take a spin-off you do by greatest angle and the process starts to be one in all the CPU and this makes it work with this distribute pointer. Now, we try to get it upstream because it's more or less nice what we thought because there's a lot of not a lot but a lot of users which disable preemption just to become a stable distribute pointer so we assume it would be nice to get a runway display in place so that it is this and we get preemptive and separate for preemptive disarray. However Peter, this schedule guy was afraid that it would happen with more and more people usage the outcome of it because right now, if you get a large preemptive disarray section, you can publish right now and see this preemptive disarray but if you go ahead with preemptive disarray what can happen is that one process has a lot disarray reaction and disarray migration gets scheduled off and we had something that this broken other task would run in C3s 2 or 3 so by it's not the case the C3s remain unused and for that reason it remains in the T20 set for an hour We have broken locks and the reason for that is the replacement of Air Force landing sections so it's basically a spin-off with a specific move and to try to get this upstream at some point yeah what we have are networking bits networking bits are specifically RT are just simple tiny pieces that break here and there so it's not one specific thing it's locking which are locked that way and other ways with small pieces here and there we have the printer interface which then comes along with zero drivers and this is something I would like to re-run at some point the problem is we cannot use grid cables in this area because if you run an application and someone writes something to the console you don't want to get a high latency because of it so we delay it and once you get delayed the problem is you hit a specific error which you don't see those things you see them in DMS but at that point it might be too late so while it works in general the interface should be re-run at some point another thing we have is the bit spin-offs which is not usually a spin-off it's just a spin-off that uses only a bit to set and un-set so while this works for vanilla and people are happy with it we replace those things with simple spin-offs and in general we have one spin-off but there are some code and texts forward I think we'll use two of them so we need probably a bit of spectrum and think about what we'll do about it what we'll do about it we have RCU RCU the patches we have in RT went almost upstream the problem is that once they did they were people with enterprise versions enterprise distro is running and they were very very slow so what we do in RT is we take the RCU context where they run the context of free of memory and we offload it to a specific thread so we can actually have RT broken with one CPU and probably RCU threads everything else off the CPU so everything RCU related to that is another CPU and this is one of the things we need for RT and we need to think about it how we set it to upstream so it doesn't hurt other people another thing is simple wait-cues actually simple worker-cues so it's like worker-cues but it's way way way simpler the reason we came up with this is a bunch of users which have something like the 4-link interrupt which gets fired if we have a few context if the CPU overhits or x86 so what the code does which is a few registers and then it schedules a normal worker-cue but we cannot have this in RT because the worker-cue handling is way too complex so this is something we need to think about and then we can get sorted I think we have probably 5 or 6 users with that much we have a DECASH tribal and it's not DECASH specific we have a few trials and the problem is probably the same there's one code path which takes the loxing AB order there is the other code path and we can take the loxing from the self-order so since they cannot take the lox we then simply spin lox they try the other way around and do a trial-loop the trial-loop on vanilla brooks because the owner of the other loco where the trial-tent fail is running so if they try the trial-loop once, twice, or three times then the fourth attempt at the latest it should be released because the other CPU released the loco but with RT on the other hand it still not going to be preempted so the user might not go to the CPU and it gets even worse if the user of the owner of the loco was in the same CPU would have preempted the process has the highest priority so at that point we never have the chance to take the loco all for some progress so we tried a lot of things and in the end we end up with CPU chip and CPU chip is a hack actually see what it does is it sweeps for like 1 millisecond and it hopes that the owner of the loco we will try to get gets in the CPU and releases the loco so this is what we do it's always a hack but this is what I need to do of this work we have a few in the work queue communication this is well the schedule guys help you about it because you get out of the schedule but the working guys want to keep it in there so this is mostly need to talk and convince them to take those in the critical code we have a bunch of GPU sections what basically does it is it does and it does AES with ABX as an implementation and while they are doing the critical algorithm they are not able to get preemptive and we need to be preemptive as often as possible so we have a bunch of preemptive and preemptive and preemptive to get networking so this I have a few ideas so we can say it may not work but this is also one thing we can still do I don't know if these are the sleep and sleep knocks this is actually a major piece of the patch search which basically implements everything that we need to fix up after it this is basically the part where the sleep knock is in the average age and another one can take on the security with one of these we need the preemptive software queue which is something that doesn't happen but we don't get preemptive at a cruise and we have a bunch of prior batches where we also we are trying to grab at this preemption for performance reasons but there is no need for breakfast parties in the preemption so they do it often just to get it fast to come whatever they are doing we have a few theories of that right going towards mainline we have more or less three categories things are required unlike sleep and sleep knocks we need those then we need things that are actually nice is lazy preemption lazy preemption is something to go for the part that we are slower on I.O. without lazy preemption so usually what vanilla does is it works as sleep knock and that is preemption at that point the scheduler is aware that once the intro is completed the task continues very running with the sleep knock if it wouldn't take the preemption then another task would run for the security instead the problem arises once the other tasks want to take the same sleep knock of course it then goes off the security end and things like that have been informed our team before so one had a sleep knock another one wanted to have the same one and everyone wanted the same one so everyone went off the security end until the original process gets back on with lazy preemption we have another hack flagged into the task where we set the same lazy preemption so once the scheduler sees that there is a task holding a sleep knock because it is a lazy preemption the scheduler is sure we get back to the same task before and if you get however a RT task with a high priority this doesn't matter anymore so at that point the RT tasks get busy for you so we get the same behavior with preemption we had vanilla but it's not fixed to any other tasks that RT so RT gets its own behavior and to those things I'm not exclusively trying but nice to have for performance and the last part which we should have but it's not necessary to get back to working are things where we fix up drivers for no reason but to get this every preemption or have this one on a few not this average because RT we usually have in terms of this average we rather practice so this is the roadmap like we can ask but this is like all of you with other tasks we get done any questions? the question was what are the chances of getting a spin on vanilla right here it's not that bad I mean we have things like which went almost upstream but Ian would say do it this way and take it I'm going to talk to Peter for the reasons I mentioned so this goes back to the part where it gets really first upstream and first we will try to get all the other bits that are more like MN and the bit spin blocks and so on and once we have this then we will try to get a spin on which is anyone? yeah the question was it depends sometimes we have one patch that covers like 5 drivers one we have to drive a driver most of them live in the IA drivers IDE department so but there are also others Julia went ahead and said hey you have this many things that look broken in IDE and I said yes but who cares because usually we see people who use the things or they don't fixing a bunch of drivers for no reason except to make it work but I'm not sure it's worth it yet so we get reports from people that used to grab this Able we had those in the I915 project by Intel and I think it was that one so they did grab this Able and later they grabbed the Able and there was a comment we want to help the technology with this change it was clearly wrong and we asked it why we do this and he went ahead and said no don't do this at all so usually it's easier to not lightly change those things but go and drive a driver so once we have those grab this Able replacement it might also be good to talk with the people in charge and ask them is this still required and we live without it because this also we have a change in x86 which went upstream and basically the thing was if we have to flush a lot of cash lines there was a change in the season more than this amount that goes ahead and does quite a bit better when everything appears this is very bad about key flush one by one it takes more time but it doesn't fight ladies in a bunch and we managed to argue that this is also bad for conflict we have so we managed to hide this change already in the other car but it also hurts a conflict we have ok, thank you for your time