 All right, so if you were here for my last talk you already saw these pictures, but hey, sorry, so I'm laying the light I was the developer advocate for rel I came with the odd Qualification and I've done production applications in almost every major language with the exception of Python Which is kind of entertaining So yeah, so I did that for a while and then I kind of moved into engineering to try to help Improve the developer experience for users of rel primarily and then I moved into Working on Fedora next project And I've been kind of working on that for a long time And I'm the modularity objective lead. All right, so Hopefully not so many people here for this before but so basically the World that we're kind of operating and it's been changing over the years and it's shifting to be much more developer focused You know as we've seen with the rise of containers It's we have these monster stacks that we build our applications on top of very thin layer almost these days on most applications The way we do software development has changed and that we can basically delete entire architectures and rebuild them in you know a matter of days, if not weeks versus months and years the Software as we distribute it in distributions has kind of a single use case For the most part and it takes and it's not very discoverable if there is available other versions or alternate use cases that you might want and so You know a lot of these things are starting to kind of come to a head And so what we see is results like the advent of containers, right? So it's not so much that containers You know whether containers are good batter and different It's more that they're obviously a clear reaction to restrictions placed on developers on how they can build and deploy their software There's other examples as well But that's kind of one of the clearest ones that clearly the distros are doing something wrong If you need things like containers in order to be able to run the applications they want to run in the way they want to run So that's at least my thing So moving on We have you know, sorry, I was a slide behind So now we're starting to see as well that the containers Are kind of going all over the place, right? So we have this this huge growth in types of containers With different trade-offs and different use cases You know, so you kind of have like rocket I would say it's like the clearest example of something that is heavily targeted at sys admins for deployment Docker was heavily targeted at developers Kind of bypassing the system administration Then you have also things like snaps and flat packs, which is trying to do basically the application isolation on the desktop And these are all kind of overlapping Functionalities, but they're not the same right there. They have different trade-offs different goals And you'll probably start to see them come together and we have to some extent already, right? Like OCI is starting to try to set a standard for at least the outputs even if the goals are kind of different And then cryo is kind of a newer one. That's really specifically targeted for running You know kind of cluster set the cluster Containers in a sense or clusters of containers. I guess it's a bit of a way to put it So you have a bunch of these different solutions That clearly like I said, I think clearly show that there's a gap in What the Linux distribution is giving you on a server that these things are trying to solve However, the problem with a lot of these is that there's really it's really difficult to know what's in there or why it works, right? So we can look at it from the outside and we can do lots of things to try to make it safer, right? So for example, you know, there's big push around with like crowd for example to make sure that you know having Privilege inside the container doesn't mean you can have privilege outside the container So those are all great, right? But that doesn't tell you it if you have a sequel injection error, right? I mean it doesn't defend you at all. Okay, so And if you don't really know what's exactly in the content inside it could be really dangerous, especially as the Model around containers and a lot of the time is people are encouraged to just pick something up off the shelf and start using it Without having a good idea of its provenance. So these are these are dangerous And I think these are things that are getting solved and will be solved over time But it still kind of shows that there's You know that the things that we've been doing in the distro to ensure the this problem not happening Isn't working for what developers and sysad means need today. So However, there's kind of the flip side or sorry and to kind of further out onto that we have this curation, right? We we say in Fedora We work on making sure that we know the problems of something we make sure that when you ask for something to get what you think You're getting we sign it you have a trusted partner all these things and again We're starting to see some of that stuff in the container world But the distros already have it. So how can we try to start to marry these things? and the project I lead right is the modularity project but What I wanted to show this talk is here are some other distros of what they're doing Which in my opinion is the trying to solve the same problem in different in other ways And you know, obviously it's always good to look at other people's solutions to figure out you know where the gaps in yours are So Here is how they're changing. This was the best I could do for a changing picture, you know So I just turned to the left side so animals on the next two Extras is the first one and then Susie modules and other we'll talk about and then quicks and nicks I'm sure there are others, but these are kind of the ones that I thought were interesting for this talk so the idea here is They make alternate versions of software available they do it with Basically creating It's kind of weird how it works But basically there are new repos enabled when you want that piece of software so you want the latest version of postgres and Then you basically enable the topic and now a new repo is added That has this newer version of postgres and any dependencies that it has and so there are Not I meant to count them. I want to say there's like 20 of these at this point give or take and You know so you can have but you can have multiple versions of things you'll get two different repos And you know so you have postgres 9 is available for example, and then postgres I think it's actually 9 6 and 9 8 as well as postgres 10, right? So you have different versions available and you kind of can turn them on and off using this new special command called Amazon Linux extras And you enable the topic you enable a version and that's how you get So and depending on how much time we'll have all demo but Let's see. Oh, and so packages can also override right so you can have something in these repositories that is Overriding was in the base. So it has a few problems that in my opinion of how it works so It's it's kind of interesting that in the when you run the application itself It warns you not to turn on too many of these Which I think is kind of concerning In all the documentation and I will say what we've already had I think we have this problem a little bit too But in all the documentation it warned over and over again that this is not supported in production And so if you use any of this stuff, it's not supported, etc I don't know if that'll change, you know, maybe it's kind of the tech preview transitioning to production usage, but The the part that concerned me was the if you have too many it will not work One of the other key differences that I think that modularity does that this doesn't do is that if I enable if I enable the kind of Tri-layout says so There's no there's kind of no sticking to the stream in a sense, right? So it's whatever's in the repository is just available There's no tracking about the fact that you know, I'm on postgres 8 and I now have been upgraded to postgres It just happens and it's silent just like a normal rpm update This is one of the key things I think with modularity is that you are warned before you do an update So that you can choose to switch streams The other concern and we actually Would have had this problem, but we were concerned about it is that it's a repository repository Her topic and the problem with that is that both D&F and young start falling down Pretty hard when you have a large number of repositories So if you move your whole system to using these topics you would end up with you know, 30 40 50 repositories And the package management tooling doesn't actually handle large numbers repositories very well You've used 700 repositories Yes Okay, so the point was made that we can do 700 repositories in DNF. I know when we tested it I guess a couple years ago. It definitely could not Yeah, so actually so the common was they use young and Amazon When I was digging around I actually think both are available But I don't I don't know if young is just a similar to DNF like Fedora has been doing because I didn't I missed getting to that step I was really surprised Yeah, so apparently maybe 700 repos is fine. I'm surprised so oh And then the other thing that I Was it was interesting and I couldn't quite figure out how this hook worked, but basically if I say Yeah, I'm sorry, you know kind of Amazon Linux extras install Oh, no, that's how it works. Sorry, I just figured it out So so you do the Amazon Linux extras kind of enable whatever the topic is and then you go and install What you think of this kind of a high-level package in them in there which has is a meta package And then kind of gets all its dependencies and so when it When you enable it it says hey, we recommend you use you know, basically this You know this name of the Postgres or p.m. So you have a few things there So we have this kind of concept in modularity called profiles which allow you to have different use cases for given pieces of art So postgres you might have client you might have server right you'll get different rpms install depending on which profile you use This has basically a single profile that you use and then and then the other thing that I I don't really like about it But it's it's not anything particularly necessarily wrong with it is that it uses name manually to separate The different versions of postgres. So it is postgres SQL 1 0 is the name Rather than kind of using the version number in some kind of proper metadata area So those are a few things that I found Challenging about it, but it does work You know, it's it's available today You can I don't I didn't see a way to kind of use it offline even though you can use animals on Linux to offline You can actually download the distro But obviously, you know most of our p.m. Repos are online anyway, so that's not too big a deal The other interesting thing that I thought was that they actually use that their CDM network with basically individual hashes for each of the rpms rather than there being like a Web page right that you can go to and actually kind of see everything in the repo Which I also thought was interesting, but again not really a fault just different Sorry All right, so that's Amazon Linux extras Any questions on that? All right, we'll see how we do All right, so it's just a modules is another relatively new thing And is as far as the last I looked is only available for the enterprise version and this is a like seven or so different repositories that have kind of kind of a H is where topic the kind of a topic area That bring new versions of software together that are related to that topic So like there's a web development one that might have new versions of php and might have new versions of you know Web servers, etc, but they're kind of all bundled into one repo They are as far as I can tell essentially similar to like rel extras or like rpm fusion in a sense Like here is an extra repository that has new or let's say different potentially newer versions of software and made available to the end user in a supportive way in terms of you know for enterprise Susie and You know so they have this kind of web and scripting was one of them legacy is another one and there's a bunch of others But essentially you're just adding the equivalent of rel extras or rel It's not quite like optional, but more like extras So it's pretty straightforward, and I love that the massive name collision is confusing So I Guess I already said this You know and and I don't know where they're going with this It's a little hard to tell there hasn't really been updates of like additional Of these buckets in the time. I've been looking at it So I think it's just you know customer demand It's getting very warm All right, so the next one that I think is really interesting is next to us and wicks so the way this works is Quite different in that when you install a particular piece of software it actually kind of puts it in a non-standard location and Then essentially adds in sim links to the correct location or from the correct location to the this alternate location And it'll so as a result it allows for full parallel installation as well as parallel So you can have a complete clone of you know, whatever set of libraries and then it provides them to each application so it's it's very sophisticated and And it seems to work well. I haven't run into too many problems with it and It's you know, it kind of uses they have a bunch of other nice features like the entire OS is actually it does the atomic thing where You know, you can have infinite rollback and even even adds it to grub So that you know, if you try to do an install and it fails, you can always roll back It uses a declarative Mechanism for describing the configuration of your of your entire OS So it's really nice and it works quite well However I put this on a slide Yes, the user adoption is tough because it's completely different like you you know The package like the usage of the package management and stuff is just not you know It's not DNF and yeah, right? It's quite different And how you approach managing your system is quite different because you use this declarative syntax inside of a configuration file That kind of declares I want Thunderbird Versus, you know what I think of is what we would do normally is you know, we'll install blah blah blah In a sense, it's actually more like managing a system with ansible. So in that way it is similar, but Like for general user adoption, it's pretty tough You know, like said, I really like it The other massive problem is it requires new package, right? So our 21,000 or something packages all we need new package and then But to the end to the end point it is a very You know, it is a it is a good solution. It's good at what it does and it may be Even if we adopt, you know, kind of stuck with our rpms, etc It might be a good way to shift if we wanted to get to parallel installation data I'm just curious Did anybody investigate trying to change? Rpm build an Rpm on the back end to do this instead of trying to rebuild the world Yeah, so the question is basically could we in You know in in the things that build rpms in a sense cause the output to Result in something like the the next simling tree like this guy so It was kind of toyed around with You know, the there's other The problems you also have to kind of convince the applications So you have to pass that information to the applications as well So like you have to tell them which set of sim links to be using Why? So like if you have parallel installation you need So application a is using library foo version 2 and application B is using library foo version 3 You also need to pass that information to the applications themselves And with the sim links you can turn around next We have the soft line. I have I have ideas of how yet we can do that So I think I actually so having to change advocating context. I think we can lie to it Yeah, so so basically kind of is like, you know, are there mechanisms that we could use to make This result but transparent to both the applications and the users and the maintainers And I think it is potentially possible I don't think it's trivial. No, it's not trivial. We might also break everything right you try it. Yeah All right, so go ahead So in fedora, we have nvrs and our rpm names How are these identified in the file system? Well, like the rpm is a great question. Oh, sorry So the question is like, how are the different versions identified in the file system? We don't actually identify in the file system, right? We use them in the rpm database It's kind of similar is that it keeps track of What it put where and so when you try to install it again, but there's no conflicts Because you can have parallel installation Right, but I was like looking at the distribution and I saw there was a courier tails Like how is it? What is a file name that contains courier tails 825? It's it's a it's a directory and with a uuid Yeah, like it creates a uid places it in there and that hash has generated build time So there's a there's a discoverable mapping, right? So the tooling when you go to query the system will tell you where it's at Yeah, sorry. So the the question was, you know, how do you how do you track down where? like what version of something that you're actually, you know have installed or you're using and When it puts it in this alternate location. It actually is this It's yeah, kind of like a uid but basically there's this generated hash that is actually input into the hash is Directly made from the binaries and all of its dependencies. So you can I think that you can even reverse it, right? Yeah, yeah, so you can reverse the the hash and know exactly all the things that Built it as well as it runs with and that's where it kind of gets stored off under But how is it distributed? Oh Typical package management like repos and what's interesting about it actually is that it will check the the next repo And if it's there, it will pull binary, but if it's not there, it'll actually build it locally Which is kind of neat so you get both that kind of source and Binary distribution mechanism at the same time And so what what further or what follows from that is that if you modify one of their spec files Locally and then you know kind of install it. It will rebuild it locally. It'll be your custom build with whatever your changes are Like I said, it's really it's really quite sophisticated And I think to Adam's point. I think there's a lot of things we can learn from it that but it's just it's going to take a while and What I think modularity one of the big drivers with modularity was really about Getting our flexibility back And so big changes to the infrastructure that allow us to do new things And so I think the steps following from here in modularity are about like how can we now take advantage of that new flexibility about how we build things and maybe maybe start to think about things like this, you know So that's kind of an idea Let's see So then we move on to modularity Let me just can I say show of hands like do we want to talk about modularity itself? Yes, raise your hand All right, cool. All right, so The idea of modularity You know what I was trying to point out in some of the earlier slides, right? It's like these are some approaches that people are taking With the exception of nix both Amazon Linux 2 and to say are both post us doing modularity So I don't know if they were following and use some ideas from us or whatever But we we haven't really been able to take advantage of looking at their stuff and applying it back You know where there might be lessons learned because they're pretty new But this is kind of the idea with modularity is that we're trying to stop being or trying to stop being focused on the RPM as The the know everything component Because we have this like kind of mindset that we you know that in our source tree You know, it's just leads us to this binary and everything's wrapped around this like our PM binary Both our distribution mechanisms as well as our As well as how we build them etc. Etc. And so it's very very difficult for us to look at to be able to kind of Build or understand in our systems sense anything that isn't an RPM And so what modularity wanted to do is trying to be raised kind of take step back from that a little bit and say can we describe things in terms of essentially source And then result in binaries where the binary artifact is just one output type Rather than being a beat the be all and end all of the entire pipeline so right now our pipeline or But before modularity started as our pipeline knew how to build our PM basically that's it It didn't know that it was dealing with terms and sources and that kind of stuff And so as a result over the years, it's been very difficult to build things like jars, right or very difficult to build containers Because they're different output artifacts than our build system knew about So what modularity is trying to do is say, okay, can we describe things in terms of the source control or in terms of the You know basically a pointer into the source tree and then Kind of inject things like spec files or whatever Later on in the process so then we can kind of build up these binaries and result in different kinds of output artifacts So kind of that's where we kind of started from and so what we led to is that okay So we can describe What the pointers in the source tree are through this thing called a module and defile Which is basically just the ML file with pointers and then we can Kind of build that as a collective set we can build them all together And we can make them rely on each other and we know that they were built together So it will work together and then we can ship that Now unify component as a unit, right because nobody actually cares if they got the Libx and L2 Right what they care about is the application that depends on it right that you know which which libraries they are et cetera Nobody generally I mean except for developers, right? We generally don't install a library right what you're doing is installing an application And so you want to know that the unit itself works together and that's where the term module comes from is that we're trying to describe The and there's a guy at Red Hat actually can call some installable things Which I also kind of like they would end up with IT and that would be funny too But we what we care about is the thing that is installable at the end point And so that's what a module is trying to describe is trying to describe the thing rather than the individual components without losing the Decomposition that our camel arms right so the decomposition is also really good because then we can share libraries Across the system you know you know simplify or the what's available et cetera So that's what we're trying to do with the module So we build the binaries, but we build them as a unit and then we can decide what kind of artifact output types We want right now. We pretty much only build our pms, and then we secondarily build things like containers But the flexibility is starting to be in the infrastructure to do something different Right so the question is what other types of or do we have a timeline for trying to deliver other types of content not really? In that what we want to do is have the flexibility to do so not we didn't really have a plan that Tomorrow we're going to start shipping jets As soon as somebody asked for it, we'll Yeah, I mean it so you know it basically is that no plans. Yeah, there's no there's no Right so the so what we want to do is have an architecture that supported that type of change Because the end of the day like the modularity stuff has some you know like everything else has some end user benefits But the goal was really about making sure that we have the flexibility in our infrastructure To adapt to what users? Want right because what we're trying to get to is kind of what I was getting to it earlier in the slides is that we need a way Let's sleep again. I Forgot to install caffeine. Sorry I also apparently don't know a password so What we're trying to do is You know enhance the flexibility, but that doesn't necessarily mean we have a goal for what that flexibility should use for You know, we just want to open it up So yeah, and then we can kind of target You know the output artifacts that we want to have One of the things that we have not gotten to yet that we wanted to get to when we try with atomic Was can we describe the output artifact as a module in D itself so that we can say okay? You know here's the set of components that go into this container type right or this iso and so we kind of toyed around with it or whatever but you know It was a lot of work so we think to it but maybe someday and Really to your point right is that what we're trying to do is enable innovation around how the distribution works and how things get done Not so much because we have a particular goal like a set of shipping gems tomorrow It's more about we need more flexibility in the kinds of things we build in the kinds of things we ship because the Infrastructure has changed for what we need And then lastly the goal was We wanted to we want a user to not experience any change at all Unless they want something that's non-standard. So, you know DNF install Postgres SQL should just work and If they decide that they want something that is not the current version of Postgres SQL Then they can go and choose something different, you know, so DNF install And this is one of the things that modules are trying to help with right is like DNF install Postgres SQL I believe gives you the client and DNF install I know I've got it backwards sorry Postgres SQL gives you the server MariaDB gives you the client So, you know with the profiles we're trying to make it so that you can actually know that there's client server available We know that but it will make the defaults to work as much as they do today so that If you type DNF install MariaDB you get what you expect But then you can start to work on multiple versions or you can start to work on different profiles By adding a little quote-unquote syntactic sugar to your command rather than having to learn a whole new application architecture, you know mechanism for doing this. I Think that might be all my slides Yeah, I can show you the Amazon Linux if that would be cool In theory go ahead the network in here is terrible and I have to bounce to another machine Yeah, go ahead. So you would allude to the fact that this could open up the possibility of shipping other types of artifacts Has anybody started to map out what that would look like how we could potentially Achieve what the curate content stream concept was supposed to So the question is has anybody looked at okay, so We've backed up for a little bit so one of the one of the challenges we have in to Dora land right is that the application like application languages Have many many libraries right huge numbers of them of huge variability and quality You know as some of you might remember, you know npm Lost one particular library and the entire node.js world fell apart for a few days So there's all these libraries kind of coming out and coming out and some of them continue to be maintained some don't it's ever It's basically impossible for a distribution a Linux distribution to keep up with that We have some of those languages are very friendly to us and try to help us do that So Python is a great example, right? So Python tries very hard to make it easy for distributions to distribute them Whereas things like Ruby don't they do that's not their goal, you know, Jim, you know, it just works Why would you need a distribution in the middle? So there was a push a couple of years ago To can we stop packaging those things as RPMs and instead? Find a way to make them available through the normal Fedora infrastructure Such that the end user can make choices about getting any, you know, library that they want But also understand its quality. So provide the information sir and licensing and licensing So and that's actually another example Why we can't like automatically generate RPMs from gems for example is that it's missing data that we require To create RPMs or it's allowed to miss the missing data as well So we're trying to figure out what could we could we share that information directly down the user from directly from the upstream so that we wouldn't have to kind of keep up and one of the Ideas there was can we use modules to just help solve that or change this infrastructure to help solve that And I would say yes, the idea has been considered but not really pursuit But if anybody out here wants to start a project doing Content distribution like that. I would be very excited about it Because it'd be really useful to have the information that of you know, like this MPM library How much is it used, you know, who's it used by? And you know, what is its license etc. Without having to have had an RPM maintainer look dig all that information up, right? So Oh, sorry, I was gonna show you guys a demo In theory See how many blank don't worry about it. It's hard to run the OS You guys see that right? All right, so this is just all I do is download their Some sort of image I can't remember what type is KVM I assume and You know kind of started digging around so if you do So oddly enough It seems familiar kind of looks like the DNF module list So basically these are all the different topics you have It's a little unclear to me whether it's a topic and then it's a version of that topic or if the versions are Topics themselves like I just don't quite understand their link up But it's probably some idiot not because it's bad So if you if you see there, I actually have three versions of postgres enabled Because I wanted to kind of see what would happen and it will now pick postgres tech, right? Because it will always choose latest and But you kind of just You know do what is actually just see if I can Easier Yeah, so you just see I just enable it right? I mean it's not you know not that complex Not just similar from what we have and then I would go along and install it and You know and then I get the the version that I was looking for the thing is that when you When I think it's kind of interesting Is if I try to do yeah, so if you notice the top line there So This is kind of interesting But basically so if you have installed a library or a thing from the topic and then you know kind of remove it and disable it I'm not actually sure what they're worried about but they seem worried that this may not work So like you know as far as I know young DNF, right should be able to handle this pretty well And without so much trouble now. I'm curious But I saw in their documentary, I can't type all over my shoulder So it does say that they have DNF available to but now I have to go day to figure it out Right, right, yeah, I'll go full around at some point now. I'm curious Right so the statement is just that they replace the other version of it available, but if you notice I was doing some searches before Not everything that is a topic is available currently so there's no gift for example The only gift you can get is through this top But Tomcat there is a like a base version available and then there is a topic for a Tomcat 8-5 I think that the base one was seven maybe So there it's not necessarily required that the thing already exists and then it overrides. It's both So okay, so the question is what happens to the base the Tomcat 7 that's available in the base when you enable the Tomcat 8 Topic it's actually exactly what would happen if you got Tomcat 8 from RPM fusion Is that you just now have two repositories one has a higher NDR and so you get the higher NDR. That's it Good and it's mostly just commentary. I find it interesting that mate desktop is 1.x, but rust is just one Yeah, so the comment is basically that the the naming of the versions here, right? So mate desktop is 1.x and what was your example rust? Yeah, so it rust uses the numeral one. I actually think we're gonna have the same problem in my library Well, it's good and bad so the idea is that some things are Good about Basically doing like z-stream being only bug fixes right and y-stream being Your backwards compatible and x being you know major changes not everything PHP for example There are breaking changes between PHP y versions So the naming convention you end up with is what policy do they follow? and so Rust in theory and I'm not sure if these are good choices or not for the individual examples But rust in theory is doing a good job of only doing breaking changes on x's and mate desktop is only doing a good job on y's Does that make sense? Yeah, so Apache actually has this problem too Even though like they are actually really good about not doing breaking changes on minor versions people don't trust them So people really want Apache or hVD 2.2 versus 2.6 So, you know, we tend to use the minor version there too. Sorry go ahead No, my point was just that if it is a breaking change it should be a separate stream Right, so the statement is that if it's a breaking change to be a separate stream Yes, so this is actually what's more equivalent to our streams So we would call the module itself mate desktop and then we would have a stream that was 1.19 like this Actually, what we this is where the thing I was talking about really would happen So you would have maybe 1.19 and 1.20 as two separate streams because we didn't trust the live versions Yeah, so my point was so you have like post-crest rule 9 6 and that's like a stream that that tracks something That will have Z stream updates whereas mate desktop doesn't have a 1.19 and 1.20 Pretending that those are breaking changes. I just thought the inconsistent naming was odd Well, so this does not seem to follow it doesn't it doesn't well So the comment basically is that the the naming is odd and I will also point out though It's partially because they're using the evening so they need to inject Information into the name. No, it gives you what how to choose the right thing. Yeah, I know I'm sorry I don't think we're missing we're talking past each other Yeah, it doesn't follow any convention Right It's Brendan Stern So is tempting as it is to debug Amazon nomenclature do we have good guidance for stream names and fedora? We're starting so basically the question is Do we have consistent or do we have policy or whatever documentation around? How to mean things that are related to modularity? we have Basically a working draft It's not it's not approved yet. It's close And it's mind-boggling how messed up it got so quickly So literally so I'm talking about the different profiles for example And when you're talking about a database, right, it's pretty easy client server, right? But let's talk about say Apache or something like that where you might have a dev version versus the production version Yeah, we literally have dev develop development All meaning the same thing on different profiles of different models so Yes, we have a working draft. No, it is not implemented and oh my god. It needs to be So that we have this kind of convention the other problem we have let me speak one more point is Is that sometimes we are going to want to name modules by With some bit of name-mangling by using some version in there for example Python where we might want to do allow for parallel install ability because First of all, Python can tolerate it second We have a Python that we use as part of the US that is not necessarily the same as the Python that the user wants to use So we need guidance on that as well so that Right, but I want consistency in the naming in like how you name the version of the names of the module So that if it comes up again, they don't get inconsistent But sorry good Yeah, really it should be in the next few weeks Yeah Yes, that's in there too. Yeah any restrictions. I mean it's policy that we expect to change You know, it may not be a hundred percent right the first try But we got it we're gonna start with something and say here is the mechanism By how you should choose your screen names and what and how you should you know Separate it with hyphens or whatever Just so that we get consistency. We may want to change it over time, but at least we'll be consistent. Yeah, you know One question I have is Especially when we use like traditional text managers, etc in comparison to containers Which you brought up at the beginning somewhere that all this should Somewhere move into the direction of modularity is well Let's say I want you install a next cloud and some other php application and one needs php5 one needs php 5 to 4 whatever when I use traditional package management and it's defined like this with Relationship to all the test isn't the dependencies Of course this may cause in the conflict and then both or at least one of the application is full drown Is there any Intention to change this or is it just so Let me share a paraphrase. So basically the problem is you have two different applications. They're not using the same shared libraries So we refer to that problem is the parallel install ability problem And there is no intent to fix that at the moment You should go use two different user spaces and the reason I've been using the term user spaces, right? So that could be in containers. It could be DMs. It could be you know, kind of whatever mechanism you want to use But what I can help you with is that you need php5 and php6 They'll both be available So when you build your container to do both applications, you can actually get those applications both of them Even though they're not using the same versions of things The other thing that's kind of interesting here, too is because we are using this defaults infrastructure if You have a container that is you know using php and you know that you want it to walk You know major versions or you want it to walk like something you want to walk major versions because we are doing this default stuff You can actually inject a different file into Etsy With a different set of defaults than what is standard. So when in your docker file, you can say DNF install php, and you'll just get the right version So that's also kind of a cool thing. I think Adam knows this stop. I can't tell so But yeah, yeah, thank you. Thanks for coming