 Good afternoon everyone. I'm Mohan Bodu and I work as a releasing engineer for Fedora and This talk is about how we are doing ribbons and mass branching in modularity and This talk is for everyone who attended the last talk so everyone and who are trying converting their packages into modules and trying to create more modules so I Before starting I just wanted to ask how many of you know how we do mass ribbons and mass branching in regular RPMs Okay, not many probably one third even in this one. So And how many of you know what actually mass ribbons and mass branchings are for? Okay some of you so Anyway, I want to give a brief introduction about mass ribbons and mass branching what they are and how we do Within RPMs and then I will talk about how we are doing them in modularity so Master is nothing but rebuilding all of your all of the Fedora packages that they are like all the active packages So the reason that why I do it is if there is a change in G-Lib C ABA change that affects on Indirectly or that directly all the packages in the distribution so we rebuild all those packages and In the recent mass ribbons that we We started like two weeks back. They were about 22,000 packages. We rebuild and The way that we do mass ribbons in RPM namespace is we create a separate site tag for This master bill and we use the site target to build all the Packages and once all the bills are done, we will merge them back into the raw high tag so The way that it is done right now is Alpha numerical order, which is not great because if you can do it in the If you can depth solve and do it from there probably there will be less FTBS FTB FS which are fail to build from source but currently we don't have that so well But I don't know how many people have attended the Improving package experience and automation talk half an hour ago So one of the things that they were talking about is rebuilding all the packages that depend on your on your build automatically, so if they do that probably we don't have to do mass ribbons because Gillipsy changed a beer changed all the let's do all the ribbons and I don't have to do this back. Maybe I don't know Depends they might add a flag like Right, you don't have to do Yes, so I'm not saying that's That I'm not saying it won't solve the problem. It might solve the problem so mass branching Mass branching is the one where we actually create new release from Rohit, so Currently F31 is Rohit August 13th is when we branch F31 From Rohit which at that point Rohit becomes F32 So the way that we do it is we create all these branches in PDC, which was product definition Center Merlin just talked about it and Then we add all the branches in disk kit for you guys and add the tags and targets and stuff like that and then The F31 branch which we call at that point is a branch trees will be ready and After a few months of testing and stuff like that. It will be released as Active release So that's what mass branching is Yeah Sorry Yeah, yeah, so just like Rohit we do a nightly composers for branch releases as well So people there are a lot of people that are on follow branch releases Yes, we enable composers. Yes. Yeah So that is also part of mass branching itself So Can anyone guess how we are doing mass rebels and mass branching in modularity except you Okay, so Let me reveal the surprise Before we go I got there itself or let me postpone it actually so in order to understand mass branching and mass rebuilding modules you need to understand stream expansion which Both the process depend on so if anyone have Built modules in federal you might have known about stream expansion like it will when you used the Build requires and requires her streams you can specify Sort of reggae kind of a thing if you don't specify anything it will just reveal for all the active releases or you can specify Just particular releases that you want to build for Like 20 f31 and f30 f30 so it will only be for for those two releases and You can say minus f30, which means it won't build for f30, but for all the other active releases so it's really important to understand this one because mass rebuild and mass branching or Totally depend on this one. So mass rebuild in modularity. So There are two types of requirements one is built-in requirements. There is one time requirements So mass rebuilding modularity looks at built-in requirements. So it will look at what platform does your The build type of what platform is the built-in requirement for your module? so you will specify that you are in your module and defile and Then the our tooling will look at This module MD5 does a stream expansion thing and identifies if it is Buildable for f30 or at this point f31 or not so and If you're doing for mass you will for the f31 so it will identify it and it will rebuild for that platform But here is the thing it won't just rebuild for f31 It will build for all the releases all the streets and I will explain why and What's the problem with it later on? But We use Rebuild strategy strategy as all Which means we are asking MBS to build all the package packages that go in it as well not just to reuse them rebuild them all and There are about 150 plus modules currently Which we need to rebuild Actually That one Needs to be started last week, but we hit some issues. So I Have to start it sometime this week. Oh Before before going to the next slide. Okay now back to that question that I asked can you guess how? mass branching is happening in modularity No, it does Actually, there is not much of difference between mass rebuilding in modules and mass branching in modules We are just doing mass rebuilding again, but by adding a new platform and Which will be the new raw hide and we are using a different rebuild strategy because at that point of time Nothing is built for that new New platform so rebuild everything for that platform for the old platforms We don't care if you are you going to reuse the Reuse the whole stuff. That's fine Yes, we just rebuild them and by adding the new platform it will rebuild everything that depends upon the new platform So the modules team expansion again plays a part here if you specify nothing that means it consider The new platform as an active platform and rebuild it so The two problems that we talked about is We are rebuilding all the modules For mass rebuild we are building a rebuilding all the modules for all the streams And for all the for all the platforms So the reason is it's hard to identify the dependency chain Which module stream is depending upon which? module stream off for what platform so Unless this guy finds a solution for that we have to rebuild everything and The other thing we are doing is rebuild strategy all for mass rebuild We are doing this because of the reason that I explained a few minutes back is we Let's say someone committed some changes to the disk gate and they didn't build it and This one will ensure that everything is built and Then the module is built on top of it. So That's the reason why we are using a rebuild strategy as all So that's kind of how we are doing mass rebuilds and mass branching in modules Questions So So the the question is basically if there is a Package that depends on a module and then it depends on Another package, how do you do mass branching in measurable? So the package part which is an rpm level thing which will be handled normally as it was doing But when it comes to the modules It will rebuild the package as well No, no, no, we don't have that dependency depth solving thing yet. So Yes, yes Yes another module So there is a thing called FTB FS that we do like fail to build from source we file bugs So if you're during the mass rebuild process of our pms if your package will fail I mean it might So you do it and Yes, we I don't think we have a solution for that right now probably it's more like hey this bill field You go and deal with it kind of We do But again, we don't know where to start right it might be like a module depending on rpm and then depending on Like probably we have to do twice of each thing Sure I'm not sure how many packages our modules will get affected by this And again, one of the thing is we in mass rebuild of modules we rebuild everything Doesn't matter if the things got changed or not for all the streams and all the platforms No, no, no, but that like based upon how many modules are affecting or packages affecting for that Do we really have to go through this or just let the maintenance know about them? That's why I said we have to do Yeah Sure, but at the same time even the normal RPMs if I show you that one Third point we are doing alphanumeric order even if a package depending upon other package Even that might fail So we don't have the tooling right now to go through the depth solving and like identify which have to be built first and then Yes, yes, I actually that's why I mentioned that it needs improvement, but Yeah Modules that need Yes So the question is Have I encountered this module bootstrapping where a module depends on its own or a module depending upon Its own thing, but on a different stream. I have not so And again, it goes back to the same discussion So we will try to rebuild it in the order that it sees it and it might fail or it might not So We don't do Any check-ins when we say master build we rebuild everything Every repository everything it is every active repository we go into every active repository I mean that those things might fail. So that's why we fight against Yeah By repository Yeah So Probably it was So we use PDC to identify the active repositories or active packages I've seen like a couple of instances where it doesn't have the data Probably when we created the PDC entries in it So if it doesn't have the data it thinks it is a retired package So that's how that might have skipped it So if you have seen it again, please create a ticket In And you want to be the message like master built for for modules and you'll be rebuilding all the branches for No, no, no, we only do it for a ride Only right, but for modules we rebuild for all the streams that are active That's what we I just said like rebuild all modules all the stream all the active streams All the modules for all the active releases that was one of the problems because we are not able to identify the dependency chain of what module stream depending upon what other module stream on Which was built on top of a different platform Yeah, we do come it I think the commit might say like a master built for federal F31 or F30 like depending upon what release we are but it for for Modularity we do an empty comment and the district Yes The active module streams We use Yeah, so the question is how do We identify what are the active streams and what are the active modules we use PDC which stores all the old data and Based upon that we identify what to build and what not to build So when you say what when you say Retired I don't think there is a direct way to do it like How you retire packages like fit package retired. I don't think there is one for modules You have to create a relinch ticket and we will update the PDC entry for that same like today is the retired Yeah, but we never got Yeah, any more questions. Thank you