 Hi. Thank you all for braving the maze to get here. It proves that you're all the best and bravest, and I'm glad to get to talk with you. I'm glad to see that none of you got eaten by the grue. Anyway, I'm Will Woods and Stephen Gallagher, and we are here to talk about eliminating scriptlets for fun and profit. It's not actually that fun, and there's no money to be made, but we're doing it anyway. And I am sort of gratified that you're all here to see this because it means that we're actually going to try and do this thing. And I kind of have a question I wanted to poll you on. Are you more here to figure out why we're going to do this or figure out how we're going to do this? So, why raise your hand? Okay. And how? Okay. Good to know. I'm glad you all are with me. Oh, the cursing that I'm trying not to do. Anyway, so what are scriptlets? Well, it seems like you all already know that, but I'm going to run through this just because it's good to make sure that we're all on the same page about what we're actually talking about. So, scriptlets are the thing in RPM. It's like a shell script, usually, that runs in one of several, so many different hooks during package installation. There's a million of them. Pre-trans, pre-in, post-in, pre-un, post-un, post-trans, trigger-in, trigger-un, trigger-post-un, trans-file, trigger-trans-file, etc. There's the 20-something. And what they're for is basically in practice, we use it to handle anything that RPM doesn't or can't do on its own. Things like running LD config after installing libraries, adding and removing users, enabling newly installed services, generating SSH keys and things like that. The problem, what is the problem? Well, I think you all already know what the problem is. The problem is that it makes everything bad and it makes me personally cry at night. It's true, I've seen it. But, I mean, here's the deal, right? Everything runs as root, which is horrifying. And one of these times, I am actually going to set things up so that I don't think Adam is here, but during the talk, his computer is just going to catch fire or something because I'm going to put scriptlets on his system because you can do that. It's right there. Oh, good. Well, this time, you're safe. But the fact that we run these, we execute arbitrary code as root in the middle of the package transaction every single time we do every package, sometimes like multiple times per package, makes me lose sleep at night. They're weird and they're hard to maintain. Like, everybody who had, nobody, well, there are some scriptlet experts, but most packages are written by people who don't really want to deal with packaging, and so they do the bare minimum to get it working. My favorite is that a lot of the Java stuff is packaged. All the scriptlets are written in Lua because the guy who wrote it, the person who wrote it, wasn't a Linux programmer at all and didn't know Shell, but they figured out Lua pretty easily. So they just wrote it all in Lua. So it's doing things like making soft links, but doing it in Lua because they didn't know about LN. It's fine. That's totally valid. We allow this. It's really hard for anybody else to maintain. It makes everything super-duper slow. I think Lars is here or was here, but hey, Lars, you can verify for me. It's very slow. It's very slow, see? I mean, we... See? We all play with Lars. Yes, Lars, absolutely. There's a demo I've given in previous versions of this talk of a prototype tool that my team built that basically does a system install, but it does it rather than doing all of the RPM stuff, it just kind of takes all the RPM payloads and mushes them together and emulates the scriptlets. And usually an install like that takes about 10, 15 minutes. We do it in six seconds. We haven't even bothered optimizing it yet. It just takes six seconds. It's IO bound. It takes that long to write out a tar file. So scriptlets are why everything is slow. Everything we do with RPM could be hundreds of times faster and way easier if we just get rid of them. And I know you're all aware of these problems, but it's good to sort of have a shared understanding of this so that we can explain it to everybody else so they stop getting in our way and making more scriptlets happen. They're also, they are under designed. And I don't want to pick too much on RPM. No, I really do want to pick on RPM a lot, but I'm not going to because it's kind of mean. It's just like taking candy from a baby. It's just you feel bad about it. But the candy is delicious. The candy is delicious. But yeah, it's kind of being to beat up on a package format that was designed in like 1990 and its main goal was to be fast when you loaded it off of a floppy disk. Like it's really good at that. RPM is great if you really need to read the header quickly off of a floppy disk. We don't do that as much as we used to. So it's not super great anymore and it's kind of become a problem. And opaque and non-terministic, this is the thing that really kills me is because I want installation and upgrade and image builds to be safe, secure, reliable, and fast. And as long as we're executing scriptlets, it can never be any of those. So anyway, just to say, the reason for that is when we want to build an image, like you said, it's a lot faster to do an install with no scripts and then make a few changes after the fact. And what we want to be able to do is introspect those spec files or the disk it and see, oh, this package wants these users and this thing in here. And so we can reimplement that for the image builder, do that once at the end, a single F-sync, and we're talking, what, three orders of magnitude performance increase by doing it that way. Which is significant, especially if we're talking about things like CI. So this is sort of a long game that we're playing because we're not going to get rid of them overnight. A lot of the complexity of building the things that we build has, over time, accumulated in scriptlets. The point where there's tens of thousands of lines of code that nobody really understands, but if you don't run all of it, your images don't work. And so we sort of have to slowly figure out how to replace what all of it is doing. We have to go through all the scriptlets, find out what all of them are doing, and replace them all with something else. And that's not going to happen immediately, so I can't just be like, today we've deleted all the scriptlets, now let's all drink to our heart's thought. That's not what we get to do today, unfortunately. But we'll get there. Anybody want to guess what that does? It does. This is one of what, eight different ways we found that people do that? You know, greater than, and then the file name and bash will create the file for you. There's like 18 different ways you can create a file. And so it's really hard for us to look at scriptlets and tell, oh, this one's creating a file. They're not introspectable. So we are, like, we're going to end up as a community having to manually read through every scriptlet in Fedora. And that's not great, but every package has a maintainer, right? So it's really, you just only have to look at your own packages, hopefully. So one of our goals is to write documents that we can help people who may not be deeply experienced packages figure out better ways than what they've been doing. Right. The thing that we've messed up historically is that we didn't give packages the tools to do the things they needed to do, but we gave them, you know, the Swiss Army chainsaw that is shell scripts, and they've used it and sometimes cut their own fingers off, and sometimes ours, and sometimes we cut off our own fingers. They put this line there. And also we have to have dev for that to work, which means that now, because some scriptlets want to have, because we can't predict whether or not a scriptlet will need dev, and this one will break if dev isn't there. Therefore, every scriptlet has to add to an environment where it can see slash dev. And we just have to hope they don't do anything bad. So here is a fun bug. This, this, we had a bug in Fedora 26, specifically in server. Server doesn't come with the GUI enabled GUI on your server, so you install GNOME or KDE, and it wouldn't come up. Like, no matter what you did, it just wouldn't start. And, of course, the bug, the fix turned out to be in the Fedora release package, and it turned out to be that we forgot that percent sign. Obviously. It's... For those of you who don't know Lua, which includes me when I wrote this, the difference between this line and this line is that this line will replace any... This line will remove the set of files that starts with 8 and 0. This line will remove any file that starts with 8. Right. Yeah. The other terrible thing about RPM and spec and stuff is that not only is it, like, touring complete, or several touring complete languages, but it's several of the worst touring complete languages you can possibly use. Yeah. Lua and Shell. But the real problem here was less that the script that was wrong and more that it took some seriously deep dive, some serious deep dive, and some genuinely dedicated individuals to figure out that's where the problem was. It is an enormous maintenance burden for the people working on Fedora and people who just want to be packages. My big goal is that packaging... Like, using packages should be the same way we use libraries. Like, when you run a dynamically linked library, you don't think about how linkers have to work or anything like that. When you build code, you don't think much about libraries. It just kind of magically happens. And packaging, in a way, is kind of like your dynamically linking a file system, except everything is terrible and nothing works. And I would like it to be a lot more like, you know, you assume that when you run LS, it's going to figure out how many libraries it needs to load. And there's a lot of them, but it just happens. We don't think about it anymore. That's how packaging should work for people someday. In fairness, that is the result of an awful lot of work from an awful lot of individuals. What is? That linking works that smoothly. And it's our turn to do the same for the packaging process. It is our time to sort of figure out how to make the file system the way that you build a file system image work as reliably and predictably as the way that ELF and linkers and loaders make your memory space for each process. And if you don't know what I'm talking about, it kind of proves my point. You shouldn't need to think about it. Anyway, oh, yeah, here's all the here's a list of what happens when you install a package. For every package and so RPM has to run fsync before and after each of these steps. And so when you're doing a package or a system upgrade, I don't one of the things that I did before this was I worked on upgrade tools. I actually have written like four of them, sorry. But pre-upgrade and then fed up and then system upgrade. And part of the reason I'm so mad about this is because the reason that upgrades take like an hour to run is because they have to do all of this. And the ironic part is that your end system something like 80% of the files are unchanged. So like it takes an hour for you to change, you know what works out to like 10% of the data on your disk. And it shouldn't be like that. But this is what we're doing with our time. So and it's very common like 50, like half the packages in your system probably use it. We already have established, but I'm going to tell you right now we've done prototypes. We do not need scriptlets to make working images. We've all there's more than one of us in the room that has written an image build tool that skips all the scriptlets and everything can be made to work. So like we can do this. We just, and I'm thanking you for coming along with me on this ride. So yeah, what are we going to do about it? And I did, yeah this is so yeah. At some point I did actually go through and read every single scriptlet in every single package in RHEL. And I did a lot of it in a bar because you have to have alcohol when you do that. And what it comes down to, the way that we are able to build an image in six seconds is because I spent way too much time figuring out what scriptlets do and realizing they don't need to be there. There's six things that they do. And if I'm wrong about this, please, we will talk about that later because I really want to know. But like nobody has been able to come up with an example that falls outside of these six tasks. I would like to hear some because I do want to make sure that we're getting, we're giving people tools to handle the things they need to do. So first one is users and groups. And this is new. So thank you to Harold and company. We have, we're introducing some new macros. We're trying to get this in force. Wow, that is entirely unreadable. That's really unreadable. Sorry about that. Well if you want to go to the OS build link here, you can see the file I pulled that from. But we're going to add some macros to RPM, hopefully in Fedora 30. So rather than actually adding users and groups, manually in your scriptlet, you just kind of do this stuff. So this is going to be part of a two-phase approach that we want to take to changing out users and groups. The first is how many different ways did we discover that people were doing this? I think it was at north or south of 11, I forget. Creating users and groups? Yeah, in different ways that people have done that. Despite the fact that we actually have clear guidelines on how you're supposed to do it, there are definitely more than eight different ways that we have seen people doing it. Yeah, I found a binary, I found binaries, I didn't know there was a command in the password or in the PAM package that can create users for you. You didn't know that? Of course there is. So the first step there is try to get everybody to move over to using a set of macros to accomplish this that we can then centrally manage to make sure that everyone is following exactly the same process to create them. Further down the road we've been talking with the RPM maintainers and they are willing but not yet sure how to make this a first class feature of RPM. And so when that day comes we'll just change the macro definition and all these packages will automatically be using RPM native mechanisms instead. Right. And that's the goal and there's an old saying that I just now made up about how you have to build a bridge from both sides of the river. You can't just go forward and you can't just be like, we're going to burn it all down and start on the other side. We have to deal with what we have now and figure out how to get to the world we want to be in. So this is how we get from the world we're in now to RPM natively managing the things we want to manage. One of the interesting things about this that I think will be retained is that one of these things may be in the files but it's going to also, it does generate a prescript and you're like, well, I thought we were getting rid of Scriplets. That's more Scriplets. Well, yeah, we'll get there. Because one of the things that actually happens here is that it drops a file into place that other tools could read rather than just blindly executing the scripts. We can look at the, there's going to be like a JSON file that says it describes the users that would be created and your tool can find that file in the package, well through the package headers read that file and be like, oh, this package wants the users and then do the right thing so you don't need to execute code at all to make, well, inside the image to do it. So yeah, that's users and groups. We have a story for how that's going to go forward. There's more details in the packaging committee ticket and there's one more thing I'll mention. You'll notice that the if you can read it, that the scripts do not say, the scripts are osbuild underscore group ad and osbuild underscore user ad and that is because we're trying to make this mechanism at least a little bit more open-ended as a framework so that if there are other common scripts that we discovered, we can merge into this. We'd rather try to incorporate into the same framework and save ourselves a little bit of work and simplify how end users use it down the road. So we're leaving ourselves room to add more of these things, but we don't have specific plans for them yet. So for system cuddles type stuff where you're starting, well, where you want to start or enable a service because it's just been installed and you need to set it up, that's what system view presets are for. Please use them. I get very angry when I find that people are calling system control in their scriptlet. It makes me very upset. I do believe there is a packaging guideline that says you were not supposed to run system control in your scriptlets. So don't do that. There are very few exceptions that are occasionally granted when bad old behaviors can't migrate to the correct behavior but those are only allowed for a release until you fix it. And do you have to go back later and delete it? Yes. Good. Network manager recently went through that. Yeah, so the short answer is if you have or see a, if you see something tell someone. If you see a package using system control in a script, tell the maintainer to stop it. Or start. I've seen packages that system control start really bothers me. Which is not great if you're trying to build an image or the package is installed in Anaconda. Right. Yeah. Don't you have any services that have to be stopped during that update? That is, they can survive updates when they are running. Don't you have such services? So for the purpose of this talk I am ignoring updates. Because frankly I don't give a shit. I however do. Yeah. That is actually a case that we probably need to do a little more consideration on. Talk to us after the talk. Yeah. We do need to think a little bit about that. Right. When I talked about the six tasks. Oops. This is only during initial installation. There are other things like obviously migrate config files and things like that sometimes. So there are other tasks that happen during updates. But even still a lot of those we'll get to the next one on the topics. Some of what you're saying about upgrades will be resolved by this next one as well. Right. We should use the presets in the current system but isn't there also the rule that we should actually call system set presets that stuff. So the question for the recording was should we make sure shouldn't we have permission to call system control preset to apply certain things. And the answer to that is the standard system d macros that you're supposed to be using will handle that when it is appropriate. Right. You shouldn't be typing it yourself. But your script shouldn't be calling it. So empty and default files like the example I showed well that's what it's called. It can create an empty file really easily. Most of the cases it can also copy in a default configuration. I'm going to put together examples of these at some point. I keep saying I'm going to do that and then it was like Christmas time and I didn't feel like it. But somewhere I will be putting together some examples of how to do common tasks and especially if people want to tell me I need to do this how should I do this. But most of the time where you're copying files around or moving files around and even I think temp files can do minor edits to files like if you're setting a default configuration parameter those things should be handled by tempfiles.d which is an increasingly misnamed tool but whatever it's fine. So if you're creating system specific data you should be doing that in a specific service and I will let Steven talk about this. And this is going to answer some of the questions about this upgrade. So one of the problems with generating images or copying something to be a base for a virtualization that you clone is that if you are generating system specific data or need to do some transforms on an upgrade doing that in the package updater not a great place for it because chances are there's a good chance in the modern system that that's not actually executing a running environment anymore. It may be executing in a container build or a virtual machine build and so what we want to actually convert people over to doing is if you're talking about any kind of a service you want to add on a little helper system D unit that fires as a dependency on your main service who starts up and sees do I need to do any initial configuration or upgrade process do that and let the main service start. So if you upgrade your 389 basically this would just wait until your next reboot or your next time you restart the service and it'll do the upgrade at the start of the service instead of during the package upgrade which is a lot safer too because you don't actually know what state the system might be in in the middle of a package transaction. You can do this and run it in the end state. One example of one of these that we did fairly recently is Apache if you have ModSSL installed the first time when you installed the package the ModSSL package it used to look at your system see if you had a certificate installed in the standard location and if not it would generate a default self-signed certificate. This was really bad for image generation because first of all all of your images would have exactly the same certificate that we were going for and beyond that even if you deleted it from your image you still had to instruct whoever was cloning it on how to recreate it for that image. So we added one of these services and now whenever the first time you start or if you delete this and restart it you'll get a freshly created certificate and that's also side benefit that's now happening in a running system where you've had time to generate some entropy so you're not running the risk of creating the certificate hanging your image creation or your install process. So yeah in general if you're creating any sort of system specific data you should be doing it using these sets of guidelines and yeah there isn't really much more to say about that we've got this one sorted out please don't do that in your package scriptlets so for system config stuff this happens sometimes where people like hard roll rules or like turn SE Linux on and off or insert a kernel module holy hell please don't do that I don't like it when I'm trying to build an image and during the image build the image build tool tries to insert kernel modules on my host system that's not cool man please don't do this to me so basically there are zero cases where this is an acceptable solution and there are non-zero cases of them in Fedora right this is a bad idea change my mind and not this change my mind afterward if you don't mind though we want to get through this slide the last the last is sort of a big category it's caches and catalogs it's dynamically generated files that I am forced to admit we do need for a working system so file triggers are great for this LD config is the canonical example every time you install a library you don't need to run LD config but you really do want to so in Fedora 28 we handled that automatically using file triggers we kind of I guess didn't tell people or maybe we did just a memo it's not enforced I guess we can enforce it now because as of Fedora 30 there are no scriptlets that run LD config anymore it happens automatically you don't need to worry about it it is not your problem so we're working on equivalent solutions for install info when you have an info file for GTK update you don't need to crash on various other tools that are like that if you maintain one of these tools and you aren't using or you don't have a file trigger set up to handle the thing where all of the descendant packages need to run the thing to get themselves hooked into the cache or catalog talk to us and we'll figure it out etsy alternatives is one that sadly is still necessary and sadly is still in the core system in that we ship even the most minimal system possible still installs bin noodles and we ship two load or two linkers because of reasons that I don't want to think about so the plan there and I don't know if anybody who works on that upstream is in the room but I've filed a thing with them and we've got like vague consensus that yeah it'd be a good idea to have a config file format for that and you just drop a file into place and then things happen with normal file triggers nobody's written that code nobody's written the file format I proposed one but we haven't actually done that so if anybody wants a nice little bit of thing that they can contribute to upstream and you know be a hero that's something you can work on talk to me I should put a link there that's really about it those are the six things we know about these are our plans for getting rid of them if we all work together we can roll this rock up the hill and be done with scriptlets once and for all eventually so yeah are there any this would be the place I'll let you say what you were going to say sorry we'll start with your question so there's a scriptlet that I know enables a boolean and that's what the so there is at least one package that uses sdlinux stuff to enable a boolean and the sdlinux guys that was their suggested solution do I understand correctly? it doesn't sound to me like it was this let me try to rephrase the statement too so there exists a package it's a debugger of some sort okay the crash handler for KDE needs access to ptrace permissions and the sdlinux guys didn't want to give them generalized access for the package so they said that users should just enable the ptrace sdlinux boolean and so they put it in their scriptlet I'm going to be honest that makes my skin crawl that should absolutely not be something that is done without a user's express consent I think we need to find a better way to do this but we don't think it makes sense either to be honest yeah KDE doesn't agree with this decision either and I would be happy to remind me about this later I would be happy to go and talk to the sdlinux folks about doing that right I think that's a good example of a place where well all you really want is for sdlinux booleans apparently we need some solution for packages to be able to enable or disable them they should have a tool that has a drop-in directory you should drop a file in there it's better than the scriptlet solution but I still don't agree that sdlinux booleans should ever be changed without a user's consent I don't get to decide that but I do get to decide what are we talking about I'm sorry the question is if we've seen any packages that dump their database to a file before doing an upgrade isn't that the recommended way to do an upgrade for MariahDB or MySQL I feel again that's gross and it makes my skin crawl that should also not be happening in a scriptlet we should be talking to them about doing that in a helper unit that seems like something that should happen as a helper unit one of the other fun things about scriptlets is we don't have any idea how much disk they're going to take up which is why sometimes you try to do an install and it's like in midway through you're out of disk because scriptlets can do things like back up in an enormous database without telling you and then you're just out of luck and that should happen that would make the upgrade unreliable whereas something like a service unit would mean that the upgrade succeeded and then your individual applications had an issue that you can then go and resolve that's an interesting question I think that probably there needs to be some sort of so when we do start talking about and at some point this talk should include we should start talking about upgrades and migrations and probably part of doing an upgrade or a migration would involve backing up that might be a thing that some stuff needs to do I can't assert that that's never going to be needed and so I'm not saying you should delete it mark it though put a be-all-comment around it that says hey this is doing an enormous backup and then I think in general if you see we should start marking things like that so we can figure out which packages do this sort of thing and then start figuring out what we actually need to do and also to be able to pull those things out and put them in their right place Florian has been trying to interject something Florian, I'm sorry? I said there is a way to collect more data that can refer to tasks but you can't be able to put ghost files into your package to make room for doing stuff and it's actually done by a cargo package sure we will discuss you can reserve space with ghost files but it doesn't work for dynamically created files so if your file grows depending on the number of other things on the disk and there are a lot of other things on the disk then we have no idea what's going to happen oh boy thank you for that so depth mod is a catalog it's a list of two of the kernel modules that are on the system so it is a dynamically created file that does need to be created for proper function it should be probably handled by a file trigger something should notice oh hey there's a new kernel and then kick that off at least in the RPM world and then something like we have to agree on what that path is and then other tools can key off of the fact oh this file now exists and now I know what to do to install a new kernel so both of those really I don't know what magic happens in new there's a lot of weird magic that happens in new kernel package like that's also what it does it adds your new bootloader entry and things like that you're creating dynamically made files that do need to be there and written correctly for that to work so yeah file triggers should happen there we'll work with you and by we I mean he pretty consciously been avoiding thinking about the kernel case because that is definitely the most complicated one we have in the distribution at this point yep it's updating a cache it would be very nice to get rid of all of them but if we manage to get it down to that's the only one left I'm still going to throw a party there so depthmod in file triggers works in other distributions I repeat for the recording that's good to know cool any other questions or okay so to repeat the question back it's basically what do we do about people who after we've had someone like Igor go through and do a mass package update to remove the scriptlets end up adding it back in either intentionally or accidentally because they're keeping their spec file in matched with Apple which may not have that functionality I think we probably want to take that one a bit to FASCO I think we should probably be setting a rule that they're no longer allowed to do that and just start writing some scripts to monitor the that tar ball of all the spec files and just see if those get re-added the autofile a bug and say no you can't do this because I suspect that more often than not that's happening by accident rather than on purpose so I think that will probably work itself out if we can automate letting people know about it so oh actually I have I wanted to propose sort of an idea okay so one of the problems that we have here is we have to go through an enormous amount of and luckily we have talented people who will actually write tools to parse all of the scriptlets and try and find places where you're doing naughty things and fix it for you but we can't always do that and I'm thinking that one of the things we might want to start talking about is whether we should split scriptlets by task or you can have two post scripts but if you have upgraded stuff that goes in a separate post script from your initial install this may be a thing we want to do that would require changes to RPM I think no you can somebody tell me if I'm wrong but I'm 90% sure you can have multiple post scripts and it just runs them in order you can have multiple post scripts but the last time I tried it only the last one you wrote ran okay well we'll not forget I said that pretend I said nothing we'll figure something out anyway it was the last one I think there's six this is the only one about yeah seven years ago it did a distribution but it just shows that it's possible and also in terms of all the stuff do you mind if I ask which distribution has already done this because I can't read that oh okay well that's good to know because we should really be exploring what you did and try to replicate so you're saying that it's really difficult so wait did I hear correctly you're saying that Alt-Linux has a config file format for alternatives we use a different config format for alternatives but we do this we don't have one at all that doesn't exist I sort of proposed an idea but we never wrote it but you have one let's talk okay we're gonna steal that from you thank you for writing it it's stealing oh it's open source we're collaborating in a unidirectional manner absolutely could you elaborate a bit how would you like to not just edit markers but how would you like to implement this I'm actually gonna if he's willing to I'd like to have Harold explain this one because you're closer to it than I am these macros call out these macros call out to some Lua code which then internally stores them in a big variable and then extends this JSON string with the data which groups to add and which uses to add in the beginning and then when you call the OS build pre generate a pre script like you had before with the user add it should be even with systemd users if you want to and the OS build install thing will dump the JSON file basically to disk install it or can render the temp files or systemd sys users files also depends on what we like to have it to render and the OS build files just pick up the file name it just generates the general concept is that the OS build whatever macros build a configuration in memory and then we have other things that write out the appropriate things to do whatever you need to do in pre or in install to actually activate that configuration on the system the nice thing about this is if you're not if you're installing the package but you're not going to be running scriptlets the configuration it generated is still in the package and so we can make the changes we need without running those scriptlets and it's in a format that is easily parsable that's a lot of spec file work it's significant so the question is it's a lot of spec file work my return argument is it's less complicated than the script it's replacing and it is extensible for other tasks there must be a better way to do this there must be a better way to do this is always going to be true I mean the better way to do it is not do it and actually I do believe that Florian has talked about adding users and groups as sort of first class objects in RPM at some point this is the right to go that's going to be years down the road and at the very least if we've implemented that in the middle it will take a long time for even if we added that today it would be probably a couple of years before it was on all of the systems even just in the fedora infrastructure and then so we can't rely on it for many releases and then when you start thinking about say something like sentos or rent out enterprise linux it wouldn't be available there for several years probably so we need some sort of way to get from where we're at right now which is crazy town to the nice place where RPM handles users and groups properly for us so I agree that the better way to do it would be to have RPM just to handle it but we need some sort of a transition so that's what these are for they're not my favorite thing either but what are you going to do I have a question about fire triggers so are you fine with packages doing pretty much anything in fire triggers or do you also want to restrict these things because I have some I must because I have at least to create to create the uses of fire triggers that are deployed one by one actually at fedora and one is so weird that I won't ever want to submit it to fedora and you can do because the fun thing is you can react on fire in storage by other packages no matter whether they're installed before or after your package okay so the question is I think to boil it down the question is what restrictions are we willing to put on what various file triggers can do because you can do some pretty strange things including trigger on other packages files I think the I don't think we have a clear idea what those restrictions are going to be yet but I think we've established as a policy that when you create a file trigger of any kind you're only allowed to do it for things that apply to your package and if you're doing it for anything else we should probably say no you can't do that yeah I think that's a good point and do we have any restrictions on what packages can ship file triggers or what can be done and it sounds like the answer is no we don't and I really think we should my goal would be that for the most part packages aren't doing any of them just because it's a lot of upfront work yeah and it's usually not necessary so to convert the handspec dictionary from the system handspec to the format expected by Kube Web Engine in the Chrome format and there's actually a tool that ships with Kube Web Engine that will do the conversion for you and I actually wrote a file trigger that we never even start a system handspec dictionary the file trigger will recognize it and automatically call the Kube Web Engine dictionary conversion tool and store the converted dictionary in a Kube Web Engine directory okay so the statement was there's a file trigger to convert one dictionary format into another that's used by KDE and it's technically triggering on something else's data I feel like that's probably something we would allow on an exception basis and probably prefer to have a rule against it without an exception I think that would almost certainly get an exception but I think you'd probably have to ask for it alright any other questions cool so I guess summing up let's kill any scriptlet that we can just for the fun of it not just for the fun of it we're not going to get rid of all of them so it turns out that when we asterisked this is a lie at the beginning that was not true because the profit in this is easier maintainability and faster deployments so there really is profit it's just not monetary I think anybody who's tried to write any sort of image build tool has eventually come to the realization that unless we deal with scriptlets it's never going to be fast or reliable or secure and we got to start somewhere so you all braved the maze to get here you're the best and brightest to help us do this so were there any other things we needed to talk about I think we covered it well thanks all for coming