 Hi, my name is Steven Smugin. I'm the current chairman of the Apple Steering Committee. It's a rotating task. I got it for today. Today, I'll be giving you a short talk on the State of the Union of Apple. I'll be going through these slides rather quickly because we'll be doing it in 20 minutes so that there can be a time frame between for the next talk afterwards. Just a quick couple of definitions we have. I'll commonly use terms in here in case anybody wants to know why there are short acronyms. Today, we'll be basically going over the state of what is going on with Apple in the last couple of years since the last time I did this talk and hopefully we'll transition into the next talk, which will be a workshop on what we want to do with Apple and related build systems in the next year. Hopefully a bit sooner. Okay, what is Apple? It's as it can be as it's a milk, a generalized meat grinder of packages. We take packages from Fedora and people will take certain packages from Fedora, rebuild them and stick them into a repository that people who have enterprise Linux, which would either be Centos, scientific Linux, or the granddaddy, Red Hat Enterprise Linux will then use. The packages are usually packages we do not have inside of REL already. We focus on REL server, so things there may be stuff in REL workstation and such, but if the package is in REL, not in REL server, then we will duplicate the package in Apple. It's we just reached our 10-year anniversary. We've been doing this for Red Hat Enterprise 4, 5, 6, and 7 various other little things over time. We've done pretty well. I just talked about the packages we generally want in this release are what we prefer to be stable or LTS packages. There are ones that are really short-lived. They don't tend, people put them into Apple and we usually, they end up being pulled out within six months or a year because the both the people who are using Apple and the maintainers are too disparate in what they want. Things like Seth and Gluster have tried to be in Apple at times and they usually do not last very long because it's just the lifetimes of those packages and what people are wanting the same. These are, I'll give points to whoever can name one of those people on that picture. Islamabad is the first name of the guy in a stove hat. He built bridges. He built stone steam engines. He built a vacuum-tuned type of a loop back in the 1830s. Didn't go very far, only about 20 feet, but people use Apple to build things, big massive things that all through infrastructure. Currently, Apple is a growing, is a large, large part of everything that Apple Fedora produces. This graph here shows all the different releases we have in Fedora over time. The blue colors are the Fedora releases. That's about 20% of all, when people check in every day, 20% of everybody is checking into the mirror managers, our Fedora people. 80% are systems running Apple somewhere. It's about 1.2 billion per day and growing. Architecture-wise, it is mostly x8664. There are, there's a growing number of PPC, but it's not very large. And I put 0.2%, it's that was just a rant of the last 10 days. x8632 is falling off. In the last year, it's gone from 7% to 4%. Growth-wise, this package here goes EL7 systems, then we will see EL6 boxes. So there's a big giant drop right there. And I get asked usually, so I put this in. That drop is the day that Centos moved EL5 to Vault. And every system that was automatically doing YUM update for EL5 stopped checking it. There's a, over time, people had to hand move to say checking the Vault versus go to the main site. You can see individual IPs that were showing up a lot beforehand going back and pulling stuff out, out of the Vault. But, and then once it started, once YUM started getting a good configuration that we've been checking with us. But, generally, the drop there was every Centos 5 box in the head Apple enabled and going away. The boxes up below, either were configured to use some other mirror or were railed boxes. Why are they using Apple to build big over-engineered bridges that are incredibly stable? This is Firth Bridge in Scotland. It is a massively over-engineered bridge. It survived multiple bombs during World War II, directly landing on it because even though it was built way before any of those blockbusters that were built, the people who engineered it wanted it to not fail on the first or second or third of any high winds, high tides, train running full speed off the tracks, etc. People who build enterprise packages with Apple want stuff that is stable and well built. It's very hard to pull out exactly what packages people are using. But, you can kind of get a general idea. The ones that get pulled in, they look at, do a YUM install off of one of the mirrors, usually end up either a branch package or a core package that these things require, or a branch off of it. The R packages are incredibly installed every day. When we had Python 2.6 in REL 5, it was the most number one package installed. When it went out into life, Apple 5 saw a large drop of people installing things versus just checking in. Configuration management, mostly it's system and tools, and then the infrastructure they need to deploy things underneath those system and tools. So, I'm standing up here to talk as the head of the steering committee and the people who are in steering committee, most of them, are sitting on second row. What have we done as a steering committee over the last couple years? This is a time-lapse photo, because I didn't want to try and do a live gif in the presentation, because it messes up on a lot of these But we've basically been breaking a lot of things that we did not break in the past. Some of this is us and some of this is our upstream, our upstream being Red Hat Enterprise Linux. Things that we did right, we got ARCH 64 and PPC 64 LE added to the architectures and they worked without a charm, without a problem. People, Peter Robinson and the people working with him built a ton of packages before we even had to, worked out all the problems before we had, and basically it was a turnkey drop-in. There was some stuff in mirror manager that needed to be dealt with, but it was much smaller than we were expecting. PPC LE 64 is in our small set is well-used over the 64. He had a successful end-of-life of EL-5. People will come into the mailing list and stuff saying that EL-5 is broken. I can't get this package. And it's usually because of the sento's problem. Once I get them aimed at fault versus aimed at stuff, they can install this package on us. It's our archive that we moved everything into archives. People still need packages daily. We have people installing EL-5 packages daily. The guy who isn't here, who's on the committee, Avidj, I think has a bot. Just every time somebody answers, he just starts replying back how to turn vault on or enable EPL-5. Because if you go to our website, we don't tell you how to get the initial package anymore. We've been dealing with massive changes in 7.4. Rail 7.4 has been very painful. There was a long split between us and sent us because sent us had to rebuild a whole bunch of stuff outside of stuff. But even inside of our things, we had packages that had been pulled into rail that had been in our stuff, and we had updated beforehand, not knowing that it had been pulled in. And packages have been dropped from rail that haven't been moved into Apple. And so various things that we're expecting things have broken. The repo closure has a list several dozen long. Our big change that we're having to deal with soon is that Ansible is moving to rail extras, so we will not be updating it internally. That was our fun toy. You will have to find some other way to update it quickly ourselves. Many other packages have been moved in and out, as I said, various fonts that had been in rail before are gone. And this presentation found most of them. I had to rebuild packages so that I didn't get to tie fonts. Things we haven't done that we've said that we would look at doing, and it's looking like we may never get done. There has been an long outstanding request for X8632 and ARM32 being built in Apple by using the ARM trees or the sentos trees. The differences in the build systems and our own how Koji works is looking like it's too much of a change that we can do inside of our own build systems, and we will have to find some other way of doing this. Another big problem that keeps coming up is that we remove packages when they're orphaned or dead, and people still use them. They don't care that the package is no longer seeing security updates. They don't care. It doesn't do whatever it says. They have build scripts that deploy several thousand systems with that package. Currently that we get them to get it out of Koji, but we need to find a way to get that better done, done better, gooder. It needs to be gooder, because gooder is not good enough. That's another request that Average seems to have a canned response for, of how to find stuff in Koji and get it. A lot of the Rail 5 stuff are packages that we remove beforehand that they want out of Koji. Where are we going to go next? This will actually be our next talk. The next talk is a workshop to go over the state of things that we would like to, well, I would like to as the Royal Weed, because I stood up here in front of this, and the other guys decided they would sit down. You've been very good. You've been holding back. In this next workshop, I'm looking at wanting to put in extra packages and sent us an epic tree. Archiving old packages regularly so that people can find old packages in a way that is usable for them. Using an optional tree structure for packages that are hard to install parallel wise. Currently what we do for certain packages like Python and such is we would try to install Python 2.7 and 2.4 on the same system using user site lib and then try to have it separated out. It is a lot of maintainer help. That made it so that no one wanted to update things and it made broke a lot of other parts and building stuff. This is not meant to be SCLs. SCLs are something completely different, and I'm not trying to replicate that. This is a single use of optepel to put certain packages in, so there wouldn't be like six different PHPs or whatever. Or if somebody wants to parallel install PHPs, that may be the case, but we're only going to really deal with one of these things. And only one will be active. It's not like there will be alternatives and all that kind of stuff unless somebody really puts in the work for that. And then we need to talk about all the stuff that's going on. The rest of this building is going to affect rail 8. I have no idea when rail 8 will come out. I have no idea what was going on with rail 8. I keep my ears closed anytime anyone mentions that. I don't want to know. Someday there will be a rail 8. It may be actually rail A for all I know. Let me just take the bottom bar off. But there will be a next generation rail. When that rail comes out, the stuff that's going on in Fedora for the last two years and probably how many years in the future will be in that. We need to work out how Apple is going to manage or deal with stuff in that kind of world. I don't think we're going to jump on any one bandwagon. We will try and see if we can jump on what bandwagons we can drop on. Whether it's modularity, containers, or some other thing. I expect we will be doing the same thing we've done for the last 10 years for most of the packages. Just because most of the systems we have are still talking about SunOS 4.4 from 1993 as the good old days. And that's how they like the world. And that's okay. I think the 4-button 1, 4C was better. That's fine. Okay, questions. I actually left time for Heckling. Hello. I do not in this talk, I have it as an email, send it off the list. There's three tickets I put into rail N. I did not find the packages that rail had dropped until I noticed it when I started doing the talk. And the comforter font that had been in there, there was one or two fonts that had been in, I was doing the repo closure to see what was broken. I went, there's fonts things are requiring. We never had those fonts in our stuff. And so they need to be in rail or two of them. I think they're probably in extras or something at one point. Not a really care about package. Okay, I do have access. I think you have access to speed data. The reason we keep access to the data a bit locked down is the data, some of the data we have has IPs and other information that people consider personal information. And while I'd like to make sure that when I give any data that it is been wiped to that, so that I don't cause the door or any other group side issues where somebody says, well, law is in there. I didn't say that you could use law. We are currently it is done. And I'm pretty sure that some of most of the stuff is clean. It is also tied in with the AW stats, which is clean. So I don't have, I haven't separated them out completely. Once I get those separated out, then that part will not be behind the curve or spiral. They will be put up however all the slides are done after this talk. I hadn't been asked where I was supposed to give them beforehand. So they're on my laptop. And I will make sure that something happens to them so they don't become a dead drive or something. They will be made available. A general talk right, which turned into a 20 page thesis that I'm writing on all this stuff, which is in much more long form, will be a blog with all these slides tied into it also. So there's more information on what this slide's meant. Yes, sir. There is no way to track specific path to model. All I can do is I can go through the downloads. Young does not give you any information about what package was being installed. Young only says, pull down the metadata, give me a list of mirrors that are close to me. And I will go to that mirror and pull down all the packages I needed. Sometimes what I found is I was doing this and testing it out. Sometimes if a system only needed one package for a whole stuff, you get the package they wanted. If they needed 20 different things, you will get a ton of packages being pulled in and all their dependencies and all those sort of stuff. And it's very hard to tell on a lot of these packages they need to depend on each other, what the request was. You pull that package, you find out you pull out all these other things. And if somebody has a partial install of those packages, it's... Now, if there's a large enough R data thing, and somebody who knows how to massage data, you probably can clean it out. Okay, I've got three minutes and the guy is going to beat me with a club. All right. Anyway, so I can't pull out... This data that I pulled here was a best guess. I went through six months of data. I looked at all the things I then sorted of what was asked for off of our own mirror boxes the most. I then went and found all of the internal IPs and all the build centers like the CentOS build system to pull in packages from Apple. And I disallowed those because I figured a build system is going to pull in everything and we can't... I can't... There's too much noise. This all came out from that. R, then I went and did some reverse stuff, which is the part that I... I... Where I'm feeling kind of clumsy because I don't know if I'm... Our stuff mostly came from educational institutes and like CERN, which I don't really know if you count that as an educational institute or a big fucking rocket on the earth. But NASA, CERN, things like that. Climate Bay and fail-to-ban and security stuff like that all over. Rackspace uses a hell of a lot of that. NGIX and Node.js. A lot of that. You come out of AWS and things. Not just everywhere. Configuration management everywhere. After you get to the first... There's like a 10 packages, maybe 10 packages that have like a high number of hits. Most of those packages are actually Node packages, not leaf packages, but limb packages. They are all required by other things. So I had to... Then the next set are leaf packages because a leaf package... Multiple leaf packages might pull the same limb package and that bumps the limb package up. Pulling all that out, I got these. If I went for the other stuff then lib something, lib something, lib something would be our most used packages. It's hard to get real data. I can get some off of sub-mirrors. Some people will give me some data off their mirrors and I can kind of do that. The problem is a lot of the mirrors also end up being... They only see half the transaction. They may see only some of the downloads and then for whatever reason. So you see this thing where somebody pulling a package and you're going, that package makes no sense or they pull in the same package 30 times because they didn't get the package all the way downloaded. This type of data is really noisy. So I really would like to have find somebody who knows how to obey that doesn't give you dinosaurs and that. This stuff, like I said, I can make available. It's this big CSV file that I'm going to redo as a large JSON or MongoDB or something because the big problem with my thing is it's an AUX script that I wrote to go through this three years ago for a... Somebody needs to do a talk in two hours and I need this data and it's been added on and added on and added on. It's like four AUX scripts so and a Python thing and some other stuff all kind of tied together. Yes, it's become very temporary. So like one of the problems is, is the way I broke it up, the PPC data is all PPC data. It doesn't re-differentiate to PPC 64 and I have to go through that with a separate script. So when I'm looking at the other data for the computers for all of Fedora, most of the PPC users are Apple Mac users on Fedora 12. They're pulling the PPC 32 down. They don't. They don't. They've never upgraded since then. Yeah. But the other problem is that this data is representative. It's only useful in that you can only use it for determining sort of what's going on. Most of those PPC users are mirroring this stuff. They're going to have a mirror locally. I'm not going to see it most of the systems like the ARM boxes or anything like that. They're going to have it in the satellite. They'll pull Apple into their satellite and all their PPC boxes. I mean the Lawrence Liverbott Miller systems I know of that are PPC. I don't see them. I see one box. I know there's 70,000 of them in the cluster or something like that. So I don't want people. I don't like to use these numbers too clearly because it can be where people go, well, you know, we shouldn't focus on that anymore. Although x86 32. Anyway, I'm done. Thank you for your time and questions. We will now go to our next talk.