 Just like that and it works. All right. Good morning. Thank you guys for coming back for day two of Ubu Khan We have a lot of great things planned today We have two wonderful talks We have a lunch break. We have un-conference planning sessions, and we have un-conferences so an un-conference course is when the people here get together to discuss topics of their own choosing so This has always been a gigantic success when it comes to Ubu Khan and basically After lunch we'll come back here and we'll have things a whiteboard and we'll have some sticky notes and some markers and basically you'll be able to work together to decide what topics you should talk about in the areas and are we in ballroom I and J for that? We are here and here So then conference sessions take place you come in here, and then there are actually doors as ballroom I and J might might be the opposite. We'll have signs, but they're they're back on other side of the stage, and so I Don't know what we're gonna talk about it. That's part of the fun So I gotta sneak preview last night at the party a number of people were coming up with ideas that they would pose for the Unconference But you do have this morning and lunch break to consider Topics of interest that you want to talk about with others and brainstorm about so We'll have that on conference planning session at 130 Yeah, if you've never been it's a really kind of neat thing So make sure to if you find a topic on the board that that you like be sure to attend if you Don't find a topic on the board you like, but there's an extra space write that down on sticky note and Stick it on the board. So be sure to check it out in the meantime before then we have before lunch We have two great talks. The first talk is is Michael Holm whose talk is sponsored by canonical and He's gonna talk about sort of how canonical relates to the community at large. I believe is what you said And then after a half-hour break, we're gonna have Sergio come up and talk about About snaps from source to snaps I think is the talk and That's that journey from you have source code and you want a snap That works on all the blue choose everywhere And that's gonna outline that so it's gonna be a fascinating look behind the scenes How that works or if you're developer, it's not behind the scenes because it's your job and you're doing it so There's a lot planned and That's it. Well Dive into Michael for now. Fantastic You should explain the raffle and then oh, you know, I hit so many people as they were coming in But you're right. We do have a few others. So let's let's recap most of you when you came in should have been given a raffle ticket and You may recall from yesterday that we had the good folks from next cloud who are raffling off one of their next-cloud boxes So after Sergio's talk before we break for lunch, we'll do our raffle and Choose the winner once we identify the winner then We'll send them over to the next cloud booth on the expo floor To pick up their next-cloud box That further do Michael Hall For those who don't know me my name is Michael Hall. I'm a community manager. I work at canonical It's my job to bridge the company and the community And so Richard asked me to do a talk about what canonical does for the community what our role is what our responsibilities are So you've probably seen something along the signs a bunch to sponsored by canonical or provided by canonical If you look at any of the scale sponsorships, they'll say something like that So it's not, you know a bunch to a canonical product. It is sponsored by or supported by So what does that mean? So to tell the story of canonical and a bunch to we've got to go back in time a little bit to 2004 Where mark shuttle worth started a company to make an open-source desktop for everybody But really to do the story justice we've got to go back a little bit farther than 2004 We've got to go all the way back into the 1990s This is not what he looked like in the 1990s, but I like my job. So I wasn't gonna stick an old photo up there So in the 1990s Mark was a Debian developer. This isn't something that a lot of people know But this was his start into open-source was in Debian and specifically he was the Apache HTTP server maintainer in Debian And he did that for many years In the mid 1990s, he started his first company. I think it was his first company anyway the important first important one Thought this was the mid 90s. The internet was taking off security was starting to become important and thought was one of the two big SSL certificate companies verisign was the other one So he ran this for a few years and it did really well Well enough that the other big company verisign bought it from him for Quite a bit of money. So this is what made mark Rich to begin with so now he's a millionaire He wants to do something important something world-changing now that he's got the ability to do that But before he does that, you know, he's a young newly rich guy So he decides that he's gonna go and he's gonna travel the world for a little bit So he travels all around the world 128 times in eight days as the second space tourist So he got that out of his system and he came back to earth and that gets us back to 2004 Where he now he's you know, he's gone. He's had his adventure He still wants to do something that's going to have a meaningful lasting impact And he wants to do it with open source and with Linux because that's where he got his start So he starts canonical and canonical will be What starts a bun to? So he's got a company. He's got an idea for a project What he doesn't have is anybody to work on it other than himself So he goes and he does the obvious thing. He gets on a boat and goes to Antarctica In the way that you do But he he still was looking for people and there's not great internet down there So he actually went and he printed out all of the Debian mailing list archives Onto a whole bunch of sheets of paper and he spent his trip to Antarctica Reading through the mailing list archives to find out who the important people in Debian were and when he got back He started sending invitations to them saying hey, how would you like to work for a company building a Linux desktop? They were interested, you know, that's what they love. That's what they've been doing So he got them all together in the UK To talk about what they wanted to do. What was what was their Linux distribution going to be and how is it going to be different? And the main takeaway from that Was they came up with the Ubuntu promise that Ubuntu is free. It always has been and it always will be and that was kind of a Big thing for a commercial company, you know the until then you made money on open source by Holding something back just for the people who are gonna pay for it You know you sign a subscription service or you buy a license for it or something But they decided they didn't want to have that they didn't want to have an enterprise version That's only for people who pay and an open version for people who don't pay They wanted to have one version. They wanted everybody to have the very best without having to pay for it So the Ubuntu promise right from the beginning was a guiding principle for canonical and it stuck with the company till today and that Really has an effect on everything that we've done As a company for the Ubuntu project Early in 2004 they needed a platform to build this open source operating system on so they developed launch pad. This was Like seven years or so before there was a github Sourceforge was around but this was the first of the new generation of project hosting sites And it was free to use if you had an open source project anybody could use the resources on launch pad They had their first in-person event in Matero in Spain This was the first actual Ubuntu event and right from the beginning It was open to anybody who wanted to show up there So it was you know a new project not many people knew about it Not many people who knew about it cared about it yet, but they had a small group. You can see if they filled a couple of tables So these were people that were paid by canonical to fly out there They were put up in a hotel by canonical. They were fed by canonical. So Right from the beginning, you know, we're spending money getting people together to work on this project that we're giving out for free We needed source control something to manage all of this we had subversion and CVS, but they weren't real great so There wasn't anything There wasn't a distributed version control system that worked really well at that point So in 2005 Canonical started developing Bazaar And this was a year or a month or so before the first get was announced and a month and a half before mercurial So it really was the first out the gate of the new version control systems And again, it was open source. It was made to be available to everybody It got adopted by the GNU project. I think Mozilla used it for a little while so again an investment that canonical was making for the wider community 2005 also saw the first of the Ubuntu flavors show up with Kubuntu and Flavors are an important part of the Ubuntu project. They are Not the product that canonical is making but they are something that canonical is supporting So every Ubuntu flavor in addition to working in the archives with the rest of Ubuntu They get free Build servers to build their packages Servers to build their release ISOs. They get hosting on our servers for their ISOs The canonical people who do testing of packages will do Testing specifically for the flavors and help them debug problems when they come around So every flavor that we add is a little bit more Resource that has to be committed to to maintain that so Kubuntu was the first all the way back in 2005 and they're still going strong Today with 12 years later The second Ubuntu sprint was in Australia Ubuntu down under and you can see there's a few more people there, but it's still a pretty small group We didn't even I searched for group photos and there were no group photos for like the first half of Ubuntu's existence So you're getting usually ogres Photos because he reliably took some So slightly bigger crowd and again most of them are being paid to be there by canonical They've flown out by canonical even if they're not working for canonical early on we would sponsor community people And we still do that. We've got a lot of community people here that get sent out to events by canonical So back in 2005. We were still, you know Paying flights and hotels and food for people in the community because getting people together to work on Ubuntu was important 2005 was also when the community council started This is one of the two main governing bodies in Ubuntu. The other one's the technical board They make all the technical decisions, but the community council is in charge of the community and the project around it and canonical really Follows the community council's lead so the canonical is not in charge of the Ubuntu project the community council is and Canonical is committed to that the people in canonical. They care about the community council you know if the community council has Concerns about something or wants to change Canonical is usually really receptive to that because it's important for the company to work well with the community 2005 was also when the first loco team started showing up This was a spontaneous thing that people you know who are using Ubuntu found other people using Ubuntu near them and just started having events and They started up on their own. They started popping up all over the world And so we started we wanted to support them. We would send them out Packs of CDs. We would send them swag for conferences You know all they had to do was say hey, I'm doing this event I would like some stuff to give away Here's my mailing address and we would Create swag and we would send it out to them and again. We've been doing that since 2005 2005 was when we got our second flavor with a job on to an educational specific flavor and just like Ubuntu They get all of these same resources All of the same human resources on each release So every time we add a flavor it costs canonical a little bit more to maintain that We started using IRC heavily and IRC is nice But if you have a conversation or if there's a conversation in IRC and you're not there for it You miss it unless you've got log bots and log bots require some server running them and they require a server to host the logs So can I cannot Canonical started doing that for all of the Ubuntu channels So any Ubuntu channel on free node can request a log bot and get it added there. It doesn't have to be Something specific to the development of Ubuntu. It could be for a flavor a sub project a community team Whatever we'll host that for you 2005 is also when this guy joined canonical the first community manager hired full-time to Help the community and grow the community and he would Lead it by himself for a while, but then a team was built up around him and at our height We had like six people focused solely on community that was what we were paid for that was our day-to-day job is helping the community They had a sprint in Montreal getting a little bit bigger here as you can see Somebody decided that Montreal in like February was a good idea. I don't know why So again, you know More flights more hotels more food to make that happen 2006 saw the start of the long-term support releases Before that every Ubuntu release was only supported for 18 months 2006 canonical said people need longer support than that 18 months is too short We need to give them a longer window to run a stable version of Ubuntu and we're going to commit to supporting a release every two years that has that extra support and That's not just the main Ubuntu flavor that includes, you know all of the other flavors too in the packages that they have Most of them will commit to an LTS cycle too We got our third flavor in 2006 was Ubuntu and if you were at Liz's talk yesterday, you saw the full history of this So again, same hosting resources and people resources that all of the other flavors were getting They had the this was the first Ubuntu developer summit that used that name that was in Paris They still weren't naming them after the release yet But again getting a little bit bigger more people that are being sent out there Canonicals still paying for most of them to fly out there and to stay out there We wanted people to take advantage of their Ubuntu knowledge Companies like seeing certifications that helped get you hired so Canonical worked with the Linux Professionals Institute to develop a certification program For Ubuntu so that people can take the knowledge that they've gained as part of the community and use that to further their professional career And that ran for like eight years or so We also developed a new init system because Sysbian it which, you know kept on working didn't work very effectively or efficiently so back before there was system D we started this and it was used in Ubuntu, but it was also used by several Fedora releases a red hat release centos and Oracle release And in Chrome OS for a while, and I think Chrome OS might still be using it. I'm not sure so again something Canonical saw a need for committed the engineering resources to and Put out there for everybody to benefit Then they had a UDS in Mountain View co-hosted with Google and you can see we're starting to become a little bit bigger now We're actually getting group photos, which made my slides easier And again most of these people Either directly as canonical employees or indirectly as Sponsored community people were flown out there and put up by by canonical We started a ship it program because you know We wanted more people to get a bun to the more people that got a bun to the more of an impact It would have and the easiest way to do that was to put a CD in their hands in 2006 Downloading an ISO was still slow for a lot of people burning it onto a CD was still slow for a lot of people So we started a program where you could literally just go to a web page and say I would like a CD And we would mail you a CD and a single CD is not that expensive to make and a single CD is not that expensive to mail out But we were mailing out tens of thousands of them And so it added up, but we kept sending them out for years and years and years To anybody who asked for them Just to spread the reach of a bun to and you'll notice a recurring trend in a lot of this is you know Wanting to spread the reach of a bun to where people are using it how people are using it Increase the impact of that So ship it did a really good job of that that was how I got my first a bun to install was from a ship at CD The next UDS was in Seville Spain again getting a little bit bigger more people flown out to more people put up in a hotel all to work on Making the next version of an operating system that we're giving away for free 2007 we saw the first of the desktop effects. This was Not an easy engineering task to do it actually took three release cycles of being planned to be default before it was Stable enough to actually do So a lot of engineering a lot of engineers in canonical were put to this task But having that that stupid spinning desktop did more to entice people to try desktop Linux Than any other feature to date that I'd seen you know people would dismiss it as some nerd thing Until you spun your desktop around they're like, how'd you do that? How do I get to do that? so, you know Time time and you know money that we put into that paid off for the wider Ubuntu project in terms of getting more people interested in it Had another UDS in Boston again. We're doing these twice a year flying people out putting them up cost between a thousand and two thousand dollars a person to get them out there and we're doing that twice a year so in 2007 we had maybe 100 200 people at most In 2008 we expanded ship it so not only we were we're gonna send you the Ubuntu that canonical is making But we'll send you kubuntu CDs also I don't Liz did we ever send out zubuntu CDs and chip it no, okay? Well, we did we did kubuntu at least an edge of Ubuntu So again stuff that's not the product that canonical is actually making But we're still going to produce CDs and send out CDs for their users 2008 UDS in trepid the first named after releases. This was in I don't know. I think this was in Prague and again, you can see we're growing a little bit more still where Probably a good five six times with that first that first sprint was and again flights hotels putting people up in 2008 netbooks came on the scene and Netbooks looked like they were going to be the big new form factor that everybody was interested in and The GNOME desktop did not work well on a seven inch screen. I had a seven inch screen netbook and it was terrible But canonical wanted a bunch to to compete there we wanted Users on these new form factors to be able to use a bunch to so for the first time we developed Our own interface instead of just repackaging what was there and that was the Ubuntu netbook remix And that really set us down a new path We saw before this we just packaged what was already out there People would you know produce something for existing hardware, but nobody was anticipating future trends and trying to Position Linux and open source to take advantage of those we were we were reactionary Instead of being proactive So this was the start of canonical investing in future product lines and future form factors for a bunch to and a bunch of users Again, and we had another UDS bigger still more people More expensive to do but that was how we built a bunch to so we kept on doing that Continuing on with this new direction of canonical trying to anticipate trends and Steer its own path We introduced a new set of notification systems notify OSD to try and simplify the notification experience clean it up make it smooth and slick and a Lot of people didn't like this a lot of people didn't like canonical, you know steering its own direction But we wanted to be able to Take advantage of new trends as we saw them instead of you know always playing catch up to other people So off of the netbook remix We started taking a more active role in the design and direction of a bunch to and where it was going Another UDS now we're big enough to fill a small auditorium. So a lot more people still mostly Paid for by canonical so UDSes are starting to get pretty expensive now in 2009 we Because of the Ubuntu promise because we weren't going to sell Ubuntu and we weren't going to restrict access to Ubuntu or the Ubuntu archives in exchange for a Subscription we were always looking for other ways to make money That didn't limit what people got access to in terms of our software So in 2009 cloud storage was big There was a lot of money to be made in cloud storage And unfortunately as we found out there was even more money to be lost in cloud storage because it was kind of a race to the bottom But users wanted cloud storage and we wanted desktop Linux users to have cloud storage And so we rolled out our own to support that and we gave five gigabytes of storage free to everybody who Ran Ubuntu and all you had to do to run Ubuntu was to go download a free copy of it off the internet Continuing our design drive we really Worked the way indicators were on the desktop if some of you remember the old GNOME status icons and how they would move around and break Every week or so So this was a big change. We built it off of some existing protocols from other projects And got some adoption The the library and protocol behind this is used by KDE now also But again still driving the design experience to take advantage of new things and as part of that we wanted to make it a platform for application developers to you know the third-party apps were having a really hard time in Ubuntu They weren't very discoverable People use the defaults or the well-known stuff and they would go to snap they can get that There wasn't a really great way to Browse for apps. So we created the software center to try and Fix that and it used the same packages in the archives But it would give ratings and reviews and it would give screenshots and let people leave comments To try and make it a better experience for app developers and for users who didn't know you know what the best apps were So again put more engineering work into that to try and make Ubuntu better for everybody We also started the 100 paper cuts to Put that last little bit of polish Ubuntu has always been about putting polish on existing things But there was still a lot of tiny little things that were not Blocker bugs they weren't important enough bugs to really warrant stopping other work to get them done But they add up and so we started the program to identify all of these little things and start Committing to fixing them and putting you know somebody's time and energy into getting those fixed In 2009 we also adopted app armor. This was before Us it was a novella and Sousa project We started using it in Ubuntu and as we used it more and more and got more involved in it The other developers were moving on to other things. So novella still owns the trademark of it But we've now become the primary developers and maintainers for app armor Another UDS bigger still now in a slightly bigger auditorium And again every one of these canonicals paying for the people they're paying for the rooms at the hotel where the conference happens at Lunches were always provided. There was always some kind of party at it and the bigger it got the more expensive it got Again trying to anticipate new trends in 2009 The cloud was the new buzzword That nobody quite knew if it was going to be a big thing or not But canonical thought that it was and we wanted Ubuntu to be there. We wanted it to Be the preferred cloud operating system When the cloud took off and we started off by supporting eucalyptus which at the time was the big open source Cloud platform that had a lot of potential So early on before most of the industry even Was bought into the idea of the cloud being a thing. We were Working to position Ubuntu to to take advantage of that Another UDS now we're filling a rather large auditorium Again, a lot more people a lot more cost to do that and we're still doing that twice a year 2010 we probably had 500 or so people attending a UDS In 2010 we worked with a group of companies to start focusing on arm support In the previous UDS here in UDSM was actually where Lenaro got its start before it even had a name It was formed there and the Lenaro connect their Their developer conference was co-hosted with the Ubuntu developer summit for several years We would share the venue and the location For that and we also shared a lot of attendees a lot of people who are there for a bunch of developer summit We're all so part of Lenaro and going to that Which actually required us to change some of our software We had a summit scheduler that we'd used for UDS for ages and ages And it did a really good job of checking for conflicts for a single event But it had to support two events happening at the same time with the same people now And so we had to spend a few weeks updating that to support co-hosting Lenaro with UDS I already talked about comp is and desktop effects But in 2010 comp is had a major rewrite to version 0.9 And we actually hired the main comp is developer at that point Sam spills something Sam So he worked for canonical for several years to keep to keep comp is going That's also when we started getting interested in open stack Again, we invested in eucalyptus heavily, but by 2010 open stack Had the momentum behind it and it looked like it was going to be the one to really take off And and it was you know, it's the huge thing now. Everybody's running open stack or supporting open stack So we got involved in that early on To take advantage of that and make sure that a bunch who worked well with it open stack ran It used a lot of launch pad services for a while for planning Their events. I think they might have used our summit scheduler for a couple of open stack summits And I think they still use launch pad for some things. Don't they Liz? So, you know, we're still supporting the open stack project and open to on there And it's done really well for us, you know a bunch who's now the biggest Guest platform on both the public and the private cloud because of these early investments that canonical made To make sure that a bunch who was positioned well to take advantage of that We also introduced a new archive for third-party applications Like I said earlier getting getting apps into a bunch who was not an easy task It almost always required Well, it always required creating a debian package and it almost always required getting that debian package into debian first and Then waiting for it to sink over to a bunch who in time for a bunch who's freeze So could sometimes take, you know six months to a year to get your application into a bunch and the Ubuntu Software Center So to try and ease that we opened up a new archive with different archive policies that were more restrictive but let you update things a little bit quicker and We formed a community board around that and we had a canonical team behind it also To review apps and get them in there And it didn't work out quite as well as we thought and I'll get back to that in a minute But again, we were investing on making a bunch to a platform For users and for developers and making it more attractive for them Another UDS this time in Orlando now we're too big to even take a group picture inside an auditorium So we all had to gather up outside in the Florida Sun to get our picture We were probably 600 people now at a UDS and again More than half. I don't know if it was most but at least more than half. We're being paid for by canonical to be there Continuing in our design push We wanted to have a font that represented Ubuntu a nice open source versatile font that supported a bunch of different locales That Ubuntu supported so canonical paid a design firm to come up with a new font just for Ubuntu And that new font was released under an open license so that anybody can use it Of course 2011 brought us unity This was kind of the the culmination of all of the design driven work that we had been doing for a few years now Back in you know when the NEPA remix came out Mark saw a need to support screen sizes of different size and different form factors in different use cases and We wanted to do that in a way that didn't fragment The ecosystem so we wanted to have one interface and one platform that would work on all of those And that's what unity was and that's why it got the name unity because it was going to unify all of these different hardware platforms So that landed as the default in 2011 as a result of a whole lot of work by a whole lot of people We finally got another flavor in 2011 with Lubuntu a nice Very lightweight desktop interface that is still going strong today Again, just like all of the others they got hosting resources and build resources and testing resources That were paid for by canonical We had a UDS in Budapest and you can see we've we've grown really really big now UDSes are a Million plus dollars of pop at this point just to get people together to develop on Ubuntu And we're still doing them twice a year We started a certification program because people wanted to have hardware that ran well with Ubuntu and they wanted to know What that hardware was? So we set up programs to test hardware to identify hardware problems and get them fixed We put together a hard firmware test suite so that anybody can run These validation programs against their hardware system 76. I know is still using that 2011 we were back in Orlando again with an even bigger crowd definitely at least 600 people now Being sent out to what to every UDS and Again trying to anticipate Yeah, Richard so this was This was in October and we actually had Halloween in the middle of UDS So somebody showed up There was a costume party one night and somebody showed up in that and they came back out for the group pic I don't even know who's in that. Does anyone know who was in the suit? I'll remember from that was a Robbie Williams in the the Hulk costume He's in there somewhere Yeah, John. It was the hot dog that year if anybody's seen that picture All right, so Continuing trying to anticipate new form factors for Ubuntu to take advantage of 2012 we launched the Ubuntu TV project to try and get on smart TVs Unfortunately, that didn't take off. We put a lot of engineering work and time into it And that that's another thing that canonical it plays an important role in is that you know We are willing to invest in things that won't work out We'll take the loss on things like the Ubuntu TV For the opportunity that it provides, you know, we took a risk on cloud cloud paid off really well TV Not so much, but it did kind of set us down the track of thinking outside of the the PC form factor 2012 in Oakland again much much bigger crowd Another really expensive event This everyone knows California is not a cheap place 2012 we introduced the Amazon lens Like Ubuntu one this was looking for a way to Make money off of a product we're giving away for free and The thought was that all of our users most of our users are Amazon customers, too They go to Amazon to look for stuff to buy Amazon's got a program where they'll kick back a little bit if you refer somebody to them So it seemed like a win-win to us Clearly a lot of people thought otherwise and we heard about this for a long time But you know, it wasn't just the Amazon search scope that we developed These were the scopes that we developed for the smart dash and you don't hear about most of them because most of them have no Financial transaction for canonical, but they were very useful and you can search your bookmarks from the dash You could search for github projects on the dash or man pages or your Google Docs The there was an effort to make a hundred different scopes like this I think we got 50 or 60 of them eventually done But most of those were paid for or were done by somebody being paid by canonical to work on those We had our last UDS in Copenhagen. We had a little over 700 people at this event I don't know what it cost, but it had to have been at least, you know, like two million dollars to do at this point UDS has kept growing and becoming more and more expensive and The development pace of a bun two was getting faster and faster too So we found that we were already doing a lot more discussions mid cycle outside of UDS Just out of necessity So the decision was made After the UDS are to stop doing the in-person UDS events We started a nap showdown to try and take advantage of this new third-party app Propository that we had introduced a couple years earlier so the community team we Got some prizes together and we led a months-long app developer campaign We improved all of our documentation. We provide tools like quickly to make it easier to make apps We had the application review board to review them and tools to check packaging and we got I Think we got like 200 submissions In the end of desktop applications new desktop applications that didn't exist before out of this contest And unfortunately we hit a roadblock after the apps were written People really struggled to get them packaged and to get them packaged and into the archive even with You know the more lenient extras archive. We only got about a dozen of these 200 or so apps that were written Actually into the store where people could download them 2012 saw the start of LibreOffice and the document foundation and from the very beginning we've had somebody paid for a canonical employee on the Membership board for the document foundation and he's also one of the major contributors to LibreOffice itself and was instrumental in Fixing their their build process and making that faster and Smoother for them and he still works for us and he still works on LibreOffice This answer and 13 saw the release of the Ubuntu phone Like the the TV before it we saw this as a new opportunity for Ubuntu to get more users and more contributors into the community And it did we got quite a new number of people that came to Ubuntu As developers of the phone or as developers of apps on the phone Because we'd stopped doing the in-person UDS we decided to take it all online So we spent some time reworking the summit scheduler that we used for all the in-person events to support a massive online event so we use Google Hangouts and IRC and we run multiple sessions at a time over multiple days all online now and Initially in 2013 We were doing it even more frequently We were doing it every three months Instead of every six months. I think actually we tried to do it every month right from the start and That was a lot to ask for people to put together Things to talk about and plan out that that frequently instead of actually, you know doing the work But we still do these we're back to every six months now But we get the community together and canonical together and people from you know outside of our immediate community But our upstreams or our downstreams that are taking advantage of Ubuntu Together to talk about what they're doing and what they want to do with the future of the project To support our phone work We created a new SDK and UI toolkit for applications and a whole bunch of different services to make them work And we learned a lot about making apps by trying to support it in a phone because the way Android and iPhone were doing it was so different from the way we had tried to approach app development on the desktop before so we were learning a lot from them as we went and what we learned from them on Developing the phone. We also brought back to the desktop Which is how we got to click packages on the phone and now snap packages on the desktop So all the lessons that we learned About what it takes to get apps on a platform We've put into this to get apps to desktop users and Cloud users and IOT users of Ubuntu We got a new flavor in 2013 with the Ubuntu GNOME Again like all the others that we were still supporting they got resources and time from canonical To make that a well-supported platform We also saw the introduction of Ubuntu Chillin Which was a collaboration between canonical and a university in China to make a Chinese user friendly version of Ubuntu with apps that worked well over there and services that worked over there and in Chinese instead of Western languages So two new flavors they both get the same kind of support as all the others were getting We introduced a download donations page on the main Ubuntu.com Where when you go to download using the big links the big buttons that say you know download Ubuntu now You'll be taken to a forum with some sliders asking if you want to donate a little bit To canonical and to Ubuntu and one of the sliders is specifically to give money to the Ubuntu community and The community team worked with the people in canonical who are working on the donations page to make sure that we could Get that money to actually use directly on the community So any money that you donate on this slider goes into a special fund specifically for Used by the community and that comes to our team and then any Ubuntu member Who has something that they want to do for Ubuntu that costs money or requires a device or travel they can come to us and Ask to use this fund to do that. So we've used this to send people to conferences all over the world We've used it to send people specific hardware that they need to test a driver or fix something We use it to sponsor events We sponsored like debconf we sponsor the ubucons. We were using it to sponsor this ubucon if anybody was in The party last night or enjoyed the coffee today or yesterday that was coming from this fund and in the past Three years using this we've been able to give that Over a hundred and fifty thousand dollars directly to people in the community who are doing good for the community And all of this You know, we put out quarterly reports or at least we're supposed to on behind on that again But if you go to a community that a bunch of comm you can see most of the last three and a half years of Reports to see where all that money went and it's really done a lot of good for the community We even though we stopped doing the in-person UDS where we got, you know, 700 people together We were still doing team sprints as a company. We had always done them even when we had UDS But to try and keep that community involvement in what we were doing We started bringing community people to these work events instead of a UDS So if there was community people working with a team and canonical that was sprinting we were able to invite them to join us So that started with the app sprint and the phone sprint in Malta where we were able to get in a bunch of the app developers And people working on the platform itself To come join us there and to do some work and they did some really good work a lot of the Apps the default apps on the Ubuntu phone came from these guys We completely redid. Yeah, Richard 10 minutes, okay I'm almost there. We're 2014. We're good so we We completely redid our developer portal to try and support this new community that was growing up around the phone And application development and that turned out to be a much bigger undertaking than we thought it took the better part of a year to get that built and deployed and Get all the content created for that We also ran a Program to reward the people who are developing these apps because most of them they were developing and publishing apps For a phone that existed in theory, but not in practice yet so We asked Canonical to Make 200 shirts so not a very big run But a small run of 200 shirts that we could give away to the first 200 people who were helping us improve our platform We had another sprint again phone and app related and Invited some of the same people and some new people in Washington, DC again Canonical pays for the flights and the hotel and the food for everybody who comes to these events 2015 we got yet another flavor with a Bantu mate Increasing that the flavor family and again getting all of the same investment from Canonical that the others were getting We also stepped up our investment in Ubu cons Ubu cons grew from smaller You know one night events as part of other conferences to all day or multi-day events And as they grew bigger they had more of a financial requirement on that So Canonical stepped in to help make that possible And we've had several all over the world We've had them multiple years in South America in Europe in several locations here in the US also For for many years we ran global jams every six months for each release where we encourage local teams to get together And do work on a bunch to either testing or translating or future development And to support the teams that were doing that We put together the global jam packs, which was a small smaller than our conference pack I was just enough swag to give something to the people who showed up for that event And did something to help a bunch who out And we gave this to any team whether they were a Verified approved local team or not as long as they were running an event and they can show us that they were Going to have an event we would send them a box full of swag to give to the people that were there We also bought a meet-up pro account We'd had the local team portal for a long time that was developed by Collaboration between Canonical and community people But people like meetup.com and there's a lot of people on meetup.com that we weren't reaching But to use it for a local team you had to pay a monthly or yearly subscription for that So in 2015 we bought a pro account which gives us a bunch of different chapter accounts So now we can give any local team that wants to use meetup a free chapter account that they can use for their team So they don't have to pay for it anymore I know the California team has got one Florida team has one several in Europe are using it now So we're hoping to ramp that up too. It hasn't been used quite as much as we were hoping yet There's a little bit of resistance to it But it's it's another thing that Canonical offers to the community And then late last year we got yet another flavor with a bunch of budgie Is our latest addition to the flavor family again getting all of the Support and resources that all of the others were getting So that brings us up to Oh one more last in 2016 We really started paying more to make ubucons happen last year last year Was the first ubucon summit here in california That cost Quite a bit of money because you know, we're renting these rooms from scale And the convention center cost scale a lot of money and in the past they had given us the rooms for free But as we got bigger and we started using more of the rooms You know, it wasn't right for us to take them and not help scale financially. So We're paying for the rooms. We paid for ubucon europe's rooms We paid for some of ubucon la's They got the rooms for free, but they they needed food for or money for like coffee and stuff So in 2016 we really started spending more and more making ubucons bigger and better And as we started doing work sprints around snappy, we were inviting people to those two. So in heidelberg last year we brought in app developers like vlc Developer to come and work with us on making snaps work for them As a developer and their users We also support the ub ports project I've been working with marius the lead of that project for better part of a year now Helping him with logistics helping him Get information out of canonical or get things fixed in the Foundational parts of the ubuntu phone platform For his ports and introducing him to People like fairphone who he's just worked with to get a port out To the fairphone too, which we'll have at our booth later if you want to combine and see that and now in 2017 Part of the work that we've put into the hardware certification has been developing a bunch of tools to help us test hardware And our software on that hardware And one of them is stress ng which really As the name implies it stresses the hardware And that revealed a bunch of bugs that we found in the linux kernel when we were running that So the developer behind that started submitting All of these fixes to the linux kernel as stress ng identified them So in the linux 4.9 kernel he broke into the top 20 Contributors to the kernel there. So we've been making a lot more contribution to the linux kernel lately So that gets us up to today These are just a few of the community facing websites that are hosted on canonical infrastructure right now Obviously things like www.ubuntu.com, but also important things for the community like the planet and the forums The wiki the help site The local team portal for all of our local teams And even things like people.ubuntu.com where a bunch of members can host files for free So There's you know over a hundred different community websites That are currently running and that's not even counting stuff that's you know been running the past and is no longer around So That I hope gives you a better idea of what canonical's involvement and contribution to the wider ubuntu project is I don't know. Do we have any time for for questions? I would like to take time and I also want to say you know I was talking with michael about the idea for this talk This was enormously well researched It was beyond my expectations and it really did cover the full scope of how The company and the community partnered together in the and what canonical brings to that support So so much work went into this would you guys mind another round of applause for all that good effort? It really made me feel like I was there it felt like you were there Well, I I learned a tremendous idea for part of it I had to go research a whole lot of this because it was all before my time Well, it was really well researched. I appreciate the effort Question How large a check has mark shuttle worth written to support canonical and indirectly ubuntu? I don't know exactly But it's at least in the tens of millions Of his personal money that he's put into this Uh, why does the 100 paper cuts program come to an end? Uh, it ran for a year or two years I think it just kind of tapered off as the the more egregious paper cuts got fixed Yeah, the question. Um, I know that edge ubuntu has kind of gone the way of the dodo bird Is there any like a chance that maybe canonical would maybe put together a meta package that could just be joined to an existing flavor Of ubuntu so we can get the same functionality I think the edge ubuntu meta packages are still around so the ones that contain all of their like app selection Yeah, I think they are still in the archive. Hey, thanks And here's oh, there we go Hi, I've been coming to these things for I don't know how many years now and I know this has probably been answered and also asked a zillion times, but What the heck is ubuntu? um, is it is it A flavor of linux. Is it some user interface on some phone that we might see any uh decade now or Or how do you guys see it and what the heck is it? So the easiest way to describe it is that ubuntu is an operating system It is like windows or os x It uses the linux kernel and a lot of other open source software to build a full operating system But it is it is what will power your computing device Whether it's pc or a server or a tablet or a phone I hope that answers it What is happening with the phone I asked mark when he was here last year and I think we're about That close, but I would like to have one device where I carried around I plug it into my desktop when I go home and I mean into a into a docking station And everything is there So we we had a couple of phone partners who put out hardware with a bunch of phone pre-installed and then we had a few Um devices that we supported as development targets um When we put out the first commercial devices in early 2015 we kind of froze the base of the ubuntu phone On that version of ubuntu. So it was an ubuntu 1504 base That we put the the phone packages on top of and we ran it like that for You know over a year And it got to the point where it was getting harder and harder to keep working on top of an older base But the work required to move it to a modern ubuntu release Was pretty big Because we'd introduced a lot new a lot of new things we'd introduced system d into the new Ubuntu we moved on to snaps as the packaging format for it So the work right now is going into getting Device images built on this new platform and that's requiring quite a bit of engineering work To make that happen So devices are kind of stuck right now We don't want to put out new devices or get hardware companies to put out new devices on this old platform Because we don't want to you know support it for another five or so years But we also don't have the new version of it ready yet for them to use so Devices are kind of stuck until we get that new 1604 and snap based versions of the device images working Talk about a little bit on the timeline when like juju, moss lxc lxd Yeah, so this was kind of desktop focused because that's the part that I spent my most most of my time And so it was what I was familiar with But we did do a lot on the cloud side with uh, he was asking about juju and lxc In order to Really take advantage of the cloud. We needed a way for people to Use it to scale the way the cloud wanted people to use it to scale and doing that manually was not easy So we did develop tools like juju And mass to make that easier and you know put those out to the community and to The upstreams to to take advantage of that and I think that's part of what has made a bunch too So popular on the cloud is that it is easy to take advantage of what the cloud gives you over a traditional server setup And lxc2, which is a lightweight container format for linux that makes it easy to Add density to an existing server and now lexd Is it lexd or lxd? lexd So lexd is the the hypervisor for for lxc containers and that makes it super easy To spin up a container and manage them and we're taking advantage of that quite a bit now with snaps and snapcraft That use lexd containers to to cleanly build your snap packages I think we have time for one last question a few years back Ubuntu had a relationship with Dell computers providing Ubuntu installed on Dell computers that were sent And sold what happened to that relationship and are there other uh ubuntu relationships with Say laptop and computer vendors that relationship is still there Last year here at ubukan summit. We had barton george from del give a keynote about The work that he was doing. It was originally the xps 13 laptop that they developed And that proved so successful for them that they've expanded it now to a whole line of Laptops and desktops that they offer with a bunch who pre-installed Um We have relationships with hp who's been selling some laptops and desktops In other parts of the world And lenovo's doing some of those too And here in the us we we work closely with like system 76 who is here and give the keynote for us this morning To to you know support their line of hardware also All right, it's that all our time. All right. Thank you everybody now Before we break I've been handing out raffle tickets to most people there are a handful of people who have not yet had raffle tickets so um Take a break, but drop by me if I don't catch you first and i'll make sure you have a raffle ticket because we are raffling off a Next cloud box at the end of our next talk Yeah, richard. I think my um 25 friends walked out the raffle tickets. I'll so give you some and i'll i'll get those 25 friends of tickets um Very good Thank you everyone Welcome back. Looks like we're missing mic 7 audio here. There it is. Thank you Um Also quick double check. Are we getting audio out to the live stream? Yesterday we had a lot of sessions with no outgoing audio But if you have a chance to look into that, uh welcome back from the break We have another great talk coming up which we'll introduce in a moment But first i want to recap we have had a lot of scheduling changes So if you're basing your time on the printed schedule My apologies, but the printed schedule is no more Um, so we have had to do a lot of last minute changes What's on the board outside is very current The schedule as we have it at ubukan.org is very current the print schedule is not current There have been mixed thoughts about the app But i'm just told the app updated moments ago So it seems like it finally got in sync with what's on the board But when in doubt ask me or check the board because we do think the board is current And and that being said uh all the uh some of the talks switched around but the actual times did not so um as soon as this talk is over we have uh lunch and a break and unconference uh sessions Scheduling and then the unconference sessions and that is all accurate Yeah, and we get kind of back on schedule at that point. So um, we'll have a talk here We'll take a break for lunch. We come back at 1 30. We have 30 minutes of unconference sessions Just for the benefit of the use who uh who just came in Um, the the latter part of the day we'll have space for four unconference sessions So if you have something that you would like to talk about or work out in a work group format with your friends Or make new friends with it Then we'll have a place during that half hour sign-up session to propose an idea Once we get four of them together with good interest and We have another break once the expo floor opens. So to recap we have lunch from noon to one Noon 12 30. Thank you 12 30 to 130 back here at 130 back here at 130 for unconference Then uh two to three The expo floor is open and we know the expo floor opens and everyone wants to be there So you get an hour in the expo floor then we come back here and the unconference sessions will be Behind these doors on this side and behind these doors on this side And we'll have them signed for which sessions are where uh, but the unconference last year worked out to be really good So I hope you guys um Have been thinking about unconference things you want to propose and you will have a good chance to see what's available to attend So with that out of the way, would you like to introduce Sergio? So Sergio has been working really really hard on Many things but lately the big thing is snaps. You've heard it all weekend long Snaps snap you've been to buntucor which runs snaps Um, so what is a snap and how do we get snaps? Uh, it's actually really really easy and Sergio is uh talk from source. The snaps is going to show you how Sergio uh Okay, thank you Can I get the swap for the screen please? Can we swap that? Thank you okay, so As uh, nathan introduced me. Uh, my name is Sergio Svesov. I'm the technical leader for uh, snapcraft Technical leader is just a fancy name for a team of two people um And or you can find me at at serges since I I sell them tweet, but sometimes uh, I tweet a lot about snapcraft and snaps these days I really started at since uh, michael gave a talk about the history of canonical I started at canonical to work on buntu for android, uh, which was killed Three months well not killed put in stand by three months after I joined Where I started working on the initial team that worked on the phone That led me to In 2013 start the snapping project the snappy project And after a year of working on the core for snappy, uh, I transitioned to uh working on snapcraft exclusively the um So in uh, and so I've been working on snapcraft ever since uh since more day tonight night includes nights a lot of nights as well And uh, if I feel confused or tired, it's because I've been I'm a bit jet lagged I almost left three hours last night and I've been traveling for 48 hours So it's a bit of a trip here. Uh, this is my second time here last year I talked about ubuntu core which was very snappy related this year. I'm just going to focus on snapcraft Um, and uh, how to get snaps in place, but um I still uh, still so everyone's on the same page Um, I'm going to get up to speed a bit what snaps are Um, how they work and how they uh, they they are viewed inside the system Um, so the the first 10 to 15 minutes are going to be a very uh sort of introductory and then the The rest of it is going to be a bit more developer focused Um, so let's get let's get on uh, so let's start with uh, what what is a snap? Um, and it's it's People like it's it's not much, right? It's just a package Um, but as uh, it's just a package among many other types of package, but it has a couple nice attributes. Um One one of them is it's an immutable one. So this means that it's once it's installed the system. It can't change Um, this of course means that you install it. You can't hack it up to do something specific that you want. You Probably can as a developer Uh, recreate a snap as you want it based out of another one, but um a snap as it is is created and That's it. Uh, this is actually very easy to to support. Um, because um, your users will always Basically have the same snap installed everywhere. Uh, so it won't be modified. Um It is a confined package in itself. It means that they basically can't talk to anything else. Uh, And can't just randomly, uh, poke at things that It shouldn't be able to. Um, which is uh, in my opinion a good attribute. Um, and with that it has segregated data stores So that means that you can't just randomly, uh, write to Locations data, whatever the your snap does it needs to as specific locations it can talk to. Um, Sorry, I can write to or read from. Um, and um, basically, uh, well It's used as a data store. Um, it it is integrated into the system through interfaces interfaces are like, um They're very well known interfaces. Um interfaces are part of the snappy system itself. So once someone, uh, Let's set a snap to write to snap. Uh, they will have, um interface name that is well known. Um, and Common across all systems that are snappy enabled. So it is a common and creative view. So it's Easy for for someone's just like, oh, this does this. I know what it is instead of a random. Um, In the case of security random app armor set comp hooks that people need to look at and parse themselves. Um, So that helps a lot. It is also integrated with hooks. So hooks are something that snappy has if you think of uh, maintainer scripts in devian, it's It's not that The other things that I run, um in in a confined manner as well to basically set up a snap from initial install Um, then, uh, this is an attribute of the snappy system itself. It is always up to date. Um, so Refreshes happen all the time. So you're always up to date. You're sort of forced to be up to date There are ways for that not to happen, but in the general common case, you're always up to date. Um, And you are transactionally updated, uh, which means that if A version of a snap a revision of you get a new revision of a snap and you update that Um, the Uh, the the whole snap is is moved to that revision and the the data stores You can you can roll back to a previous version of a snap It can happen automatically if the update fails or it can you can roll back yourself manually if the update is failing for you And the good thing about that is that the data store is also, uh, Revisioned so versioned so you you get that rollback as well And uh, all these things actually make it easy to manage in my personal opinion Um, so that's basically what a snap is. Uh So there are many types of snap, um We have uh core which is basically the The base system, uh every snap lives on top of Every app snap lives on top of the core, uh snap the core snap is basically the os Today we only have, uh Series 16, which is basically 1604, uh, this brings the common commonality to to the system and All the the snaps so they can basically work everywhere Um, then we have the the kernel and gadget snap, which is basically enabling the kernel snap type is basically the kernel fair more, um And anything that Enables the system in that sense and um in the case Of, uh, maybe a window personal, uh, it would also have, I don't know if anyone knows how a window touch works There's a small lexie container that has A small android in there that basically Works as a how Well, that would be inside the kernel snap and in this this, uh In this architecture and the gadget a snap in pc architecture is not really that important because Bootloaders and everything are very common. So there's not much enablement to do there, but on on arm and and friends, uh Bootloaders are always different. The Raspberry Pi is one way to boot the the Um view on black has another way to boot. Uh, everyone has their own, um, um bootloader, um And they also have different partitioning mechanisms, uh partitioning schemas that you have to follow So like this has to be on this partition to work for this to work. This is being this partition for this to work And so the gadget snap takes care of all of that And finally we have the app snap, which is an implicit, uh type It's the default type and it's basically what makes, uh, a system useful like With the three there, no apps basically have a useful system. Um So work in these live, uh, so on there's a boondocore, which is pure snap Based system and there you can have the app, uh snap All the apps, of course all the snaps that make stuff useful kernel gadget and core And kernel gadget and core by themselves do nothing, of course So if you build in the boondocore image, of course, you want to put apps on top of it And then on classic and what we call classic is basically every distro around that has stand, uh, the common, uh, packaging systems you can see today Like boondoo, uh, and all its flavors, boondocerver, um Fedora, suze, all those are like classic systems in our view. Um The wr open wrt is also a classic system. Um And on there you have the core, of course, because you need it because it's the commonality for everything and the, uh Well, you can have all the apps on top Um, the apps, uh, except in what some specific scenario I'm going to come back to later Uh, same app can work on a boondocore or classic. That's uh indistinguishable. Um As long as all the interfaces are provided by the underlying operating system Uh, so what does the app view? So the the the app The app can only see core as its file system So this is very straightforward on the boondocore because the core file system is what you see when you log in So you just navigate, uh, what's there on the classic system? You have your file system there But when you launch your app, it actually just sees core and, um, it has its segregated data stores And these environment variables are exported, uh snap, uh, uses data and snap data, um, which where you can write to and there's And these are revisioned and then there's a snap common Which I didn't draw here for just make things easier and that is not revision Um There is there's one exception to this rule. Let's get again, but I'll get back to that later Um, so that's the brief introduction to to snapping itself Just to get everyone on the same page and I'm going to um start into Getting into snapcraft So so what is snapcraft? It's it's just a tool to build snaps. That's all it does. It's nothing more than that Among many tools that build stuff. This is another one Uh, I personally like it because it works well. Uh, I work on it. So of course I like it and um One of the attributes as is knowledge of multiple sources. So basically you can Point to different sources anywhere like any type of vcs any type of uh tarballing equivalent, um, zip or that standalone dab or standalone rpm or Whatever you can think of we can add And it just understands that and um can can fetch it It is an easier way to package applications. It's uh declarative. So it does Provide a simple view of how to do stuff a common way since it's declarative It's mostly driven by something else that that you don't write there in the packaging. So then um Things just get done it if you think about deb helper in some sense Without all the overrides it can look like that but It's actually much more simpler. Um And um Then it has all the necessary facilities to talk to the store in this case. It talks to the boondoo store And you can do a bunch of things with that. There's nothing that can impede It to talk to a different uh store that is not the boondoo store But since there's no Demand it's that there's no store existing other existing store today. We don't just don't have that so the basics of the snack graph uh project are basically a snack graph.yaml where you just define the project um And in there you have uh parts where our which are the basically the building blocks uh, is it to To point the sources. So each part will have its uh a source entry and um all the parts are driven by um plugins and The plugins are basically what what uh give the the part the semantics it needs to actually build that source and um And the parts themselves are isolated between them. They have uh an own their own area to To pull uh sources their own area to build them and install them and then they get all mashed together at the end um So that's where the the life cycle of a part comes. Uh, all the parts have a life cycle. Um, so there's pull build stage and prime uh pull Is implemented in the core and in a plugin um A plugin for example could be a python plugin uh auto tools plugin. Um a cmag plugin So the core will will actually uh do what it needs to do with the sources and then the plugin in its own method Pull method can implement specific things it wants to do while it's pulling Uh stuff for that specific plugin to actually work um build is also um Uh implemented by the plugin and the core the core sets up the build area for the plugin and then the plugin can actually build the um With its own logic depending on the plugin uh build that actual source and Then uh the core executes the stage part of the life cycle and the stage staging area The stage step is is basically a staging area for snaps. Sorry parts to Share um different uh components that They need especially useful when you have uh parts that depend on another for example you have a part that builds a library and um then the next part will actually You want the part to link to the uh sorry to have a defense on that other part So to build uh an executable. I don't know like lib curl and then something that uses lib curl So the the library headers package config whatever will be in In that staging area the next part that comes after it will understand that um, we'll We'll we'll make package config work and uh, we'll make everything Seamless so that the the part that needs all that stuff just works and can build And then the priming area or is basically the final area where uh stuff gets uh set up to create the final snap. So whatever goes in there Um is what eventually your snap will look like And uh, so the stage and and prime area how the stage and prime Um have um our key words also that a part can have where you actually filter out uh file sets They want your part to to put in there Um So sometimes a plugin is not enough and we do have the capability of um For you to create a new plugin just inside the project you put it inside Inside the repository inside snap plugins plugin name And you can copy a plugin and override it you can um You can um inherit from it you can write your own new plugin and um We've been uh, this was introduced a long time ago But we saw that many people just did one a one liner on the plugin and and it was just not Not it was a lot of work from from for minor gain So uh, we introduced scriptlets in one of the later releases of of snap craft And those are prepare build and install And um, this is where we move away from the declarative nature of snap craft and go into Uh an area where you can just in prepare build and install you can just write shell code. Um, So prepare gets executed right before the plugin, uh gets its build method invoked So the the build area is set up, uh by the core and it runs prepare and then prepare you can do whatever you want You can patch code you can do run said you can do whatever Anything like in in vlc before this for vlc to actually be snapped we added So it doesn't use autogen vlc to actually prepare its Configure autotools stuff and it uses this bootstrap Thing it calls and we sort of add it into our main autotools plugin with scriptless we would have had the need to do this and I actually would have preferred it to be this way So we don't have like a random code that goes out of the Conventional norms of how to build source code Build takes replaces the build step of the plug and so if you may want to use a plugin in itself To pull code and and whenever it helps it needs to do But don't want it to execute its build phase. You can actually replace it with this build and it will just uh Do what it needs to do? Well, you write shell code there like you can just like make If you had to make plugin and Use the build script that you can do make whatever target you want there and and have it have it your way And install is something that gets executed after the build. So sometimes Projects don't have make install target or don't because that's convention. Uh, they don't have They don't install everything you need to you probably want to set something and modify a hard code Path and and stuff like that. So you have to install thing the install script let to actually do all those manual replacements you want to do so this is all about the building part in the end you probably want to also Hook up whatever binaries you wrote into the system and for that in snapref.yaml. There's an entry called apps And this is how you basically expose Uh an application to the system There's two there's two Very broad means to do this. Uh, one is just like apps my app and command command is a relative path inside that Private area Or you can do dollar snap slash. So if you if you in prime you had like user bin them you'd put in command user bin them And for a demon it's it's also it's just the same. It's like that you you add a demon simple entry or 14 or Whatever, uh Most of those key or one shot all those keywords are are similar to the system d ones. Um, this is just uh today it is uh an abstraction for system d And uh That's that's all it is um So it's pretty simple to add apps um in in the general case So if I had um a snap And they called uh my snap and I declared my app Um to the system it would be seen as my snap dot my app in if it were a regular binary and If the snap were called my app and I had an app called my app then the system would see my app um And if so for the deferring cases and if you want like many top these are called like top level commands We have a concept of aliases where you can declare aliases for these and then so if you had my snap And an app called my app. You can say alias. This is my app. So then on the you'd see my app exposed to the system Um, this needs some sort of a connection for it to actually happen um So then there's uh confinement um to take into account. There's uh Three types of confinement right now classic dev mode and strict I'm gonna start with dev mode first uh because uh classic has a gotcha So dev mode is basically Everything accessible for the snap you can talk to anything you want See anything you want and it's the easy way to start up Then uh, there's there are mechanisms and tooling to see What security profiles or interfaces you'd need for your app to actually work in in a strict mode and strict is The the final stage of you want in forget in your snap strict is basically nothing is accessible except for those segregated data stores and um And the way to actually access anything you need is uh through an interface and classic is a confinement value that is interesting And it gets confusing when you talk about classic systems and classic confinement Uh classic confinement only works on classic systems If you have a pure snappy system, it does classic confinement will not work classic cys uh classic confinement is special because the Uh, if you remember I said apps see the core Well, that is not true for classic confinement and classic confinement Uh app snaps actually see the classic uh system so That that brings a lot of problems for starters, uh If you build us uh a snap, um without taking any precautions You would use the linker on the classic uh system And that means that you won't work on any other system probably if you build You won't work on any other system except the release that you worked on So we have a bunch of things in snapcraft, which actually force the linker to use the one and In the core snap So then that actually enables us to Build a snap on on xeno Use the linker from core and be able to have that snap as a classic snap on trusty where glibc is different If this weren't the case, uh, we'd probably get a crash syncyte fault or some weird, uh resolve nss problem because dl open would do the wrong thing and uh Snaps would just not work there, but they do, uh, so it's a trick of that linker Uh thing we do and we also do Our path tricks so dl open also does the right thing. Uh, it's not actually our path. It's front path that we use but That actually enables us to have a classic confinement, but The problem with classic confinement is that everything is open so You probably have to trust the developer that if you're a user you want to trust the developer That built that classic snap. Um, it's kind of like installing from a ppa. It's the same all the security problems exist there so if someone if you install a classic snap that someone pushed and It's it does an rmrf on everything you have. Well That's that's the same thing that happened with the ppa. So it's Same problems. So classic snaps classic confined snaps have a couple restrictions where they have to be built They have to be open source and built Through some open ci so people can actually look at it and it has the same trust levels Could have the same trust levels as Building from the archive So for strict confinement, we probably need interfaces or snaps are probably useless unless it's a hello world So how do interfaces work? So interfaces are common across snappy Some interfaces may be available or not depending on the architecture hardware you run on Because interface is not just about security. They are also about enabling Perforals so if a peripheral is not available on the system that you probably won't see that interface so if you don't have a webcam you won't see a camera interface for example if you don't have a um Some specific i o from a raspberry pi that interface is useless on the desktop So you won't see that interface either on on the desktop So interface have very very specific names. Uh, just so people can have a common language to to work on And uh, basically you have on one side you have a plug and a slot So if you want to use interface you you you plug to it and if you want to create an interface which you can You create a slot The core snap has a bunch of interfaces it provides like a camera network Camera network and all the others you can see by running snap interfaces and Then you can also Create ones By by doing what I show here So if we look at this example, I have an apps entry my app with its command and I declare plugs And I say network, which is part of the core snap That gets auto connected today. It gets auto connected That means that I finally install the snap that interface will plug into that slot automatically um And then I have the cute uh plug here, which uh, doesn't exist on the core snap so something needs to provide it and I have another snap snap v which has at the top level snap graph.yaml slots declared and I declare a slot called cute Which has an interface of content And the content is cute and this uh, is parametrized by the interface read It's specific to the content interface as a read dot which basically means read everything from the snap On the the the consumer side on on snap a I'd have something similar declare at the top level to parametrize the uh the plug By default um like network There's no Parametrization I need to do so I just with just doing network It's fine if I need to provide parameters to that interface I can I can do that and um All that is very documented and and easy to easy to see So What what what happens here in the case of the content interface and and some other interfaces? So network gets auto connected that means I install a snap I want to plug into network which is provided by the the core snap so it gets auto connected For this cute one it doesn't get auto connected by default. Well Let me backtrack if I if I produce the snap a and snap b And install both they do get auto connected But if michael created uh the the snap b and I created snap a And I install snap b and then install snap a it doesn't get auto connected. I need to manually connect because um There's no trust there There are ways to create that trust uh through things we call assertions on and that's a store side thing so if There is uh enough of a trust between us where we will guarantee that we will always work Especially for the content interface, which is kind of tricky because it requires by binary ABI and api stability Um Then we can we can create this trust relationship for it to be auto connected and this is where the um The concept of of the platform snaps come from So katie has their k framework's five uh snap Which basically is a content interface provided snap and it gets auto connected to all their applications They they're they're creating that snaps um canonical has one for the platform uh It went to the itool kit and all the the snaps around there Which work likes by the way get auto connected as well But if you're a different developer you need to manually connect to it Unless there is this trust that we establish, but it's not given out lightly because it needs needs some sort of guarantees Well, if there if it's breaks, I guess users will stop using it. Uh, that's it so that's um All the theory and I'm going to move into examples of how to write parts now And you'll see how it's pretty easy to do. I'm going to go from simpler to more complicated And if I have time I will move to actually Going to a terminal and showing it off So Let me turn some water So let me start with the python plugin the python plugin got a lot of love Uh over the last year because um many people Uh use python So today, uh I don't I don't know how people keep track of snaps or if they do at all But uh today for example, um, so at fosdom, uh, james page who works on open stack showed All the open stack snaps he wrote the from cli to core open snap components Uh, we have conjure up which is also a snap conjure up is something that allows you to really easily use juju in containers and and um and create Like open stack instances inside containers and get started really quickly or kubernetes and stuff like that. Um So that's all using this python plugin and um So this is a very simple example where I just say Plug in python. I don't use any sources and I use this python packages Thing that uh, sorry entry that uh the python plugin actually provides not the core of snapcraft And I just say Just using pi pi semantics and syntax Give me ascii cinema one dot three dot zero And it will it will so what happens here during pull, uh Snack ref will will fetch python for you the python interpreter will set up set up pip correctly will Um fetch the packages that you need Which is ascii cinema in this case from pi pi and then it will move to build It will actually create a wheel Install ascii cinema You will have it available. So then in an app sensory you can just say ascii cinema command ascii cinema And it will just work just like that It's very simple. It's just like magic. Um So here's another one katkin katkin is used to build uh Roz for it's used in ross uh robot operating system if anyone isn't familiar with it It's also another it's a matter packaging system. So it's kind of declarative as well All all you need to do is like say where your sources are source implicitly is dot But I'm using I'm pointing it out here. Just so it's more clear I just plug in katkin and then use these specific ross entries. Just say ross distro kinetic Ross has a release cadence similar to ubuntu They basically work out on ubuntu So it makes uh so and they have their own series naming as well What kinetic is their latest one the latest lts one And I just say katkin packages And tell what to build and it just does the right thing and then you'll have your your whole setup for for ross automatically done for you Auto tools now is also pretty simple, but I also added some uh core snapgraph primitives in here So if you wanted to build them this this is how you need to do it Um Just uh tell where source are we have a source sub dear also core primitive until it build from source So check From from the source just build it from source So basically make your working directory source instead of uh dot So the plugin and then so auto tools and we give it uh a config flags, which is part of the plugin and It's like enable cscope and enable python 3 interpreter Because that's what I want in my snap I don't want the pearl stuff or the the other crazy ones and um Here we we have a new concept, which is stage packages Uh stage packages is a way to fast track your development so One way to do this would be to create a part which creates uh, which basically Stages lib python 3 out 5 for me and build it and all that stuff, but I prefer to use stage packages here First for simplicity and then because it's faster stage packages basically goes to the ubuntu archive Fetches that for me and and puts that in in my part. So it's available for me to to use so And satisfies my my build requirements and my runtime requirements. Sorry for for uh for running them inside the snap And in for build packages. I build packages is something that's installed to the to the build system to the host system And well, it's basically like a build depends and and I've been And here's the prime keyword. I mentioned earlier so When I use stage packages it brings in the devs and it unpacks the devs into the install area of the part and and the then it finally gets Makes its way to the snap and there's a lot of cruft in stage packages from a snap point of view And I don't want anything in user share. So In the final snap at least so I'm just in in that line. I'm saying hey, don't don't bring in user share. That's all To get more information about like what the plug-in parameters are you can run Snapcraft help plug-in name and I'll give you all the core keywords A plug-in provides and if you and you can run snapcraft help plugins to get all the core keywords The snapcraft core provides and with that you can basically build your your Your snapcraft.yaml or the parts part of it at least so CMake and this this gets interesting because I had a new keyword there, but CMake is just like Um It's just like Auto pulls in some way. Um, you declare so I declared the source I I said plug in CMake. I gave the comfort flags there Which are basically CMake Configure flags in the CMake terminology build packages a bunch of packages I need on the host to actually build this snap and I give it an after keyword. This is how I I basically declare part dependencies. This means that Implicitly you can't see this and I'll get into this later I depend on a desktop Qt 5 here It that I don't declare it locally, but I depend on it Snapref will do the right thing. We'll we'll set that up for you and then build the rest of it And what's also interesting about this is that this is uh, this is key pass Xe which is the reboot of key pass X2 or key pass X. I've lost track of all the names for key pass Um, and this this is part of their their ci. They already published snaps for this. It's already in place Which is kind of cool And this is all they needed to do for it to be available to any basically distro that adopts snapping So go goes also used a lot in in um In uh snapref projects, it's so lately we observed that uh The people that snap the most are the ones that are on the most trendy technologies At least in source wise like you see lots of rust. You see lots of go And all the hard stuff that that is hard to make as a dab is probably a snap today already and a lot faster people just See it as an easy way to deliver continually and um And to any system at least easily So ugo is is basically a uh a static uh site generator I use it for my own blog. So I actually use this ugo um application also use key pass x Which is a snap. I haven't migrated to xe And it's pretty simple as well. It has the um So if you know go this uh, this will look very straightforward to you is tell where the source is you You you say plug and go and uh, you you tell it since the source is so Go requires to have like a absolute import path And in this case, we're telling it. Um, this local source you have Is this import path? So then snap graph does the right thing for you actually build that and it's isolated Go path environment And this is basically a copy of of upstream as well And they have stage packages by thumb pigments to actually do the coloring when you do uh In your blog you write like code or whatever and um So there's a couple more go keywords here that you can use which are interesting One is go packages which actually goes and fetches all the go packages Or if they're provided locally by the source that doesn't and has a lot of magic to make things fast And for example and go it will in its pull step it will fetch fetch go for you and um And it and all it will go get everything without building it and then in the build step It will just build and deliver and Well, it's pretty straightforward lots of go projects around um Lexi also uses this um and um I can't talk about the others. Sorry Um go depth is uh similar plugin to go that provides it solves uh So go has a problem where you can't tag uh revisions So you're always on the latest and greatest and go depths try to solve that by By providing this dependencies Dependencies tsv file which has sources linked to revisions or tags And by default this go plugin looks for dependencies tsv. You can define it yourself But it's similar to the go um Go plugin and it has a go import path. It has go packages where it tells it built from this go source Just build juju and juju d for example Um, and don't build anything else which is uh in there By well go only build main pack. So no, sorry But when you go build something or go get something you'll you'll always get all the main main packages in there In your in your go path bin um, so go packages basically tells you just do these In a sense, it's similar to to the go one except it uses dependencies tsv and uh, this is all part of snap craft, uh, there's one more uh interesting way to do it and um, so Like it or not, uh electron is everywhere and electron apps are everywhere and Everything today basically using electron. This is hyper the hyper um This is the diff for basically creating allow for hyper which is a terminal written electron to basically provide a snap and They already have this upstream in their in their master repo and This week or next they're enabling actually continuous delivery to provide a snap for everyday use But it's super easy. They just needed to update the so this is using electron builder They just needed to update the the electron builder dependency to the one that actually can provide snaps and just add Two entries there like confinement classic and grade stable, which i'll explain in a bit Uh, so that's a run through of plugins. Uh Later today, there's a session about actually working on on on these and we can dive into Plugins specifically or expand them or do whatever you want and i'll be available for that I'm gonna stop talking about plugins right now and move on to a bit more about snap craft itself other things you can do so earlier i mentioned an an after desktop qt 5 which was a dependency And it was implicit in the sense that that was a shared part somewhere So You create up all those parts. We saw some can be very complex and difficult to create and instead of having like So a part can replace a stage package And you can if you took a lot of time on creating a part in the polishing unit Or you are a domain expert in that source. You can actually share that part on wiki wuntu.com Snap craft slash parts. It's very open. Just add a couple entries there point it to your to your repo which has the snap craft project in it and We do processing on the back end and actually provide like a nice interface from a client side for you to use So with snap craft update you grab fetch the latest Parts listing archive you can search for a part name and basically with define you can You can see what that part is and how it's built out. So there's no surprises So that after a desktop qt 5 is actually provided by this shared part you can do You can override some of the definitions in there Or you can completely just with by define running define You can just copy paste it and keep it locally because maybe it can change and you don't like change and or whatever So that's how this works. So going back to the ASCII cinema example. I'm extending this to To look a bit more complex and so if If you ignore the line from after below and below And just look at the other part. That's exactly what I showed earlier and by default on the pull step the the plugin will fetch the The the python interpreter that's in the zeno. That's python 3.5 Three years from now that might be too old. Maybe now it's already it's too old for you and you want to use the latest python or maybe you want to optimize your python and This this is how we do part dependencies. So With by saying after python dot three that means that That that ASCII cinema Part won't run through its initial pull step until the python 3 part is staged And well to build python 3. It's just just like that. It's not no magic to it That python 3 will be staged in in in that stage area The python plugin is smart enough to understand it's described in in snapcraft help python It's smart enough to detect that and use that python to Do everything it needs. So in in this case This is how we we can use Have ASCII cinema with python 3.6 And it's especially interesting with A classic confinement. This is sort of our requirement today for classic confinement because of the ld library ld linker trick I mentioned and the whole our path run path thing You sort of need to do this if you want to build a classic snap today We're working on not needing to do that, but a stage package does not have That linker information built in into its elf headers. So And we don't want to modify elf headers So This is sort of what you need to do anyways for for a classic confined snap So that's basically how you use a shared part as I mentioned So I just do a snap search Desktop qt 5 I find that I say after desktop qt 5 desktop qt 5 I can snapcraft define desktop qt 5 It will have many entries. I can one of those those could be the source. I can just change the source entry and Have it do the same thing except with a different source Or I can just completely copy it if I wanted to and not depend on on this Rolling desktop qt 5 implementation of the part Um, so that's what this all I have for shared parts So snap script all I mentioned also does a store Can talk to the store and one one of the things it does is basically It can push and release Two different channels on the store So I didn't talk about channels at all because that's a store exclusive thing But I can I'll briefly summarize around it a bit So you build your snap you have your snap there. You can snap install it Play with it. You want to reach users you You can snap craft push and release To the store release basically takes a channel name by I don't want to say convention, but by By the means we thought of We have an edge channel a beta channel Candidate channel and a stable channel By default users that snap find the snap will only see things in stable That's just because we don't want users to see things that are influx or in movement But if people track, uh, whatever blog or rc channel or main list and notice that there's edge Edge things released into the edge channel Beta candidate they can actually install them and that the The they'll know what they're getting into But by default we don't want users to see them. Uh, we also have the concept of lts channels and this is uh, when we Uh, don't want users to automatically since everything refreshes automatically Sometimes you have a major release that requires some work or like at cd From two to three change this whole data store concept. So you need some you don't want to roll to 3.0 automatically So aside from, uh, stable candidate beta and edge. We have an lts track we call so you can say, um Release the 3.0 slash stable and and stuff like that with, uh You can also snapcraft check your revision history. So you just run a snapcraft history Or list revisions and you'll see all the revisions and when they were pushed And they're released status in the store since if you forget you can check that And you can also see the status map of revision versus, um Channel, um in in a nice table with a snapcraft as well Each i didn't mention this but uh, you can only push, uh So each build is a revision each push is a new revision. Um, each architecture you build for is a different revision And um snapcraft has a concept of version, but it's only a user friendly, uh, a user visible thing so Gone are the days where you you de-put something And mess it up and you lost that version that nice version you had and you had to add a random character there for it to To actually be accepted again And so that's the reason why we chose to decouple the actual revision Of the blob and the version of user sees So they mentioned that you can push and release to anything but what happens when you build a beta app Which is basically a name. I just randomly chose Something that's in beta and you release the stable by mistake. So There's a concept of grade there Which is basically, uh, we have two grades develop and stable Grade develop is kind of like a hint In in the store. So you don't Accidentally release to a stable or candidate. So that's um Just a help a helping function for for any developer like for example, you're building in ci Um, you have a rolling thing that just moves on you probably want it to be in grade develop. So Uh, it doesn't excellent. Let me get migrated to stable and when you're in the final releases of of your Tagging your release, uh, you you tag it. So what you tag will have grade stable. It'll be good and you you can release to candidate reg So that's all building locally. There's a nice tool that was presented. I think release showed out last week and it's basically A thing for to have continuous integration and delivery from straight from github github is what is popular today And everyone uses github. So that's what we started out with But build snapcraft.io is kind of like a a front end to launch pad builders and you can easily set up your github repo to to build and continuously deliver to to the edge channel with Basically the workflow that any github integration provides instead um I don't know if anyone has used launch pad recently, but uh, the ui is kind of old looking and by being old looking is also like Hard to navigate But launchpad that did have this feature and it has like a small I don't know if anyone used recipes before right by the create recipes You have like a small create snap thing and uh, you can add the same integration if you had the Your sources on launchpad Or this is the same sort of the same thing but for github and all streamlines And easy and straightforward. It's easy to use to provide your snap so I know talked a lot and probably Poverty of everything and people might be lost and But it's just like a lot of a bit of information to get people interested and see how maybe easy or hard it is to Get things going But there is more information on sandcraft.io it has uh, basically a Small guide to get started with snaps itself and then there's a snapcraft.io slash create which has A guide on how to go through snapcraft. Uh, there's a snapcraft tour which you can take and and You can work through the different examples and build them and tweak them and just See what the problems or corner cases are in when creating snaps and recently added There's this tutorials.com which are co-labs that guide you through different types of Different things you can do with snapping itself as a system and With that, I guess i'm done. So if there's any questions I'll take them. I just want to close up saying that I don't want to say that creating snaps are easy They they are a bit hard today. There is a transition going on that needs to happen with uh Just like any transition like moving to a confined world is is a bit different to a world where everything is open I know everyone probably has well not maybe lots of people have been tracking wayland mirror migrations And how long it's taking and once it was to be so easy Well, same problems will happen with snaps themselves especially for old projects That took a lot of considerations of expected things that should be there always And so if you don't actually know the sources you're working with It can be hard to snap But if you know the sources you're working with it can be super easy Because you know what all the gotchas are Adam if you saw Harold sir if anyone knows he works for On kde and he actually did a create a snap with the The plasma desktop itself and he know all the hacks or modifications He needed for that to work and it was fast for him, but anyone else would have had issues with that But it was cool work and I don't know if you saw the demo or the screenshots he provided about it Fine kind of history. It's interesting. We're sort of following the same path with unitiate and and all this work was also Providing ways for us to create new interfaces For for all the other people coming in then not to need to worry about this When they when they tackle the problems And those are my closing remarks now. Well, that was that was a very good talk. I appreciate you're putting that together Um, I'm we are technically in the lunch hour. We do have a raffle to do But I would like to take a couple questions if we have that I'd like to make time for that Do we have questions on this? And if there's no if there I'll just add that if we run out of time, uh, we have the un-conference later today And we'll be doing snappy as well Yeah Well, that's actually not a bad idea. Maybe a snap section in the un-conference Um, would be a good option. Do we have any urgent questions while we're here? We'll take one. Okay For the for the people that are using this on desktop systems What's the plan where you need access to privilege devices? For example pulse audio which provides no Layering of security models in it and you can load plugins in the daemon So you have a very direct anti confinement attack Okay, so I don't work on interfaces specifically that's part of the snappy core team, which I'm not part of anymore But I do know about it. Uh, uh, so there is a pulse audio interface in the works if it's not already there Uh, this pulse audio interface is interesting because on classic systems the It's provided automatically by the The interface is satisfied by the the the system itself and on the budu core There's a pulse audio snap that you can install for it to be, uh, provided by that pulse audio as a slot Um, but there is an interface for pulse audio already. So Right, that's not my air expert expertise. I'll have to say i'm sorry. I can't answer that. Um, but I can I can forward you to the person that can I'm sorry Okay, we'll take we'll take this last question for now. Sure Uh, is there anywhere in the forecast? Sorry um A plan for like a GUI where people can see and manage all the interfaces and how snaps and are relate to each other and Something like that down the road. Is that part of the thinking process? So there are there are plans for um, have you ever used the budu phone? So there's this concept of trusted helpers that I should a lot say like Hey, you want this snap needs the camera, uh interface. Do you want to connect to it or not? And, uh, there is, uh There's groundwork on on the snappy core team itself to provide the mechanisms for GUIs and so snap core team is going to take care of the cli for that and then um and provide the mechanisms for Uh, the GUI folks to actually hook that up and provide nicer interfaces We have a command line interface for that right now that'll show you all the connections you have and you can connect or disconnect manually But there's like a non-demand system that's being worked on as well Okay, well, I appreciate the question. I'm I'm mindful of the time people will need for lunch So but let's do continue this. Um, you know, we can have a session Yeah, the unconference workshop. We can do that I'll have the answer for the pulse audio question. Bye then. Okay most excellent Now before we break for lunch, we have one last thing to do and remember we'll break for lunch And then come back here at 130 and we'll plan out the unconference section. Um, So we'll see you here at 130. But before you go someone in view has Has a winning ticket for um, the uh, um cloud Next cloud box. Okay. So let me shake this up and we'll draw a number here One five one eight four three nine But of course Okay Very good. Carl will will arrange to have that Some picked up you can pick that up at the at the next cloud booth Uh, the next cloud booth is here on the expo floor, which will open at two Enjoy your lunch. We'll see you here at 130 and we'll plan out the unconference sessions. Have a good lunch Test test Just a second Oh my gosh That's disconcerting Awesome. All right. So there's a couple of requirements that you have to have if you do want to cheat You have to have UEFI based firmware. Otherwise you're stuck using traditional grub because UEFI is kind of amazing, but it takes a little bit of uh There's a little bit of magic that goes on here You need to have python 3 it it is written python 3 because I don't live in the 2000s anymore We're in you know, this is 2017. We can do we can do python 3 You need to have a kernel with uh, the efi stub option turned on At compile time if you run a modern distribution, you probably have this I know bintu has it red hat I'm sure debbie and does it's very very common If not, you can recompile a kernel and and and turn that option on yourself It takes a little bit of time to set this up Much less than it took me to write all of this, but uh, you know, it's uh It's a it's a thing and then you need a little bit of courage because you're messing around with stuff That starts your computer up and if you can't start your computer up things go scary We will get to that in just a minute So There's a couple people coming in late First of all shame on you for being late Well This is way more important It's okay That's that's been my strategy. It works great Or wait my boss is in the audience So, uh, you missed the first part of the talk, but I told you what it was How to make a boot order in 300 lines of python you can't But you can cheat With these system requirements 30 000 foot overview All right, so there is just a little bit of background information that's relevant here So the kernel does have its own bootloader built into the kernel It is the efi stub and basically what it is is it's a teeny tiny efi executable That loads the kernel into memory embedded at the start of the kernel itself It basically just turns the kernel into an efi binary And so if you have a kernel that has this efi stub built in from an efi shell You can actually run the kernel from the shell Like any other efi program And it's it's pretty neat. It uses you efi But if you want to use this to boot your system, you need to manually configure it It is possible, but you end up writing commands that look like that Which are long and scary and have lots of options that are kind of hard to remember and and you need to know what your uh Uuid is and and how to find all that and it's it's a pain and and and I didn't mention that you have to do this every single time your kernel updates Yeah, there must be a better way like, you know stupid silly, uh, you know Zooming text So I wrote a program called kernel stub Which is the automatic efi kernel stub bootloader manager Which is just basically a bunch of buzzwords for it does things for you. It's written in python 3. It's under a simplified bsd license It tracks kernel updates. So every time your kernel updates, uh, it runs again and makes sure everything stays up to date It auto detects a lot of hardware information. So it's fairly robust. You don't have to worry about it, you know You know, oh, I'm on an nvme drive, which isn't listed the same in in dev I'm I'm on, you know, my main installation is on is on sd3 instead of sd1 All that kind of stuff makes all of that not have to happen It does leave grubb as a fallback. So it's very safe You don't have to worry about not being able to boot your system if something does go wrong or if if You know, something weird happens and it's only 347 lines of code as of the time of writing and that includes the big long simplified bsd license in its entirety at the top In a big block comment. So the the whole program itself is only actually about 300 lines Uh, a little bit here. Why use it? Because grubb doesn't, grubb works, right? I mean it it boots my computer up and it works fine. Well Uh, it using this saves you having to load grubb every time your computer boots up Uh, in my testing on my fairly fast system, uh, I noticed that it saved around four to six seconds Uh, which when you're dealing with 15 second boot times already is kind of a lot Um, it means that grubb related problems are generally eliminated Uh, some for example, uh, booting with this gives me better resolution on my virtual terminals During boot up and shut down as well as if you switch to one Something sometimes that doesn't work quite right. Uh, you also don't have to worry about any like weird vt handoff options Any weird graphics payloads or anything like that the efi talks directly to the kernel And uh, you don't need to choose a different os generally Because you can you can do this for any linux based os's on your system And then um, basically what that does is they'll add their own entries into your boot menu So like when you press the special key to choose to boot off the cd rom or the hard drive Most efi's will let you pick anything installed in the nv ram there as well And so you can actually pick different operating systems there even if they're even if they're all installed on the same drive So a couple of caveats to this um, obviously nothing is is perfect It does require a slightly larger esp Which is the efi system partition. Basically, it's a A small fat 32 partition that lives on your disc that stores the bootloaders for any operating systems or anything that you have on your system Basically takes the place of like the uh, the mbr On a on a traditional bio system And in order for you to use The kernel stub loader the kernel and the init rd image have to be stored in the efi system partition However, uh, most operating systems Right and now the only reason that the kernel is stored on root is because that's where it's typically managed with package managers Managers and stuff like that. It doesn't actually have to be On the system at all like with a network boot. Um, the kernel is never actually on the system at all Uh, depending on the network boot configuration, obviously But uh, yeah, so it it and it can live in both places too. You can copy it back and forth, which is actually what uh, kernel stub does Um, and it's worth noting that most operating systems already use An oversized esp the one in ubuntu for example is about 512 megabytes Which is plenty of size to store a kernel and an init rd um, the way I have this set up. Uh, it doesn't keep Old kernels on the esp you only have the option of booting the most recent one through this otherwise you would use grub So so it's plenty of space for storing one kernel one in an rd and then you know for 10 os's or something like that You know because kernel images aren't huge Um, it can be a little unfamiliar. It takes a little getting used to uh, if you're typically in grub a whole lot If you have to spend a lot of time in grub Why don't you buy a system 76 because you're obviously got something weird going on and and this would make it all much easier Because you shouldn't have to spend a lot of time in grub um You still need grub to be installed. Um, it's not a hard requirement You can definitely use the system perfectly fine and never have grub installed along with this But if you need to get into recovery mode or if you need to boot once with a different kernel flag set, uh, This doesn't let you really modify it The the entries at all you have to you have you're kind of stuck with what you enter in When you set it up And you can change it out anytime the system is running, but of course you have to boot it first So that's that's a little bit of a caveat. So so grub having grub also installed is useful for for working around that You just save having to load it loaded every every time you boot It's also useful as a fallback All of these entries are stored in something called the nv ram, which is a uefi term for basically where All of this bootloader code is registered with the firmware So the firmware knows where to look for it And if that gets reset like for example, if your cmos battery dies or someone removes it You'll lose those entries because they're they're not stored in a in non-volatile memory Um, and it's also alpha level software. I've tested it fairly well on a on a, you know, huge lab full of seven machines There's a lot of hardware configurations. I simply can't test so Having more people look at it is obviously going to help with that, but Having a having a safe fallback that we know works and has worked for 20 years is is sometimes a nice benefit. So Um, what do I need with efi stuff or kernel stuff? I need people to test it so that I can get lots more Information on what hardware it works really well with Any special edge cases that it doesn't detect properly? Um, if there's some weird thing that that doesn't work very well If there's something about it that could work better if there's something about it that that that works really well I just need feedback on on how uh, how that works Specifically, I need people who can break their system and not be like, you know Get fired for for that. Um, because you know, some people have computers they work on And if they break that they can't work and that means they get fired. Uh, and that's a problem I don't want anybody fired so Uh, so people who can break their systems if it's in a vm most vms have support for uefi I know virtual box does it's free. So so that's a good way to test it But uh on different hardware is also really valuable obviously Um, I would need help with packaging because who doesn't need help with debian packaging It's uh way too complicated and then uh help with code review and cleanup. Um, I am not a professional python developer I'm like fairly competent in it, but I wouldn't call myself professional through by any stretch of the imagination There are much better people for that and so if I'm doing something that could be done more efficiently in another way That's kind of you know, really super useful and that's why open source exists, right? How can you try this out? So the code is hosted on launchpad. Uh, it's all managed through bizarre I'm sorry. I learned bizarre before I learned git and uh, I look at git and I go This is always so complicated and I don't get it. I'm going to use bizarre Uh, if you want to Download it through bizarre and then upload it to a git branch launchpad does support git as well So you can you can you can do that if you want. I don't really want to maintain that so Someone else can do that again open source There's a read me file that talks about how to install it set it up all a configuration Um, it's basically just run an install script There's a configuration file that stores your kernel command options and then just run it and it kind of does this stuff for you Um, and we'll have a demo At the end of the presentation Bugs are also tracked in launchpad just because Launchpad makes it really easy to have everything all together You have to worry about you know, go to this weird website for bugs go to this weird website for code go to this weird website for you know pictures of dogs with bows on their heads that relate somehow And uh, yeah, so that's that's the presentation And now we'll do a couple of demonstrations. I'll show you how the program works We can get a little look at the source code and we can see Um See how much difference this makes. Um, I saw a question Uh, yes So it looks like it's not this is because I wanted to be do a big special reveal and I forgot to make it public before the presentation And I don't have a team of of of web developers back at the office Who deploy the changes as I'm talking like carl does so Because I'm just one person But but yes the launchpad url is correct and we can make that public here Very shit very soon. Thank you for pointing that out So let's go ahead and take a look at it here Um, so I have it installed on my system So I don't need to go into the the development director, but I am going to anyway just so we can get a better look at the code um, let's just uh Perfect, so we can cap the Source code here and we can take a look at it. Oh, there's no syntax highlighting there. This is basically it. There's the license Uh You can't see any of this can you? There we go. Look there it is So yeah, there's a bsd license. There's a bunch of functions for handling things and then we got uh Big main function at the bottom that actually does the I think I'd even say Uh Where is it? Yeah, do the thing Do the thing right there anyway So this is this is it. Um We're going to go ahead and I'm just gonna I'll do a couple of quick commands here to to demonstrate it It does need to be run with pseudo privileges because it's modifying your esp which is mounted Actually the way it's mounted you can't even read it unless your root So it's super crazy locked down. Um So we'll just do stub And I'm going to do dash s because I don't want to that's a simulate Here actually we'll just do the help first So here's the the list of options you can do you can pass it you can pass it kernel options directly if you if you want to overwrite stuff Um, you can specify a path to a log file. Uh, the amount of information that's logged I just use the python logging module. So if you're familiar with that, um, you can go through it that way It has a verbose option that just basically logs more output to the console by default It doesn't return any output And then simulate which makes it do it without actually doing it which is Kind of useful if you're if you're just interested in trying it out real quick so We'll just run it real quick here so we can take a look at the output it does Uh, and so here it's telling us that we found an os entry in the nvram, which We can actually take a look here. So this is what my current, uh um nvram looks like And then so here we can see it it found the os entry, which uh, it did because it's uh, right here boot uh, 000 It's telling us what that entry is and how far down the list it is Um, and what the index number specifically is it's giving us the name of the drive This is like the slash dev slash sda. For example, it just it's an nvme drive. So it's a little bit shorter Or a little bit longer It's giving us which partition name the root file system is on As well as which one has the esp It's identifying the partition number specifically because that's that's needed for The efi boot manager command that actually registers everything with it And then it's also giving us the root fs u u id and it's just pulling that out of blk id Um and uh gives us a quick a quick little bit of diagnostic information about what our operating system is and What kernel parameters we currently have configured to use? um, if I come down here and That's not the right one. There we go and add a some kernel parameters that I want to use we could say like, uh a quiet splash No mode set those are all common ones this Is not an option Uh and run it again What? Oh, you know, I forgot about that So kernel parameters you just enclose them in parentheses your uh quote marks I Yeah, not really so we can see here it it updated the uh kernel parameters that it was going to configure The place where it pulls those out by default is stored in, uh Uh etc slash default Where conveniently the grub configuration is also stored And we can cat that and we can see that literally the file is just the list of Parameters that we want to pass to the kernel on boot It will actually ignore anything else in this file. So it's only reading the first line Just looking basically at a line feed Because that's really the only other thing that you would need to configure about it Um And uh, yeah, and then there's also a Trigger that runs when the kernel gets updated that basically just runs this program If we actually run it And i'm not going to run it with those options So you can see it's super fast to update so it won't make your kernel updates take any longer You can see it has like uh what it's doing It's actually this is where it's copying the kernel onto the esp And it's also copying the init rd onto the esp and it'll do whatever the latest one is It doesn't it doesn't look for which one you're running uh possibly a good submission for Uh code code change, um, and it's listing us what the the nvram configuration is um And if I wanted to Change the kernel options to like add splash ashkay quiet Splash So we can see that it it added uh Quiet and splash and took away net.if names equal zero so There's uh, there's a little bit there for the next part of the demonstration I'm gonna have to unplug the projector and just kind of face Turn around turn the laptop around to face you guys because I don't know that the projector will That's that's uh an email Uh I wish it had come here instead of there Uh, so yeah, um, I don't think the projector will get the the The like firmware screens, but we can certainly try it out Um, so we'll just go ahead and reboot And I'll go ahead and get it into the boot menu Uh, and it's not there Maybe this will work Okay I was thinking that would be the case So here I just have uh firmware boot screen um if anybody can't see that please let me know so I can Yeah, it's it's a little tiny, but we can use it for demonstrating. Uh This is just the standard uh hardware boot screen that comes up that if I press and hold f7 it comes up We can see it has options for uh ubuntu 16 10. That's our uh kernel stub entry and then ubuntu which is with grub So if we time this With like an actual stop watch. I got it um So i'm going to hit enter right at the same time as I start the stop watch And then as soon as we get the purple screen I'll go ahead and stop it. We can see it's still going through grub grub was loading the nnd and then As soon as the background comes up 17.13 seconds, which is good. I mean it's it's definitely nothing to to be unhappy about it's sub 30 seconds That's you know, the most important thing with booting these days, especially when we're dealing with like Windows and mac. We're actually getting their stuff together and having boot times that are sub one century But now we're gonna reboot and Notice the the console resolution is not as good I'm just going to hold the boot key here And now we're going to choose our uh kernel stub entry I'm going to reset the stop watch Go And we can see it's already loaded the init ram fs, which is much faster than before and this time We have Sub 15 second boot from from what would be power on to Uh fully up fully operational system It's I I enjoyed the applause So yeah, that's it It's a significant percentage and it's it's important to note that um, you know It it it's it's one of those things that that like like we kind of take for granted now But remember when your boot was like You know a minute Back before we had like parallel booting and upstart and system d and all the other things that make the boot super fast Those were big changes, but those were like the operating system level We're down to the point now where the operating system is loading just about as fast as it possibly can The only way to make the operating system load faster is to have less operating system Which which uh, you should have scheduled an unconference talk for your Or you could do a bof that'd be good, too but uh what we can what we have to optimize now is firmware and and Sadly the firmware still does a lot of initialization that the kernel then does so there's a lot of duplicated effort in and you know, it just gets The the firmware will spend, you know, five seconds initializing some piece of hardware that the Firmware already spent five seconds initializing um, and and These types of changes where we can, you know eliminate parts from the boot process I mean we we've already trimmed we've already gotten rid of most of the fat We just need to trim it up to make it perfect and and that's how we get a really Super fast boot and and grubb is a piece of software that that for some reason doesn't load the in it already very fast on a uefi system Uh, I noticed that it it loads it much faster on a biospace system for you know, whatever reason Maybe there's more that needs to be in the in it already for uh uefi, but um And since it takes so long why use it, you know, it's it's flexible. It's it's powerful, but but we don't need to uh To include it for for most things most of the time So I think uh, that's about the end of the demo I can do a couple of the things too, but they're kind of boring But I can definitely take any questions on uefi if you have questions about uefi I did a talk about that a few years ago And also about kernel stub if anybody has any questions about that No, that's that's done and it works great. Um, actually no, that's a great I'll uh, I can demonstrate that too So and I think I have the kernel package in my apt cache too, so I'll make that super convenient Um, not sure why we're only getting part of the desktop background, but no There we go. Okay, there's that Close you. All right Nope, you're supposed to be over here. There we go. So, uh, if we do Install Linux image 4.8 10-39 generic, so we're actually just we're gonna Actually, we need to do a reinstall because it'll otherwise it'll just complain So we're going to reinstall the kernel package just so we can demonstrate the automatic Hook that that it's basically anytime the kernel is installed it it runs real quick and And does that um, it runs fast enough. It doesn't make it Any really all that much faster. Stop that There we go So here it's going through. Yeah, I didn't have to download it. That's nice Um, yeah updating the init ramfs takes much longer than updating the the kernel stub entry Because it has to build in all of the all of the drivers and stuff that it's going to need And we can see the output from kernel sub here because it runs in verbose mode on kernel updates Uh, so this is yeah here it ran it here Um and just basically updated that entry and now actually this is actually kind of an interesting way to check If we do if config We can see that it has the system d style network device names And then if I reboot when we get back into the os i'll have regular old Wlan zero style net names. Um, it's just a a preference of mine. I prefer to have those shorter. So I turn that off at the boot and Because I rebuilt it earlier with that option turned off when I did that manual run When it just rebuilt it it it updated that for me and updated the kernel parameters So i'm not sure why it's taking so long to shut down, but I can't I have no control over that part Oh, yeah, and you know what honestly, it's all system d's fault Um, so while that's going, um, are there any other questions? Um, I am almost positive that won't work Right now the code is looking for a file called vm linus to copy over into your esp And that's part of why That won't work because it's just kind of hard-coded for linux at the moment Additionally, I don't know if bsd has an efi stub loader, which is another requirement Mm-hmm. It would work Right, right. It could be I'm confident it could be adapted for for a bsd type os It wouldn't work in its current incarnation and that's not a path i'm Terribly interested in going down right now if you have a patch that would Add that sort of functionality without you know changing the way it works for linux I would definitely look and I would definitely be be open to accepting that but Because I don't I don't even have a Bsd os to try it on Uh, I I don't even have a way to even get started with it. But um, yeah, definitely it's a certainly a patch is welcome situation. So Mm-hmm Uh, I am installing that in the uh I guess that that is probably a directory managed by the debian package, isn't it? I have it in etsy slash post kernel slash postinst dot d Um, I'm not adding it to the to the kernel package Um Which I guess if if this were deployed on a mass scale that would probably have to happen But if it were being deployed on a mass scale it would happen. So Right Yeah Right, that's what that's what I yeah, and that's how it that's how it works right now Yeah, my my install script installs that file to that location a debian package could very easily do the same thing as well And yeah that that in my research. That's the way to make things run automatically on a kernel update And so that's the the method I chose. There may be better ways to do that Again patch is welcome, but um As far as I'm aware that's that's the correct way to do that Um, and I don't that that package may be handled or that that directory may be installed by the kernel But the the kernel package, but I don't think it's a problem if you install additional stuff there so Yeah, all right, um any other questions about uefi or or kernel stub or Sure Yes Or indeed with no os whatsoever Mm-hmm So that that's an interesting point to bring up That is exactly the point behind a very infamous piece of technology called secure boot Which attempts to prevent you from loading Binaries and other other efi executables that aren't signed with a pre accepted key Basically, it's it's it's very similar to how ssl works instead accept that instead of getting the key over the internet It's all directly in the machine And it uses that to validate the origin of the binary you're running And There isn't a Right right or Right Yeah, um that's a somewhat valid concern right now It's mostly solved by keeping tabs on your esp If you're concerned about it, you can you can look at that. It's also important to note that Without registering with the nv ram, there isn't any way to load an executable automatically Unless you explicitly load an efi shell With with a startup.nsh file. It's kind of like autoexec.bat if you're familiar with with uh, DOS stays and stuff But um, it's basically an automatic startup script that runs when it when the efi shell loads um, and if you if you put your your malware in the esp and then Trigger that that would cause it to to to run automatically, but otherwise the only way to get Any efi executable to run on Automatically when the system starts up is to register it with the nv ram which requires pseudo privileges For that exact reason So That is a valid concern, but it's mostly mitigated by you know Monitoring your system and if something pops up asking you for your password. Just don't type it in unless you know what it is Because right and that's that's another reason why the esp is mounted so securely like you can't read You can't read from it. You can't write to it unless your root Like like you can't even read the the contents. It's it's all entirely locked down Um, and it's done that way with uh, unix file permissions on the linux side. I don't know what windows does but Yeah Oh mac is a little weird though max are weird max are But it should ask you for your password when you do that Yeah, I I mean It's it's been many many many years since I've used mac osx But uh, I I'm fairly certain that if if there are passwords set on the you counts Oh, yeah Right that was back for the power pc stuff Right Now that's it is a it is a you know a thing that's that's worth looking at But as it as it stands right now, I mean, I don't know what windows is to protect it I know that windows file permissions are totally different max are an interesting case because max They don't mount the esp by default and they don't even use it Mac file or mac firmware actually has a driver for hfs plus built into it And it reads the the boot loader and all that stuff directly off of the the main system partition um There is Okay, and I know that I know that they also put firmware updates in the esp as well, but there's yeah, there's other things about that but So if there's a bridge boot loader in the in the esp then that that's a little bit different. I haven't Yeah Yeah so Yeah, uh It's it's it's security is always hard and Especially preventing boot kits from from being something that that'll infect a machine. It's it's a tricky problem You know on one hand, you know, you could you could solve it very easily by simply saying by simply hard coding your kernel Into the into the the firmware, but that has a downside of you can't upgrade it Uh, you can't you can't change it. You're locked into using that. So that's That's the The ubuntu kernel comes with efi stub, uh I did get in touch with a guy named rod who wrote rodsbooks.com and he's also the maintainer of refind Which is an efi Boot manager specifically designed auto detect. I think it's originally it's a fork of refit Which is for max and it's designed to be a little bit more generic But it's also widely used on max because it'll auto detect linux kernels linux kernels But uh, he indicated that um unless The kernel itself was signed with uh, microsoft's allowed signing key Um Loading this with secure boot enabled wouldn't work Uh, because the the kernel itself the way the way secure boot works in in ubuntu is there's a uh, a pre-loader called called shim which is signed with microsoft's key And shim allows you to install different vendor keys in it and it will load Like for example, it will load uh the kernel the uh ubuntu kernel, which is signed with a different key And it's still cryptographically secure, but it does it it handles the keys in a Hmm possibly Uh, yeah, I mean without without getting the linux kernel itself signed By microsoft it at every for every update right? This this isn't something that would work on a secure boot system, but um secure boot it's it's like it's well intentioned It's certainly not the evil thing that a lot of people make it out to be but it is somewhat inflexible and it's hard to work with and Right, right Right right and and Yeah, and it got kind of blown out of proportion, you know like like the there are some legitimate concerns with it I I it's I don't see it as evil. I just don't also I also don't see it as terribly effective. It suffers from a lot of the same problems that ssl does and that you have to Trust a single authority microsoft has already had A leak of one of their keys Which if you sign your efi binary with it, it will load on any secure boot system And microsoft has no way to even revoke those keys. So it has flaws that ssl doesn't even have Exactly Right right and so there's There's Right So so really the only way to to solve it is Just develop something other than secure boot that does similar things in a different way. That's that's you know revocable Basically, you know something that and and you'd have to keep this secure on the operating system side But something that you can manage from within the operating system would be would be the real way to solve this Something where the keys and the firmware are managed through an operating system program Uh, which would then would then you know, they would install keys into the firmware They would verify keys. They'd revoke keys out of the firmware if they were compromised Exactly right right and if it's if it boots it it would have already been signed But that's not secure boot because secure boot is already protocol. That's defined. Maybe it'll be secure boot version two uh hard to say but um Yeah, a side note, but uefi and secure boot. They're not evil. They're actually really good. They're awesome like pieces of stuff They enable all kinds of cool stuff like this to work Uh uefi in particular lets you do uh, you know like GUIs in your firmware Because it's just so much more flexible and stuff like that and and and you know Hopefully you should never have to go into your firmware to configure it because your your system integrator your oem has done that for you But you know if if your end user has to go in and change something. Why not make it easy for them? Right, you know, there's no reason not to not to to have it so inflexible There's an entire shell that that has you know the same basic capabilities as as dos That that can run solely on The the software that's based in firmware It's highly extensible. You could load you could load a file system driver for extended partitions And you'd be able to read extended for linux partitions from the firmware Right, so there's there's it's it's incredibly flexible Extensible as the the e, you know, that's what the e stands for is extensible So, you know, there's just a and it and it lets it allows the same hardware access as a as a BIOS does so Right exactly that's the firmware itself is is uh Physically on the machine So you can't boot it unless you've got some external hardware token Which would be Right I'm not a Kernel engineer, so I'm not the best person to answer that question. My understanding is that they actually have fairly good support for EFI level devices. I mean there's a Couple of of directories in the sys partition on on linux that that talk about EFI I haven't really explored them a whole lot My my understanding with EFI is mostly about how how the protocol works and and the initialization and Actually managing and working with EFI at a practical level the engineering aspect of it is not something i'm super strong with Um, but but that's like that's an excellent question and and there's a lot of potential utility there So definitely something that's worth worth checking out Right, yeah, um, yeah, I mean That's a that's a uh a valuable thing to to have happen I'm just not experienced enough to be able to say yes, that's supported or no. It's not unfortunately always learning Uh learned a whole lot in the past couple weeks working with this kind of stuff and and all that so Uh, I would That intel documentation is probably a great resource. I know it's a bit terse or verbose, uh and and dry, uh So it's not it's not like, you know riveting reading, but it it certainly would be helpful for that I I mean i'm working for a hardware manufacturer. We're working on developing our own hardware so i've read through the atx back Too many times which is once And uh, we are we're an oem. We're just like dal, uh, except at the only operating system we deal with as a boon 2 So we actually like, you know, we have our our laptops We also do desktops and servers. I'm not sure if you saw carl's keynote yesterday morning Where we demonstrated we have a new a new gpu server. You can put up to eight Tesla gpu's in it and uh have close to 40 000 cuda cores in a single machine for for doing deep learning compute neural networks that sort of thing You can we also demonstrated a new arm server which has got 90 96 cores running at I think two or 2.1 gigahertz In a 1u server chassis So we do servers we do desktops like traditional tower based desktops. Um, we have an all in one We've got very tiny ones based on the intel nook platform and then also a full line of laptops ranging everything from, you know, 15 watt ultra portables all the way up to Machines with with, you know full You know, whatever the current I think it's 45 watt desktop cpu's with dual desktop gpu's built into it In a in a laptop that you can fold up and take with you So, you know that's that's what we do basically So stuff like this is obviously interesting to us because because it helps us deliver a better experience for our customers Awesome. Any other questions? Alrighty Thank you all for for coming out and checking out my my Terribly amateur piece of software that does everything and I'm very sorry for lying to you about how to make a A bootloader in python because that's not possible, but but uh, you know, that's another thing about efi someone could make an efi Uh based python interpreter and you could legitimately have a bootloader written in python It's not impossible at all so No, i'm not So, all right Thank you