 The second two. Test, test, test. All right. We got it together, didn't we? All right. Wait. What's it actually okay when I'm like this? Person in the back? Is this all right? Can you hear me? All right. Okay. Up more. Funnier? Okay. Okay. Well, I can only do one of those. I can only be loud or funny, but not together. I'm not that kind of comic. Yeah, that's corey shtick. I can't do that. It's all dry and subtle. There we go. All right. So sometimes in scale, they have someone that comes in from AV and does an intro, and sometimes they don't. I don't know. This is the first talk of this year in these sessions that weren't like the tracks the other day, so I don't know whether someone's going to do that, but I do know it's one minute after. So I'm going to go ahead and get started so you don't have to wait too much longer. So... No. Although you should always have someone give an intro, by the way. I was a long time ago, I went to a talk about giving talks, and they said you always want to have someone give an introduction that the audience wouldn't want to see more than you. Actually, you know what, Bill, maybe you should... The audience doesn't want to see more than you because what they'll do is they'll introduce, and then what happens is the audience then they say, okay, and give it away for Kyle Rankin and everyone's sort of like... And it's really hard for, if you're clapping for someone, your first impression is really positive. You're naturally like, alright, I'm in a good frame of mind for this, and then as long as I got you for the first two slides, then probably it's okay. If I blow the first slide, then it's all over. Alright, but we're not going to do that. So my name is Kyle Rankin. Thank you. Yeah, right on. I am the Chief Security Officer at Purism, and we produce a security and privacy-focused hardware that runs free software. This is weird. This is the first Linux conference I've been to where I've worked for a company that's had a booth at the conference. Some of you, this is old hat, but for me it's sort of interesting. So you can actually go to a booth and see the stuff that I do for a living. It's awesome. And see me after we can just talk about whatever you want. In addition to that, I also write a monthly column for Linux Journal. I'm an editor for them. And I publish a number of books on Linux and security and things like that over the years. But today we're going to talk about the cloud. But before we get started, a little housekeeping. I'd like to issue a couple of trigger warnings. So just to be cautious for our crowd, there's going to be some content in this presentation that might upset or bother some people in the audience, so just to get this out of the way. So if you are a veteran of the Unix Wars, this talk will contain content going back to the time of the Unix Wars. So if you're suffering from any PTSD from that experience, I will be recounting some aspects of that war, just so you know ahead of time. If you are a proprietary software developer, this talk will contain negative statements about proprietary software. Especially if you work for an open-source company that writes proprietary software, which are many of them. I will be saying negative things about that. So just fair warning. If you use the cloud at all, I'm going to be saying negative things about the cloud. So just fair warning. So now that I've captured everyone in the audience, so now I'm going to go into the actual presentation. Just fair warning. But first, good news. So that was kind of a downer. But the good news is that we won, we collectively, Linux and FOS is everywhere. So as a community, we should be doing victory laps every time we have one of these conferences about how we just thoroughly dominated the world. If you look anywhere, you will find Linux and free and open-source software. So for example, in your pocket, and if you look in your pocket right now, you will probably have a machine, a computer, that is running Linux. It's the most popular operating system that runs Linux. Now there's a bunch of proprietary junk on top of it. But there's Linux, if you dig down deep enough, there's Linux in there. In your home, right now you're here, in your home are probably devices that are running Linux and a hacker is using them to DDoS that they don't like. But they're all running Linux, incredibly insecure and hardened on patch Linux, but Linux all the same. And probably a little bit of free software too. In your company's code, if you are, regardless of if you're a company that develops software of any kind, if you go to a developer and say, we need some software that does XYZ, the developer is probably going to first step, go to GitHub or go to some other repository, look for some free or open source software that they can use or use a free and open source software framework to make whatever tool that they're going to make. It's a given nowadays that in your company's code there is free software all around it. In the cloud, Linux and free software has completely dominated the cloud. If you look at most of the images that someone's going to deploy in cloud infrastructure, it's all based on some kind of Linux distribution. If you look at the code running on top of that, the services that they're providing, we're talking about Postgres databases, MySQL, things like that. The web servers are generally running free software. Most of the services are running some form of free software or another. Just thoroughly dominated across the board, whatever cloud provider. So much so that Microsoft loves Linux. In fact, on many occasions, Microsoft has professed their love for Linux. Not least of which this last scale, the previous scale, there was a keynote that was all about in lurid detail how deeply Microsoft loves Linux if there were ever any doubt. Very romantic. But why is that? Well, they love Linux because they have realized some of the challenges and that keynote talked about some of the reasons. Traditionally, they would make proprietary software and sell it. That wasn't working so well and so they started pivoting over to services and providing cloud services. So they started reaching out to all the people who had Windows server licenses in their data center. They said, hey, why don't you move over to our cloud and said, okay, cool. That's great for those Windows servers but what about the 90% of my infrastructure that's running Linux? What's your answer for that? They're like, I don't know. We have to fundamentally change our business model or we will die. We have to embrace Linux or the future of our cloud computing platforms at risk. So if we won, then who lost? If there's a winner, there's typically some kind of a loser. So first loser I would say is Microsoft. Now a lot of people sort of I'm not going to belabor this point too much because there's a lot of us who've been around from the slash.era that's this whole deep-seated anti-Microsoft $M sign that stuff. I think a lot of that's sort of a little overblown anyway. For me, but it's still worth saying that Microsoft had to fundamentally change their business model. Embrace free software, embrace Linux to survive. And as a result, they're not selling Windows server licenses in the same sort of way that they were before. They totally changed. If Linux won, then they're one of the losers. Sun Microsystems went out of business and they weren't purchased by Oracle but they weren't purchased by Oracle because of their operating system and their hardware. Pretty much most people would say it's because of Java. Research and motion. So there was a time and most of these are like a highlight of where Bill's working except for Microsoft. So yeah, most of them are Bill's fault. Research and motion, it's hard for some people to realize that there was a particular period of time where every single person was using a smart phone that had a smart phone that wasn't either running Android or iOS. It was running, it was a blackberry. Everybody had a blackberry. The president had a blackberry. Everyone was using blackberries. Everybody loved their blackberries. And it was thought at a particular point it was unfathomable that this monopolistic hold over the phone market could ever change. They just owned the entire thing, start the bottom. But when you start having a free operating system that has this ecosystem around it where people could develop applications on top of it easier than they could for Research and Motion's OS that was proprietary, then they had a problem. So they're one of the companies that lost and ultimately dropped their proprietary OS and ultimately embraced Android themselves. Proprietary Unix lost. For the most part, nowadays you're not talking about spitting up a machine and I'm going to throw HP UX on it. You're probably talking about Linux if you're talking about setting up a server. So most of these companies that were selling proprietary Unix either directly or through their hardware, many of them went out of business at one phase in this process. I'm going to talk more about that in a little bit. And proprietary embedded OSes. So before Linux was de-dossing out of your house, if you were to buy a machine, it would be running some sort of proprietary embedded OS of some kind. So if you're a person that's selling a proprietary embedded OS for an embedded device, now you're competing with a free alternative. So now when someone's building some sort of embedded device, they're almost going to default to assuming it's going to run some form of Linux. So did we really win though? So we're so proud of how we won and everyone sort of talks about, man, Linux is just everywhere, we just did it. But are we sure? Exactly. And if we did win, we're going to argue that we're sort of losing some of that ground that we gained back when we won, which is now quite a while ago. So a good example is Shareware. So when I was a kid, when I wanted to get software, I had a little, I had a a handy computer that ran DOS. And if I wanted to play a game or run some sort of software, I went down to the grocery and they had a little stand that had all of these floppy disks. So a floppy disk was this piece of media that you could put software on. So anyway, it looked kind of like a cassette tape, sort of, but it was really, so a cassette tape was this thing that we would use anyway. So I would go and you would buy, or in some cases they were, it was free. Or what you would do is you pay a minimal amount or free and you would get the software and you would go home and you put the floppy computer and you would run this game or play the software or use the software and it was like the first level was free. And the idea was you could take this floppy and share it with your friends and they could also play the first level of Doom or whatever. But if you like it, then you go and you get the pro version which is you pay for all the levels and you do all this stuff. So free software is everywhere except if you look on your phone the applications there are freezing cost in a sense. So it's all either subsidized, if it's free it's generally not free software, it's free in cost and subsidized by harvesting your data and selling your data. Or they offer a pro version where you pay money for it, still don't see the code but you no longer get an ad or your data is no longer sent harvested by somebody or maybe it is but you just don't see the ad. So we basically have seen a resurgence everywhere like we did back in the day. So your company's code so yeah, you're going to use all these free software frameworks it makes it so much easier to get started in bootstrap when you can go and use someone else's free software to develop your project that you're then going to lock up in your company with a proprietary license and not release to anybody. So we have all of these free software frameworks creating all of these products that are all under proprietary license or never see the light of day. All of the advancements that people make to this code doesn't get re-released in up strings in many cases. So even though free software is everywhere in your company it's not leaving that black hole, right? It's not going back out. OpenCore. So if you're not familiar with OpenCore it's good to start with sort of a description of a traditional life cycle of a software startup in Silicon Valley. So what you would do is you have an idea for a great piece of software you go and get what's called a seed round which is maybe a couple hundred thousand dollars someone will invest this risky investment and you give you a couple hundred thousand dollars that will let you get a small group of people you're living out of a garage or whatever you're all sharing some some overpriced department somewhere and you're hacking on code and the idea is you get this prototype and once you get this prototype that demonstrates your idea in a reasonable amount that kind of works you go back around to everybody and say hey I have a prototype that demonstrates my idea is feasible give me a lot of money and then you get a series A round and sometimes you keep going through those rounds until you either get public or more likely you get acquired by one of the big tech companies in the area you cash out and you do it again and it's that cycle. So knowing that now the idea behind OpenCore typically is you have you're following that business model you have a great idea for a service but the problem when you're in seed round is it's really challenging you don't have any money to hire developers right so what works really well instead is the community helps you for free to develop this software that's free software and once you have that prototype then you go around and you get your series A once you have your series A then you find the best contributors in that community that did the best work you hire them, you bring them in and then you start working on your proprietary add-on module or enterprise version of the free product and so they're working on a proprietary code the community code eventually when you get the series B or further on when it ends up often happening they just take different forms but with OpenCore often what will happen is the developers that are being paid to work on this you have this odd conflict between well I'm adding these features to this proprietary enterprise product if I added them to the free product instead well I couldn't sell them and so you have this conflict where you end up having mostly people working on the proprietary thing because once you get to like series B or C you have to start being profitable and it's hard to be profitable if your developers are working on the thing that doesn't make any money so they start going to that other way so we have all this free software that's out there that ultimately kind of becomes demo mode or like the freeware and the lesson the story is well if you're a hobbyist then free software is for a hobbyist that wants to tinker on one server in their house you need the real deal and all enterprises need proprietary software like hobbyist free software is a toy but the proprietary stuff is where the real intelligence comes in you need to pay for that that's OpenCore the cloud, so awesome so we certainly dominate the cloud have you ever seen the source code for any of these cloud services probably not because they're not publishing that right so it's all proprietary software that's running all of these services that your free software is running on top of I'm going to get into more into this later but if you look at the cloud today it's technically running free software but in many cases I'm going to get into it more but it's being captured behind this layer of API so essentially if you were around during the first time that Linux won what you're starting to feel if you're like me is you start to feel that we're returning to the time before Linux won a lot of these same sort of cycles that are repeating these same sorts of things that are happening that we've all lived through before if we lived through it the first time I'm seeing the same stuff happen again it's essentially the same exact play but we have new actors now all of the people that were on that slide that lost they either went out of business or completely fundamentally changed their model and came in new business but at the time they seemed unstoppable it seemed like none of those companies would ever go away they will always be here in these different businesses we have the same thing going on now where we have players in this new play doing the same exact stuff that those old guys did and there's this assumption but they're so big there's nothing that can ever be done about it so a lot of this talk is going to be about analyzing and looking through what happened back then and then how can we apply those lessons from when Linux won against this same sort of thing back in the day like about three decades ago and how can we apply that to today so raise yourself I'm going to talk about the Unix Wars so during the Unix Wars there's this refrain that went out called the network is the computer you would hear this commonly bandied about among Unix people so for example it was pretty amazing you would have the idea was you had these pretty low-powered workstations or in some cases just terminals and they didn't have a whole lot of resources so what you would do is you would move file storage, computation even full desktop and you put them all on the network in fact I saw a demo in fact Bill showed me this one time I went to visit son and he said yeah I'm visiting this other office and I went to one office and I pulled off my key card and my desktop locked and I went to a dumb terminal in a completely different office because it was all on the network the network was the computer that little dumb terminal wasn't anything this is 30 years ago 20 years ago 20 years ago so what happened during this time before traditionally you would have one giant mainframe that's running a business and you would outsource all of the support and all this really hard stuff maintaining this to a company like say IBM for instance and you would outsource all the IBM IBM would take care of all that stuff they started this migration where people said well instead of just farming all of this knowledge and work over to IBM and then managing this machine we will start getting some large Unix servers in-house and start transferring some of our workloads over to these large Unix servers so some of the players in this were Solaris AIX, HPUX, Erics and all of them had companies behind them that were generally selling hardware that came pre-installed with their version of Unix so these OSs were proprietary but what they had on top of them were open APIs so they had this POSIX standard and what was interesting about this was if you wrote an application for one of them one of these operating systems as long as you conform to this POSIX standard you were reasonably well assured that you could then take that same application and run it on the others which was pretty useful what often happened was you would have this proprietary operating system and put them right on top and so it was really common in fact some of the initial Linux startups got started by by providing support for companies that were running say Solaris but they wanted support for Emacs or they wanted support for some of these other free software tools they put on top they couldn't get that from their vendor and so there was this gap in the market and it was solved by having companies like Red Hat for instance or before that Cygnus and a couple of other companies to provide this support another thing to underscore is all of these companies had amazing engineering teams like some of the top people in the industry were working on building these operating systems building this hardware incredible feature sets incredible things this hardware could do that's just amazing completely side note like Bill also worked there so but the thing about these companies is generally speaking the focus is on hardware features and redundancies so we're talking for example really expensive servers but these servers you could go in and you could hot swap a CPU and RAM while the machine was running running production, performing load it would turn off a CPU, turn off RAM slotted out, slotted a fresh one in and it would be back up and running amazing features think about doing that there's also this assumption that everything from one vendor I'm sure there are exceptions but there weren't a lot of people that say yeah I have a couple Eric's machines over here and I have a couple Solaris machines over here but I have some like indie workstations generally speaking you went to a vendor you got all of your stuff from that vendor it was all the same color it all matched it all more or less worked together so let's get ahead from the Unix wars into the cloud wars so in the cloud wars the common refrain is so you'd have things like file storage computation and even desktops all running on top of the network so this saw this transition where you would have these large local servers in an office or in a local data center and people to get more work out of them in particular in an office back then you would have a development environment if you ran world software in the office and you'd have this problem where every application server needed an actual physical server you'd have this rack full of servers that were idle 90% of the time a developer did one test then it would spin up but the rest of the time it was just consuming ridiculous amount of electricity so people started consolidating them into virtual machines ultimately that led to people taking these small virtual machines and then hosting them in the cloud instead so the players here were talking mostly AWS and Google Cloud Platform and Azure there's others including Oracle's one and there's other people that all went in on this mad cloud money that you can get now so they have all of these cloud providers to have their own proprietary APIs they are published so you can read the documentation at least which is useful if you want to use them but unlike with Unix they're not transferable it's not like deposit standards so if you use for example an AWS API to do a thing you can get an application that uses AWS I can then just immediately do the same thing with GCP or the same thing with Azure there's an assumption that no if you're using one API it only works with that provider you have to write unique code it's not portable across typically you take this proprietary platform and then you create some FOSS tools and you put it on top of there the stack is slightly different but it's typically FOSS tools also all of these companies have incredibly amazing engineering teams, some of the best and brightest are working on building these platforms adding incredible features if you go to any sort of AWS event every single event they're announcing like 50 new services or 50 new advancements and all this crazy stuff amazing engineering teams their feature or their focus is on typically software features and redundancy which you need because things crash all the time and go down and so the focus is well here's how you spread the load out and if that goes down it doesn't matter because you have five other things but instead of hardware redundancy it's kind of like software redundancy in a way there's also this assumption that you get everything from a single vendor people don't say well yeah I have a little bit of Azure and then a little bit of GCP and some AWS over here there's this assumption that you're probably going to go all in and in fact there's a lot of people that probably even in this conference that will advise you against splitting the load or being cloud agnostic there's a lot of pools there and you should really just lock yourself into one vendor and go alright so let's go back in time again so now we have the unit tours going in full steam but now we're getting into the dot com boom and so the dot com boom could be summarized by taking a business model and then putting it on the internet so if you had a night if you wanted to make a lot of money then what you would do is you would look at a business around you you walk down the street in like a grocery store but on the internet like rolls of money right just go do it go do it you're like bookstore but on the internet like genius we'll put a bookstore on the internet all this money I mean of course it would never work but so all these businesses all you have to do is put a business on the internet and then the rules didn't apply the normal rules of business like making money and things didn't matter anymore because it was on the internet but the thing is if you think about it and you say I'm making a business but it's going to be on the internet what that means is there's this big push for web servers because you have to be on the internet and that meant having at the very least a web presence and generally speaking a whole application stack on the internet so web servers are going like crazy lots and lots of servers in general there's massive in the bay area there's massive data center growth I mean you couldn't buy up real estate fast enough and it was really hard at certain points to get your server racked in the new data center they're like no we're all booked I'm sorry you can't do anything incredible data center growth in general at this early on in this Sun and Solaris dominated the server market during the dot-com boom if you were going to set up a web and application server stack in a database stack you probably were going to do it on Sun and Solaris and like the stack looked a little bit something like Sun hardware maybe running NSCA or maybe Netscape web server if you had some extra money you would probably have C or Java middleware somewhere in there and then you probably ran it on with the Oracle database in the back end typically when people wrote software for this they were writing the software for this stack on proprietary workstations of some kind typically like maybe some sort of if they were Sun Shop maybe a Sun proprietary workstation of some kind or something like that so there's this great phrase we are the dot-com that Sun used in their marketing now they weren't talking about web servers really at that point but they were actually referring to the fact that they had a DNS server that was managing com but we'll talk about that some other time because I love DNS, it's my favorite protocol so let's skip ahead to dot-com 2.0 I don't like tattoos but if I would get one it would be like DNS or anyway so 53 so anyway dot-com boom 2.0 so the idea about dot-com boom 2.0 was take a business model and you put it in the cloud and you might be saying that like putting it on the internet and you're right but a lot of those people that said this weren't around back then and so it seems like a novel idea that to just take the same thing and put it in the cloud and so if you say that back at the beginning of the dot-com boom 2.0 you could get a lot of venture capital money by saying we're just going to take an existing business model like a grocery but the grocery is in the cloud now so what did that mean that meant massive push for all of these new web services so people were having to develop web services that would run in the cloud now the other thing that drove this demand was the fact that you had these really low power computers that everyone had in their pockets now that wanted to do things and so they couldn't do it on the devices easily as they could if you had these web services sitting out that they could talk to do some computation and come back because again the network's the computer right lots of VMs everyone's just VMs all over the place there are cloud builds going up it's no big deal and then you get your bill at the end of the month like whoa that's kind of crazy it adds up doesn't it so at this point you started seeing huge growth in cloud infrastructure companies and if you had a company that wasn't a cloud infrastructure company but had a lot of servers just to do whatever else they did they're like hmm I wonder if we could get in on some of that if we just spun up a couple more servers could you just spin up another 200,000 servers and we can sell some of that so it's like an Airbnb for servers and so it became really popular and we've been working on this so what you saw then was a migration off of traditional data centers this was made popular by companies like Netflix in particular who really trumpeted the fact that they were dealing with massive disk growth and sand and NAS were incredibly expensive and so they could save a whole lot of money by not having to worry about that storage growth and moving into the cloud instead of making AVUS worry about buying disks all day long AWS generally dominated .com boom 2.0 AWS was playing catch up at this point because they were already established they already had infrastructure everyone else was coming after that there's a lot of money to be made here we need to catch up so what you would normally see is some sort of Linux VM would be running Apache or IngenX you probably have some sort of Ruby Python Java or JavaScript middleware somewhere and then the databases you probably might screw up Postgres and maybe some sort of NoSQL database and generally speaking you developed this time instead of Solaris Workstation it would be a Mac usually every now and then someone would dabble on windows but for the most part people are providing Linux software on a Mac to go into Linux server and in this case what you would see is you'd see all of these common FOSS services what would happen is the provider would say we see a whole lot of people that are spinning up a server that's running this particular FOSS service why don't we just go ahead and roll that into our own service so you don't have to worry your pretty little head about setting up a database we'll do it for you and you can just kind of go through us and we're the experts anyway so just use us for that and so a lot of this and so you'd put an API in front of it and then they could just use your product, your FOSS service so now let's go back and time a little bit more so .com boom we're getting toward the middle of the .com boom for the most part Linux was dismissed as a toy if you talk to proprietary Unixes if you were a system at the time and said well you know I kind of thought about maybe putting a Linux web server in here or something like that or maybe running a database on Linux like what are you talking about this is a toy for hobbyists no you want the enterprise edition you want the pro version which is proprietary Unix you need a real server you can't put this little toy OS and expect it to survive like a production workload right that was until Apache came out Apache came out and it put and it was this web server that put Linux and FOSS in pretty much every single data center as again we're having this massive growth of websites right it had this killer feature which was the fact that they could do virtual hosts so before this point if you had a website you would buy a web server if you needed another website you would buy a second web server and if you had a third one you had to buy a third piece of hardware that's your third web server Apache was revolutionary in that you could run three web sites off of a single piece of hardware and people are like well that's really great I'm tired of racking and stacking all of these web servers all day long pretty cool so they started doing that more importantly it was cross platform and Linux was cross platform which meant that you could take Linux and Apache and instead of putting in on a really expensive piece of proprietary Unix hardware you could put it on x86 commodity hardware and it would work really well so then what you started seeing once people got over the concern about Linux being a toy to be a web server then they started looking into some of the other things and it became such a common combination of tools that it got a name called the lamp stack so this was Linux with Apache in the front end MySQL hosting the database and Perl I know some of you are saying what about PHP it was Perl doing the middleware later on that got replaced with PHP and maybe people started naming everything P's just so they could say it was a lamp stack but for the most part it was Perl and that was essentially what you did and it started replacing the traditional stacks where you would have some sort of Java middleware going into an Oracle database on hardware there's also this really innovative feature that they came up with at the time where you could take if you didn't want to go through the trouble of writing a complete application middleware stack what you could do is you could write a little snippet of Perl code or PHP code and you could upload it to the server that was hosting this service called CGI or common gateway interface and what that would let you do is you could hit a URL endpoint it would resolve that input and data would go into this little snippet of code it would run whatever you told to run run this little process take those inputs do some stuff maybe even talk to a data source somewhere get some sort of result and then spit it right back out back out to the client that requested it and it would do it on demand so the code wasn't running all the time it only executed when you hit that endpoint innovative stuff it was what was really it became so popular that websites everywhere including most people's ISPs you know people would say if you use this as an ISP you would get a homepage too and that's really exciting for people and they say oh and you want to do the CGI thing well cool you can just FTP up a little snippet of code in this directory and boom you hit this endpoint it will execute run but the cool thing was you didn't just have to use your ISP you could take that same snippet of code it was portable you put that snippet of code on your own server you put that snippet of code on a professional like not cloud at the time a web server that does this at scale you could put it wherever you wanted and it would run the same all those places pretty amazing and innovative when people ended up facing with the choice between well I have this database and if I render it on Oracle it can never go down there are where these database servers and data centers that must never be upgraded must never be touched more importantly they never can crash if they crash we're done but that's okay because we would spend a couple hundred grand on them so we could hotswap CPUs we could hotswap all the hardware and it was cool it didn't matter but people were starting to be faced with the choice of well I could do that and that works or I could just buy like three five thousand dollar servers running MySQL or Postgres and if one of those dies I have two more and that's fine it's way cheaper and it's as redundant so people started facing that choice well if you had that choice a lot of people would opt for the way way cheaper solution that they could scale out easier and they did so this started the trend that ultimately killed and replaced proprietary units with Linux because it was hard to justify the amazing as the hardware was it was hard to justify those costs when you could run commodity hardware with free software get similar results and have similar redundancy performance guarantees and similar scalability so at this point Linux and FOS wins the unit so let's scoot ahead to the modern age so some of you have probably heard of the technology called serverless so the easiest way for me to explain serverless to you is it's a proprietary it's proprietary CGI so the idea is you take the same exact thing that CGI did which was a lie to write a little snippet of code that executes on-demand when you hit an endpoint but make it so that it only runs on a particular cloud provider so this way I can upload serverless or CGI code to AWS but it only runs on AWS or it only runs on GCP or whoever does it so it's not portable between the platforms this phenomenon of serverless put this further separation between the developer and Linux or FOS services so traditionally even if the developer wasn't a sysadmin they would at least be interacting with some sort of Linux services because they're writing some middleware layers some sort of Rails application or something but if it's serverless all you really have to know is I write this snippet of code that interacts with this API I upload it to this service it doesn't really matter what OS is behind the scenes it's not using any of that stuff it's all it's doing is executing this code and then coming back to me with results so the ultimate goal with systems like this is to only operate with API you're not really operating with free software at all or servers at all you're simply just operating talking from one API to the other so what that means is well among other things it's harder to leave the platform because like I said this code is importable so if you're doing serverless then if you wanted to move to another platform you have to rewrite all of your code it's not like CGI where you can just pick it up and move it to someone who has a better price well the other question is is there Linux and FOS behind these serverless APIs maybe probably how would you know who knows it could be completely proprietary software I don't know so this is the title of the talk the cloud is just another sun so what does this all mean the cloud is the largest proprietary OS on the planet um a lot of people don't think of it that way because they think yeah but it's everything's all distributed across all these things but if you think about how you interact with the cloud if you do it the way that you're told that you're supposed to use the cloud you interact through some sort of web panel or some sort of application on your machine and you just talk to these through these proprietary APIs you use this thing some you could think of a server a node there's a process that's running on this proprietary operating system it spins up it does some stuff that may crash but you don't care because you could spawn multiple processes as a word and it doesn't matter but all the code is proprietary you don't know what's running this cloud you can't see what's doing all of that stuff they don't publish the code it's all proprietary um but it's running free software on top so like with Solaris but people don't think of it like Solaris right they think well but with Solaris you ran free software but you could run free software on top it's no different so as a result if you're a dev or a system then these days if you're just starting out these days back in the day you really have to immerse yourself in all of these technologies to use them and nowadays if you're a developer and admin you get very minimal exposure because of this to Linux or FOS you're not really you don't really have to know almost anything about that software to use these cloud services in fact that's sort of the goal is you just interact with these APIs instead so what does that mean that means a modern dev and admin both essentially have the same job which is they write the API glue code that glues different APIs together with their company's business logic now you maybe think oh well they have the same job well basically they do um they may be talking to different APIs so if you are a sys admin who is currently doing this you may think about the fact that well um can I write glue code as well as a developer who's doing it all day long hopefully you can if you can't and you want to continue to be in this industry doing the same stuff I highly recommend you up your coding game it's not just bash scripts anymore you really need to pick a couple programming languages and get really good at it if you want to be employable the other thing that's really interesting is that these days vendor locking is kind of accepted and expected you'll talk to a lot of people and you'll say yeah but you're locked into the platform isn't that bothering like well what's the big deal it's why would I ever want to leave this cozy little thing where everything just works um what's the problem with being locked into a vendor I mean there was at one point where you could use vendor lock-in as a dirty word and you could say well Microsoft wants to lock you in it's vendor lock-in and they be like oh well and they will be arguing with you about why it's not vendor lock-in not arguing why it matters that it's vendor lock-in but nowadays if you talk to a lot of people they say well what's the big deal who cares that I'm locked into the vendor but the thing is is your cloud too big to fail if you've used the cloud provider all of them have had outages of different severity some incredibly massive outages in all those cases if you start downtime somehow it was always your fault because there was some redundancy that you didn't know about yet that you're supposed to be using which usually means duplicating your workloads or efforts or spending some extra money on the platform so that when they had a failure it didn't affect you because it was your fault shame on you but the thing is people just sort of have this assumption that well nothing will ever really go down or if it does so many other people will be having a problem that who cares about me they won't care about my website being down because the whole internet will be down but that raises another question what happens if one cloud provider wins though right now we have some competition in the market but let's fast forward a couple of years and say that one of them just dominates and destroys the rest and the others just can't and so you basically have this platform where most of the internet is running on one cloud provider's platform only so let's skip ahead and think about that for a second what would that mean if everyone's used everyone on the internet that has a web service is more or less probably using that cloud platform if that happened would protocols need to stay open anymore like right now we take advantage of the fact that the internet typically has open protocols if I want to talk to a web server I can use the HTTP I can talk to your web server with HTTP and I can talk to someone else's web server using HTTP I don't have to care about that but it's an open standard there's a lot of open standards that underpin the internet if one cloud provider owns the entire internet or most of it let's say like 90% would those protocols have to stay open anymore I mean how I could see it playing out is you could have them saying well but that's kind of slow doing it that way we have this internal protocol that we could use instead and just inside our network we start by replacing it inside our network and it'll be way faster because it's optimized for just our network and then that works really well and then they say well now just to talk to if you want to talk to any of these hosts it's way faster so just use that so use our proprietary protocol instead there's no guarantees that protocols will be open forever especially if one cloud provider dominates so let's skip even further ahead and say let's say that they say well you know it's great that we have all of these everyone feels better about using these they feel less dirty because they're like yeah but behind the scenes is probably running Linux there's free software behind the scenes but is it and would it have to be and if they decided you know what we're going to replace this free software tool the API is going to be the same we're going to rip out this free software tool and replace it with our own proprietary code would anyone notice some of us would care but I bet a lot of people wouldn't notice because how could you know the difference okay so that's so then what do we do though with all of this information well let's look back into why Linux and FOS went the first time it started with complete dominance leading to some complacency and some arrogance remember dismissing Linux as a toy as a result innovation in certain areas slowed in particular for the UI you would see these instances where you would plug in a modern workstation you would connect to a sun machine you type a mistake and you have this weird error and I know everyone is saying well all you have to do is add this little extra thing and you add this extra scdy command and it would the backspace key would work on your keyboard again that's not the point the point is there's this arrogance that led to say well it would work if you were using a sun workstation with a sun keyboard why are you using anything other than from us it would work out of the box fine so why would we have to change it a lot of UI things were like that there were a lot of command line commands maintained all of the old command line arguments there were all these advances like in TAR believe it or not that their TAR sucked it was awful and GNU TAR was awesome and had all these great new features I mean like right now did you know that you don't have to say or you don't have to say Z anymore or you don't have to specify which GZIP or BZIP or whatever you can just say untar this thing and it detects it now you don't have to specify that anymore so it felt like that was happening and everyone was using but they're like well but no we want to maintain consistency and you should just be doing it our way anyway it's the right way so what that did was because even though all the sun engineers were super smart there's only so many that sound good hire there's more FOSS developers in the world we outnumber all of these and this is true today we outnumber all of these smart people that are working at one company however smart they are it doesn't really matter we outnumber them because there's more of us the Linux and FOSS innovations outpaced all the innovations ultimately that sun could provide on their platform that led them eventually along with the problem of cross-platform compatibility it made this hardware a commodity so what they had to do is they ultimately were forced to half-heartedly embrace the x86 platform it sort of worked and they were forced to kind of open their software and that sort of worked but it was too late by then by then Linux had already dominated okay but that's nice but what do we do then with that information what do we do now well first recognize proprietary code when you see it or it's probably better to say recognize it when you don't see it if you're using a service you're running a server you're doing something like that ask yourself okay where's the code behind the service if you don't see it that's proprietary code you know that's the definition of that so look for that code recognize it when you're using proprietary software resist proprietary services so I'm talking about things where you're using some sort of API that you don't control it's not an open standard someone's just documenting it for you to use it but there's no sort of it's not public or shared across any other providers it's just for that one vendor even though it's super easy it's way easier by the way and this is always the story with lock-in it was always easier if you just did everything through the vendor if you only bought all the things from this one vendor okay it was no problem but a lot of people nowadays don't really see those problems because we're not at the phase yet where people are exercising that vendor lock-in to do negative things yet for the most part it's like well it just makes everything easier we're not in that phase of development we're now that they have dominance they're now leveraging that lock-in to increase some prices a little bit and they don't have to maybe I don't have to be as price competitive anymore for certain things because you have to use me and stuff like that is only starting to play out and so a lot of people don't really care because they haven't seen the downside to lock-in which is once you have leverage you can use that leverage to hurt the customer the thing is our openness and our compatibility is our advantage to a proprietary company their proprietary nature is their advantage you can't see our code, we can innovate and we can do all this stuff and you can't see it but to us the fact that we are in to a platform and using a protocol that only we have is their advantage because it means you can't move off to another platform the fact that we are open with our standards and the fact that we maintain compatibility across platforms is an advantage that we cannot lose this is one of our primary advantages because that's one that they don't have and can't have because if they did that then they would say well customers could just move across all these platforms they can move away from us we can't lock them in that way we need to take advantage of our openness and compatibility as much as possible so this means if you have an innovation share it all of us will come up with these great ideas for these services or the software, all of this stuff please make it free software publish it for other people to use and share resist the urge to get some VC to give you a bunch of money and turn that ultimately into some lock down proprietary enterprise version and abandon the open core version resist that urge, share this stuff to people who have great ideas to actually publish them nowadays so I have this sort of thought you know what is the next plan but it makes me think of would lamp even exist nowadays if someone came up with this great idea for this free software stack that did all of these components would it be free software these days probably not what would happen if someone said well that's a great idea and I will have like the hobbyist version of MySQL that you could run on one node with one CPU but if you want like encryption enterprise feature if you want to have encrypted network transport or whatever so you need to add the proprietary module and pay for that and all that stuff like will we ever have a lamp stack today in the modern day with our approach to proprietary software I don't know but then that raises the question well since lamp dominated it was one of the reasons that Linux and FOS one what's the next lamp then what's the next thing that's going to save us from this otherwise pretty negative future if things continue the way that they are some of us have some answers for that come hit me up to the purism booth ask some ideas for that maybe at some point and I'm also really curious to hear some of your thoughts maybe not in like the Q&A but maybe afterwards when we're talking in the hallways if you have some ideas about that thank you alright so any questions questions down front sure so the question the comment was more like was a reply to the question what's the next lamp and the reply was what about open stack because open stack if you're saying what's the free software sort of counterpart to cloud infrastructure what about open stack and if you're thinking of the next lamp as the ability for someone in their own data center to recreate some of the experiences of cloud services open stack is that yeah yeah so but my take on it is more that well that's great that that's open a lot of people instead of going to the cloud of the proprietary providers because they didn't want to set that up themselves anymore like they're to them less expensive to outsource the knowledge and know how to do that to another provider and in the case of providers like Rackspace and others who are embracing the open standards so far at least that openness I don't know how much that they tout that openness in the marketing as an advantage versus just saying well we use this thing so for some people from the outside it could be seen as oh you're just using something that you didn't have to develop in-house to build your cloud that's great I guess but if you're the outside person not everyone necessarily sees that as a feature or like I would see someone using an open platform as a feature but a lot of people just see well how much is it per hour or whatever right so that's the first time I gave this talk the first comments I got was what about open stack though it's not the next plant in the back yeah so the comment and question was about the fact that if you want to have some sort of vendor agnostic approach or cloud agnostic approach there are a lot of vendors out there that are more than happy to have you lock into them instead of any individual cloud provider and so the idea is if you use my tool and I will support it for you I can employ your stuff I can give you a common language to speak and then you can use any proprietary cloud you want and through me and I will perform an abstraction layer against it there's some free software versions of that too I believe that will allow you that you don't necessarily have to be locked into a particular vendor but it provides an abstraction layer is that kind of the thing you're talking about no what are you talking about instead yeah and that's the problem it's a similar problem with messaging apps so I have an open protocol messaging app but I also have five messaging apps on my phone because everyone has to have their proprietary network and no one wants to collaborate anymore so it's a further sign of the state that we're currently in where everyone is so focused on lock-in at this point that they are all abrasive against any sort of open standards and all customers it requires them to compete more than just on price they have to compete on other things because the person can move easily other questions yeah so the kind of thing I'm kind of looking for I suppose and I'll wrap up here and take the rest in the hallway is there's a couple ways to tackle the problem of why do people go to cloud services for these things and to me the biggest sticking point of lock-in is the fact that they have they make it easy for you to use they host the server so they run it for you and they have all the tools in particular all the tools that allow you to manage those things in scale and there's free software orchestration tools there's also all of these companies on top of that that are providing all those other stuff and a lot of people just sort of scratch their head because they would have to hire an extra DevOps person or something to manage that to outsource it and have someone else do it to me what I would like to see is alternatives to those services that are that intentionally are designed to be portable so imagine using having a database service or having some other like an email service that kind of thing messaging service where you can transfer from one vendor to the other because all of the not just how the software underneath the free software and then there's this layer that layer that you interact with is also published maybe even how it's orchestrated how it's deployed is published because a lot of it's funny like a lot of developers will view system and work as challenging and system and will often view developing work as challenging but if you were to ask a developer they would say well yeah we'll publish the code but we should make all of the deployment stuff but if you're a DevOps person you say no no deploying the software maybe you're intentionally now going to make the software hard to deploy so you can sell the orchestration but honestly deploying software that's easy anyone can spend a couple of minutes and do that so why not publish how I deploy not just the fact that I have my QL over here or whatever so the question had to do with I talked about the Ericsson hubris was one of the problems with Sun that led to its downfall and talk and so the comment was what about the fact that we won let's say a decade ago and our as a community Ericsson hubris lead partially to this and yet to a degree in particular what it led to was a complacency because after you've won you're like well now I won and so now we can all take advantage of the fruits of all this free software everywhere and what it led to is that's the reason that people don't care as much about vendor lock-in and proprietary things right now because a lot of people that got into the industry now got into the industry during this golden era where you could count on a protocol to be open you could count on the standard to be open you could count on all these things the software to be available all of this stuff and so it read this complacency where everyone just took for granted the fact that it's all going to be open and that allowed a lot of companies to talk the talk as far as this sort of thing and then behind the scenes do all the stuff that a lot of people forgot to look for because they're like well yeah but they're saying the right words they're saying things like open source or free software and open standards or whatever and it's okay that like you know 50% of it over here is in a box somewhere I didn't see it but they're saying the right things over here that's fine look they've published the code for the thing that they don't care about they just only don't publish the code for the thing that makes the money that they care about and so we've because we won yeah I would agree that like our hubris at least our complacency because we won allowed us to allow that stuff to slide in again and so that's why some of us like this is one of the reasons I'm giving this talk is to remind people about the lock-in that we've had before why it was the problem so we can start seeing it again and starting resisting it I'll take the rest in the hallway but thank you for coming I don't know if this is... oh there you go okay hello so welcome to manufacturing the open desktop computer I work at system 76 and we are a linux computer manufacturer here in the US and we are working on building an open source desktop computer this is salio we have three different models so this is ranging from our compact machine all the way up to our massive and our massive can fit up to 4GP so there's a pretty wide spectrum there as far as customization let me go into detail in this presentation about manufacturing the machine but before we get there I'll give you a little more insight into who system 76 is why we're building an open source computer look at salio's open hardware and then we will peer into manufacturing step by step so system 76 is a linux computer manufacturer powered by a close working team of passionate open source enthusiasts our entire customer base is a linux user and we've been serving the linux community for over 14 years and we believe the computer and operating system are the most powerful tools ever created so this is the tool that people are using for a health scientist to identify a cancer cell tissue or for a roboticist to program the robot or for a rocket scientist to calculate space trajectory so it's extremely important that we make sure that these tools are powerful and efficient so that these innovators have a tool of hands so why open source why is it important that we are building an open source machine what is our goal and what is our hope I want you to imagine a future where open source software and open source hardware exceed the expectations of those who once only trusted proprietary and in this future you are going to your friends and neighbors or somebody local to provide you with tools or products that you use every day so say for example 3D printing perhaps you have a friend down the road that 3D prints the best sneakers they are the most conformed to your feet you like the style, you think they have a great character on them but maybe you have a couple of friends that 3D print sneakers it's not just 3D printing perhaps laser cutting maybe they laser cut really awesome customized lamps and they are perfect for your needs maybe we are getting to the point where you know someone locally who makes carburetors and you have a couple options so this is the future that we are hoping to get towards and we are hoping that by building an open source computer that it will start there because we are trying to encourage people to get to this point where they feel empowered to be creators of their environment rather than accepting things as they are and there is an industry that does this so I am going to take a poll and hopefully it works out otherwise I will just pretend like it does so imagine that you are on your way home from the conference then you want to stop and get yourself a 6 pack of beer I want you to raise your hand if the beer you are going to pick up is some kind of craft beer loggaditas, dogfish head great and I want you to raise your hand if the 6 pack of beer you are going to pick up is something like Budweiser perfect that went well the craft beer industry I think does a really good job of having you prefer something that is made by somebody local or somebody that you know even versus getting something that is mass produced and I am going to talk to you about a couple examples of how craft beer relates to the open source industry so let's take a closer look in comparing craft beer this is a perfect example of loving craft beer to mass produced beer there is a big difference just even in the photos so when you compare these two different models something that comes to mind is the quality of the ingredients, the love and attention that goes into making the beer the flavors, the variety the options but there is something else that is really special too so one thing that really stands out is when you are a craft brewer and you are making your beer there is a level of dialogue or conversation that you are having with your customers let's say you have a brand new recipe and you are trying it out and you can't wait to bring it into your brewery and to see how people respond to it you are overhearing their conversations do they like the style, do they like the flavor is it too hoppy, do I need to make some adjustments if you are having a conversation with them and then you are making those adjustments right away so when you think about this future where open source dominates and the expectations of open hardware and software exceed the expectations of proprietary and the projector doesn't do that the hope is that people will have the same reaction that you all had to Budweiser with anything that is mass produced because the preference will be the same in that they prefer something that is made by someone local so how do we get there well we believe that the most important way to get there is to encourage people and make them feel comfortable enough to pursue their curiosity and there is a research study that talks about the personality trait of curiosity and how important that is and in the start of the research study there was a person who was talking about the Wright brothers a friend of theirs, the pioneers of the airplane said that he believes that the Wright brothers will forever be a prime example of how far a person can get in their life without having any special advantages but the Wright brothers disagreed with their friend and they said the greatest thing in our favor was growing up in a family where there was always encouragement to pursue our curiosity so when you think about this future where friends and neighbors are having curiosity to explore things to try different things to become makers this is the future that we're hoping to work towards this is the future where people feel empowered they feel like they have options they have variety but they feel curious enough to take bits and pieces of things that they like and make the tools better for the things that they use every day and this is exactly what Linux is in fact everybody here was curious enough to not accept things as they are and to find tools that were better for their own uses they knew that there isn't just Mac or not Windows, there's got to be something better I need something more powerful for what I need to optimize my own workflow and there's got to be something better and because it's open source you can pick and choose bits and pieces that will optimize your own workflow so I spend a lot of time in our social media channel that's System76 and I gathered a couple of conversations before I was putting the slide together to just look at what kind of freedom we really get to have as a Linux user I'm going to read off a couple of these for you that works okay so upgrading from Zorin to Ubuntu looking for that pure control I originally only used Zorin because of the GUI but then learned I could just theme a Ubuntu and now I realize that Ubuntu is better imagine taking Linux Mint placing the Cinnamon desktop on it and then theming it to not only serve as a perfect drop-in replacement for Windows but be one of the most beautiful Linux desktops you've seen in a while that's what Farron has managed to do did that one work? okay the next slide didn't work but this one says I've been using elementary as my daily driver and I've been using Arch as my work driver these are both great and I have two different options so I think those are some really good examples of how much we really get to choose and customize the kind of work that we do and make it best for our own tools I thought it would be fun to look at the conversations and proprietary operating systems do I have to upgrade? dang it, it's not showing up let me see if I can fix that let's do this earlier too ready, let's see there you go it's changing so the conversations there aren't very customizable or aren't very fun so this is about Linux freedom and it's freedom because you have the choices because you're able to customize your environment we were inspired by how far open software and especially Linux has gotten being able to empower its users and we wanted to make sure that we wanted to get to a point where we felt like people could have the same option with open hardware that they have with operating systems and so that's why we started looking into developing our POP OS and then working on Thalio we wanted to make sure that we had computers that were powerful and of high quality and we want the open commuter to be chosen not just because it's open but because it really does do a good job of making people's lives better and making things more efficient so because we're wanting to make sure that these computers are powerful and serve our customers needs and a lot of our customers are using Linux because let's say they're in machine learning or they're developing or the neural networks so we want to make sure that it does really well so for us the engineering challenge was to design the open computer to run components at their maximum performance constantly without throttling by replacing the embedded controllers with free and open source firmware to control functionalities like thermals inside the computer and this here is one of our daughter boards we actually have two daughter boards so the daughter boards this is our open source solution to pulling proprietary functionality off the mother board and we're doing this piece by piece until we have the entire computer open source because we can't just go completely open and have a lot of customers who are depending on Linux to fill their needs so we are starting with a daughter board this controls SATA power and fan control and then we're working on pulling more and more onto the boards we have a couple of these that are boosted too so if you want to take a closer look at them I've definitely stopped by so we found out a couple of interesting things in thermal testing and so R&D prototyping was one of the fun parts we started that last year or probably a year and a half ago or so this is Ian who is explaining some of the cooling tests that they ran for R&D to make sure the system is going to stay cool and have a really good air flow and we found that everything matters in the cooling system design so the size and the shape of the ventilation holes the distance of the fan from the heat exchanger the wall of the computer made a really big difference and the amount of cool air and taken and of course the path of the air flow throughout the computer and the path of the air flow through the computer was really interesting to test we used acrylic prototyping to start out and so that because we were able to cut out extremely fast and put things together to test different theories acrylic prototype really helped with that so what we did is we created a couple of different shapes and sizes we created a different a few different methods to figure out the CPUs and the GPUs and where the fan should be ran a couple of cooling tests to see how the air was flowing through and how we could keep the CPU cool and then we moved from acrylic to metal last year so we tested different methods of CPU cooling comparing liquid cooling open air fans and heat pipes and we found that heat pipes were the most effective this kept the machine the most quiet out of all of them and kept it the coolest out of all the different methods this is the fun part manufacturing the machine so we moved manufacturing to our office in November we finally found a space, was it November? I think it was November and then that's when we started bringing all the manufacturing equipment we finally had R&D figured out we figured out the layout that we liked in the configurations so this right here is raw sheet metal they're probably about half of the stage and they arrive to our office in large packs like this we take the raw sheet metal and we move it over to this gigantic laser here which cuts out where all the I.O. goes all the I.O. is and then we move it over to this piece here where we are able to bend it this is Chris Fisher from Jupiter Broadcasting who's helping us bend the chassis and cut out this is what it looks like these are our naked failures and then we move it over here where this is diphthenacid and there's a second bucket to the right of that where it's rinsed and then we move it over to the oven and this is where it's dried now once it's heat or hot it's the heat that attracts all the powder coating to it so that the color stays so we make sure we get it hot enough and that's after it's hot enough we move it over to this powder coating area and then it runs through the oven and it reaches about 400 degrees that's when the powder starts to float and flow and then it cures and hardens and then it's ready for the next step which is then run through a laser again so that we can start to etch in the designs so here we have the mountains to represent the Colorado Rockies and then failure and we put little tiny touches throughout the machine so there's some Morris code at the bottom which I will let you guys decode at your your leisure Denver Denver and then so each of the machines are put together by hand one by one they're put together by hand and then checked so this is a real they're made out of real wood there's the option between the walnut and in fact we partnered with the National Forest Foundation so some interesting things we found with the wood pieces and we didn't know this until we started having to manufacture multiple failures but we found out that the tree color changes depending on the season so we really wanted the contrast of the light and dark wood and the walnut is darker and then lighter in the winter and we found that out after the fact so now we know we need to bulk order the dark walnut so that we can keep the dark color we had an event in December and we brought some press outs that come in toward the factory so that's West taking photos of putting the machines together and that's what the chassis looks like when it's completely put together so like I said there's a couple special touches that we have on the machines I'll let you stop by our booth to look at all the details but I'll call out this one because it's one of our favorite features this is a photo of our rear exhaust which you see has planets on it and this is where the planets were aligned at the start of the UNIX epoch and one of our engineers put together an application where you can enter in any date and it'll show you where the planets were on that date one of the things we're looking forward to getting is for people to send us their own failure designs and so we're hoping to get to a point where we can partner with a couple of other vendors so that we can make designing our own failure at home more approachable and accessible we don't know what that looks like yet but once we smooth out manufacturing that's one of our next steps is to figure out how to get people to feel comfortable and being able to produce some of their own chassis so if you wanted to try the application and put in your date or your pet's birth date that could be something fun to explore which brings me to one last beer story because we're talking about people being able to produce their own desktops themselves one of the questions we get is the fear for people stealing your idea or stealing your I don't know your thoughts I guess my husband is a crash brewer and he has he has one beer of his that he really loves and he takes it to a brewery every time we go out so it's a Neapolitan stout and when you drink it you taste the chocolate right away and then you taste the strawberry and then you get an aftertaste of creamy vanilla and I don't know how he does it but you taste it in all the different layers it's crazy but it's really good and whenever he's out of beer I'm always so sad because then we're left with whatever options are at the store but anyway every time we go to a brewery he brings a six pack of his beer to show them and one time I kind of got frustrated and I said stop doing that they're going to steal your beer idea aren't you concerned about that he said no he thought that was weird that I was concerned about that he said so what if they do he said he said you can find any recipe for anything online anyway and he said part of the fun with brewing is that you can reverse engineer the beer to see if you can make it better or see if you can make it taste the same it's part of the process and he brought up another good point in that if somebody happened to take that beer we just have another place to go to when we run out of beer and he said whether it's at a brewery or whether it's at a liquor store more options are always better and having it available more places is always better so I thought that was a really interesting perspective and I think that's another nice tie-in to open source and reaching the future that we all talk about a future where we're encouraging people to have more options to appreciate the options that we have in Linux and I think that about sums up no we dove deep into manufacturing but I don't know we're excited yeah so the the daughter board is open that's available on our get-ahead page and one of the as far as the chassis and the process we're wanting to open that up one of the other things we're hoping to do is to put together some kind of instructable page something similar to that where we say here's the process we've gone through here's some of the R&D tests that we've run what are some of your suggestions so we want to open up all of that so that we can continue to get more feedback there's there's a proprietary components on there right now because we're still like I said a lot of our customers are in AI and they need they're coming to us not because of the open hardware but because they need a reliable powerful computer so right now we have proprietary components on there and the daughter board function is to start pulling those pieces off bit by bit not for the graphics chip quite yet before so I was talking to the engineer about this and he said before we can get to any of those pieces we need to figure out how to get to the motherboard first the desktop was a good starting point for us to figure out manufacturing right now they're working on the laptop motherboard they're at the very early stages of that but he says once we get that part of it that's really the key to start blending into these other areas I don't know if we would I don't know if we'd create a standard layout I mean we're we're being as open as we can because we don't know the truth and what's really going to work so we're we're testing and iterating and you know opening up conversations to get feedback that makes sense yeah that makes sense I think those are some of the things too that we test things and we say hey here's some of the things that we've tested we can make it a suggested standard thing here's a test that we ran here's a result that we found and so based on all these all these things we've discovered this is a recommended standard but I mean if someone says hey I've actually tested and found this out then we're definitely going to listen yeah that's a good question I don't know if if we do or not Emma do you know I don't I haven't heard of that we so the whole goal was to make this feel approachable and friendly to match our brand and we felt like wood was very warm instead of having the cold metal and friendly yeah that's a good question right now people have been just sending suggestions based on their own theories and so right now they haven't been put into practice to really try that out but once we get there I'm curious as well to know how we're really going to work that out I don't know yet yeah so since we haven't gotten to that part yet I don't I mean officially I don't know I imagine that they'll put all the details in there with saying what they have tested on and what they're looking for and then soliciting feedback that way I know that we've tried it at some point I didn't hear the results of those tests I think they're waiting for it to to get to a point where they feel like it's something that we can recommend to our customers that sounds awesome if we can get a whole community behind it I wonder if there's a way I'm just theorizing now but it would be really cool to have some kind of lab testing area where we can say hey come in try this out here you know have a bunch of failure systems out a bunch of different types of hardware components and then have people kind of like a lab testing center where people can try out different things yeah yeah that's it's pretty scary and the other thing too we're hoping that if we can create enough buzz or weight around the launch of failure maybe that'll help swing some of these swing more people into understanding why open source is important and the other great thing too is really owning your machine because when you're buying something proprietary how much do you really own so right now we still have our regular customer base of people that are neural networks or AI developers, bioinformatics and then customer base because we're building high power machines and failure is still the same but that's why it's really our personal march to build an open source computer while some people are using Linux because it's in line with their field so we have both of those markets to respond to okay yes yeah that's what he was asking as well sorry I should repeat the questions yeah that's so it's something we tested at one point and we I don't know the results of that right now we're just waiting for we're waiting to hear updates until we feel comfortable providing it to customers cool if you guys have any other questions come by our booth I actually if you guys took some photos of the presentation or any of slides we're giving out some coasters of the daughter board so these are these are really cool so stop the booth if you want one thank you yeah and work linking the things together f prime provides a framework for auto coding this helps you generate a lot of that boilerplate code that the languages we implemented embedded systems require it also helps you generate command telemetry and events those are standard practice for flight software how you get data up how you command the thing and how you get log messages back down we'll cover more of that later and it comes with a lot of standard components it helps you build your system by providing a library of standard practice components for your embedded systems framework so that you don't have to write every single piece of your system you can implement the core piece of your system and leave all of the other stuff that collects data events and all of that to us and then it also comes with modeling and other ground support systems so that when you run your embedded system you can actually look at what it's doing using our standard mechanisms and you don't have to build all of that out yourself that's especially useful for testing because developing a ground system that can talk to a spacecraft that's a job in its own and if you want to test these things having a nice little ground system that's already working with your software will help you test quickly so very similar idea why f-prime well because we come with those standard components those standard components help you build your application quickly the biggest one here is flight heritage we've flown f-prime into space we have CubeSats up in space right now that are running and showing that f-prime can handle the job and you get all of that robustness in your application it's also designed to be modular and be reusable some of those CubeSats we've talked before they've been cloned onto other CubeSats and modified just the science instruments and the rest of the system is the same that's because f-prime is designed to quickly deploy similar setups and just change out the bits that you need to change for your project or our mission it comes with some scalability features so that you can actually do real f-prime missions and we try to keep the code base compact and lastly comes with the unit testing framework so for those of you who are not used to writing a lot of unit tests we help you there that really underlines the importance of testing so f-prime framework itself it really breaks down into ports components and topologies all of our system is a point to point network between components components are where you write your code they're where you write your application so imagine that block diagram that you all start with when you're writing flight or when you're writing software you draw out some boxes and then you draw the lines between those boxes the ports carry the lines they're the communication layer between the components the components are those boxes yourself so right now we've seen you can take that back to the napkin drawing and directly implemented in what we have and then topology is the top level we look down on the system and see all the lines connecting everything together and that's where you flush out your diagrams and make an entire system so really we can start with what is a basic diagram of a system and quickly break it down into the constituent components that are f-prime f-prime has ports those are point to point communications between components we have components which are the primary building blocks where you put your implementation your code and then we have the topology that strings it all together and as I said we were going to talk about events telemetry and commands these are flight software terminology that we use at JPL your events are your log messages so if your spacecraft is doing something I'm booting up or I'm turning on said driver these are the events that you'll see and f-prime has standard components to help you downlink those events from your embedded device to your computer or your ground system and it allows you to see those so that you can collect them in a unified way telemetry is that data that is constantly streaming back from your embedded device it's our what is your GPS position or how many pings have you received from earth or what is the temperature of your CPU all of these things are data points that you try to stream as much as possible because you want to see the health and the status of your embedded system so f-prime once again handles telemetry in a standardized way so if you're say deploying a raspberry pi you can collect the CPU temperature put it into a telemetry item and downlink it and now you have in your ground system a running record of what that temperature is and finally we have our commands our commands are how you can control the thing we have a standard way of uplinking commands and dispatching it to the device so this gives you the ability to control your embedded system and for those of you who are looking forward to the next part of the talk we have three examples of our events telemetry and commands that is the GPS message that the log message that says what our GPS device is going to do we have our GPS latitude which is one of the telemetry items our GPS is downlinking and we have our commands which is hey tell us if we have a lock yet and we'll see that later as we get into the real example of F prime so some of you might be wondering what is our F prime development flow so you start with your component and port design this is that block diagram that I told you that you write out on a piece of paper you start with that you diagram what components your system needs and the ports that connect them together then you start integrating those pieces into a system topology you build the code itself you test and then you repeat so F prime is designed to be iterative you can put together parts of the system add a component string it into the topology test it and then add another component and repeat this process and so a bigger look at the actual flow we've broken out what F prime actually provides for you and what files that these things are going into so first you have the component design that's done in the XML file next you have a green box which means an autocoder is kicking in so you don't actually have to do that box yourself but the autocoder will generate a basic implementation for you so you've now got this stubbed out file that you can write your code into then you go ahead and implement the thing itself by adding in your own code then you go ahead and generate the build which is again an automatic process you can generate unit tests an automatic process and then you can go ahead and fill in your unit tests for unit test specific behavior and then you repeat the whole process so here we can see that there are kind of two levels you have your generic workflow where you're working through the design and adding components and then you have the second piece of your workflow which is actually building the files themselves so you start with your design which you get from this left side you generate an implementation and then you fill in that implementation you generate a build you generate unit tests fill in the unit tests and now you can see you have a complete system that you can step through you can add components and let the autocoder fill in the blanks so here's our example today we're going to be talking about a GPS example we have a tutorial on how to do this on f' and its goal is to take a raspberry pi hook it up to a pretty standard off-amazon GPS receiver that happens to go through a USB cable and we're going to integrate it into f' so that f' can be a little GPS tracker running on a raspberry pi so we start with our design our design is we need a GPS module that's actually going to watch the GPS telemetry and then we need to take those three pieces we talked about commands events and telemetry and hook them to the f' components that are part of those for us so events go to the active logger the active logger is a component in the system that receives events downlinks them if possible and otherwise stores them telemetry is the stream of data and we want to send that to the telemetry channel which is the telemetry handler so it acts just like the active logger but it's handling a different type of data that is this constant streaming data as opposed to your log messages so the command dispatcher hooks up to your uplink and runs back towards the GPS to process the commands for your whole system and for just our GPS commands it sends it to our GPS unit so now we have a GPS unit that can handle its specific set of commands and emits its specific events and telemetry which are then handled by the generic system so as you can see to design an f' application here we just need to implement the GPS box we don't actually have to implement all of that other stuff in the system that helps get our data where it needs to go because that's already done that's already flying in space and we can take advantage of it so let's take a look at our sample commands telemetry and events that we're going to handle today the command that I want to handle is reporter lock state for those of you who have not used GPS it has to talk to some satellites orbiting our heads and collect a certain number of satellites before it says it's locked meaning its position that it's giving you is stable, fixed so I put in a command so that it would generate one of those log messages saying we're locked or not locked this is important so that as we're using this GPS tracker we know whether or not it can see the satellites or not and our telemetry, pretty standard GPS, our lat, our lawn and our altitude so where are you and how high off of sea level you are so this kind of gives us our XYZ coordinates and then finally our events we have an independent event stream that will tell us if we lock or if we lose lock so that if we're not constantly running that command we know what the state of our system is so it's a pretty simple example but it's powerful because now we have a GPS receiver that is on an embedded device that I could say put here on the podium and say where I am so let's look at some of the implementation files the first thing I said you needed to create was an XML file this XML file kind of determines what it is that you are going to design this is taking your piece of paper diagram and writing it in XML so that we can run it through our auto coders and I'm going to walk a little bit closer to the screen now because it's kind of small and this is somewhat unreadable I'll apologize for that so what you'll see right away is this is our component design for the GPS and for those of you in the back you're just going to have to bear with me because this XML did not print as well as I would like but here we have port definitions these define all the things coming into the GPS and out of the GPS these are things like hey this is the event port where we're going to be pushing out all of our log messages or our events that will be hooked up to something else and things like our telemetry out and our command in and then on this side over here we have some definitions of those things that we talked about so we have one channel definition which tells us what is our latitude our longitude and our current altitude I'm not going to spend too much more time on this because again these things are hard to read and I also have some slides coming up that are filled with code which will also be hard to read so let's jump right into it we saw the XML we saw how quick it was to define the ports that are our component and how they attach to other places in the system now we can see the code itself so these functions that you see here like the one at the top those headers come from that auto generation piece where you say make my implementation so this block of code you see here all I had to write to make this work the rest was implemented by the auto coder so all I had to do was start with that test driver code that I wrote when I was playing with this GPS unit weeks and weeks and weeks before I decided to integrate it with that prime it's a little block of code that said read a message, print it to the screen I drop that block of code right into this function which is our scheduled in handler and I change the prints to be the port calls that say send event or send telemetry that's really all I had to do and so that gives us our complete GPS application so this talk is going to turn a little bit more into a demo because some of that code was hard to read which brings us to our fprime inaction section here's fprime running so pi at blueberry is the name of my raspberry pi and I'm showing you just the little binary like any other c++ binary that I've deployed and I'm going to go ahead and run it I'm giving it the address and the port of my ground system this is telling fprime to go ahead and connect to the server that I'm running as a ground system if you're we'll talk a little bit more how to make this into a fully fledged project but for now we're using the test ethernet interface that will say hey go to my ground system and there you can see I went ahead and left in those prints that I wrote in the initial version of the GPS driver and it's just printing onto the screen hey I got these GPS coordinates you'll also see some stuff at the top that's fprime doing its initialization it's registering the commands in the system it's registering some basic events and starting rate groups which we'll talk about in a little bit so here's our application running for those of you who are super clever you can figure out where I was sitting when I ran this because those are real GPS locations and I didn't bother to scrub this video I think I was sitting at my desk at work so you'll probably see a little ping where JPL is here's the ground system which is what I was hoping to connect to it's connecting via ethernet cable and there you see we got a GPS lock acquired in our log events so remember I said log messages when it kicks in the first thing it does is try to listen to the GPS and eventually it gets a lock acquired here's our telemetry tab and we're going through the telemetry and you can see those same telemetry items that I was telling you about earlier latlon and altitude are being printed in the ground system so we can actually watch this thing and there are a few other bits of telemetry that are generated by fprime so I think here in just a second the video is going to jump over to the commands tab, there we go and the command will be sent from the ground that is the fprime interface let's start with some command no ops this is a built-in command inside the dispatcher component that I'm using it's part of fprime and it gives you a little we did nothing but we received your command it's a great way to test and make sure that you have communication without making your embedded system do anything else and now I'm going to go ahead and enter in that GPS command we talked about the report lock status because I want to know if it's locked or if it hasn't seen satellites yet so we ran it, we're going to jump back to the log event screen where we can see our log messages and we get an activity high saying hey we do have lock status which means those GPS coordinates that I'm showing you there are real so again if you're following along or following along at home you can figure out where I was sitting when I did this that's fprime inaction that's actually running an embedded application it looks very much like what we do at work except the number of log messages coming through is relatively small the amount of telemetry coming through is relatively small because it's not a huge application that's controlling hundreds of drivers I have the small piece of the system which is the part portion that routes commands and routes log messages and my one component which controls a GPS but we're moving on to the portion of the talk where we talk about fprime in the real world so real world fprime how do you use fprime on a big application like say a spacecraft well the first thing we have is modeling software we have a plugin it's an open source plugin to magic draw magic draw is a uml slash sysml language used a lot in our industry and it allows us to do those back in the napkin diagrams in a uml language and then generate that xml that you saw so now we've actually stepped away from writing raw xml ourselves and we're generating those pieces automatically from a design utility now I know we're at scale magic draws not open source the plugin is but we're working on it yes did you have a question that's a great question the question is why did we not choose an open source uml slash sysml I can't answer that question because that decision was made long ago the best stab I can give you at that is lots of groups at JPL are actively using magic draw as a way to do their uml sysml design and we wanted to keep consistent with what the real missions were doing yeah go ahead Len for those of you who couldn't hear because it's not actually part of the microphone it was chosen because the tool magic draw gives you the block diagrams that we're looking for to do our our diagramming we all are working rapidly towards getting an open source and available solution uml that's part of what we'll discuss in the future section coming up so the other thing you have to do is unit testing continuous integration and full on full fledged testing so when you're going to a full project an actual mission you want to do unit test on all of your components f prime supports that I didn't get into it deeply in this talk because that would have been another hour long talk on how to do unit testing what's important about unit testing etc but those same auto coders that we saw that was generating the xml, generating the implementation they generate stubbed out unit tests for you so you can fill in just a little bit of code and roll with it and start doing unit tests what we've also found that is important for f prime is the ability to run something like continuous integration I know for you um what are they called I'm blanking on the term but the deployment engineers the people who sit out there and do DevOps thank you the DevOps engineers they love Jenkins and Docker it's wonderful you can string things together you can test and just let the machines test it well this is something that we've actually integrated on several projects using f prime at work and it is wonderful because every time you push a commit up to github or whatever your content management system is you can trigger a build and run through your standard unit tests your standard runs your build and make sure that nothing broke so moving up to the level where you're doing continuous deployment on what is not traditionally a continuously deployed item like an embedded system is really important to taking care of that testing that you don't have the staff or the resources to do because now developers are developing 24-7 all around the world so if you want to do an embedded system set up a basic Jenkins project use Docker to encapsulate f prime's build system and all of your code into a container and then run the basic Linux tests these f prime just like it runs on the Raspberry Pi will run on this machine up here it'll run on your Linux machines and you can do basic testing and you know some drivers won't be there but you'll be able to test the system every time somebody checks in code and then of course move on to integration testing integration testing is where you bring in the real hardware piece the actual flight model or an engineering unit or I'm using all of these industry specific terms you bring in your actual hardware your actual embedded system and you run tests on it manually so now we have three levels of tests we have unit tests which test the code we've continuous integration that tests as much as the code as we can do automatically and then you bring in your hardware at the very end and start paying your test engineer to actually touch that hardware and test that the integration piece the piece that actually touches the hardware with the drivers and things works and then last thing you have to think about when you're moving towards a full scale mission is operations how do you integrate f prime with the OS layer are you going to do like a lot of people do in the aircraft industry and build one binary that is a combined operating system and code are you going to run on like we did with the Raspberry Pi where you have an embedded Linux and then you have a user space application that is your flight software you need to answer these questions how are you going to communicate with it here today in our demo we use the ethernet interface that allows us to talk to f prime through a standard network connection but you know I don't run ethernet cables to satellite orbiting Earth if I could I would make a lot of money in the space elevator industry so you've got to think about how you're going to do your communications how are you going to handle your radio it's fairly simple to write a radio component that talks to a radio in f prime but how are you going to do that full scale communication from the ground and then what software are you actually going to run on the ground f prime comes with the ground system that we use a lot for tests you could use that as your ground system or you could engineer something more specific to your use case so for those of you who are still with me we will do some best practices and tips on how to use f prime so the first thing you need to pick up when you're using f prime is those rate groups that I mentioned earlier a rate group is simply a collection of components that we're building in f prime that run on a clock so you have a 1 hertz rate group that says to components every second do a bunch of stuff a second later do a bunch of stuff a second later do a bunch of stuff this is how you achieve that deterministic timing that we were talking about earlier a 1 second clock tick or a 10 millisecond tick or whatever it is you need for your system going out there and telling your components do something do something do something now you can just write your block of code like we did with the GPS which is read a message process it and return and now you have deterministic timing because you have a clock driving your actions not a whole bunch of threads running on your operating system that just run when they're scheduled so this is a way of formalizing the scheduling of your system just like we formalized the memory by saying we're going to allocate everything up front and then use what we allocated up front for the rest of the program here we have formalized the timing we're saying any time critical pieces of the system will run on these clocks driven by rate groups we also have the parameter database the parameter database allows you to configure your system so this is the piece of the system that allows you to set various parameters that change the operation of the system and it allows you to formalize the way you've configured it and change parameters via that command interface that we saw earlier so you can change parameters with regimented commands that aren't like hey jump on and vif file but send a command rather and configure your system and the parameter database keeps all of those parameters safe with their default values the last another thing to think about is the hub pattern the hub pattern is a way of organizing components so that if you have to have a bunch of components here talk to a bunch of components there say a multiple device system you can run them through a hub and have that hub encapsulate the communication link so now you have your array of components on one side talking to a central hub which then runs across a air gap or a communication link or some other piece to a set of components over here now you have fprime methodology running on a multi node system and the last thing you really have to get when you're looking at fprime as a mission in the real world is fatal handling you need a good strategy for how you're going to handle your faults we kind of hand waved earlier and said fault handling is one of the most important pieces of a system like this and of flight software well fprime provides you components for handling faults and fadles and everything else that can go wrong but you need to design what your responses are in your system so that when you say get an assert because a bit flipped because of a solar flare your system knows what to do typically I like to reboot the system in that case because you're in the middle of a solar flare but you could choose to do something else you need to design that just like you designed your software so the whole system is together unified consistent and now we're going to talk a little bit about fprime in the future and we have kind of a treat this is a video one of the things that fprime is going to be doing in the future is running what is I think one of the cooler projects that we're working at JPL do note I'm actually on this project so of course I'm biased we're going to be flying a helicopter on Mars so as you saw on the last side there's that little picture of the helicopter this is the Mars helicopter project the goal is to take those rovers that we've been launching for years and give them kind of a aerial scout that can fly up sides like you see here where the rover couldn't drive and take some pictures so now we're taking that rover which is an amazingly technical instrument that can get us all sorts of science data okay I got to get my explanation this is the flight test so this is in a vacuum chamber and we've proven that this thing can fly in Mars like conditions if you're interested in about this you can ask some questions at the end of the talk because I really shouldn't go on about the helicopter for hours and hours but we are able to fly in Mars in like atmosphere so the goal of this project is to get those rovers and give them something that can get up that hill we can see on the other side it's a little extension to an already amazing project like the rover to give us just that little bit of extra information and fill in that one feature hole that the rover has which is it can't drive up a cliffside which is great so the future one of the things we've already mentioned about f-prime is we've moved open source we went open source a year ago a year or so ago and we've been trying to build that open source community so the first thing you're going to see about f-prime in the future is we're picking up external institutions at JPL who are working with f-prime helping us debug helping us develop helping us make it an easier project for you to use that's why I used this some assembly required because all open source has some assembly required and of course photo credit to Kennedy Space Center they did these awesome graphics a few years ago so that's the first thing is we're open source now you can use it you can sit down on your laptop and build a little GPS tracker like I did or maybe with some extra information you could build a whole spacecraft it's out there it's available and it's ready to use on github as we speak the other thing we've talked about already we're working on better modeling tools when we moved open source we realized the limitations of magic draw is that you have to kind of buy a license because it's proprietary software we're working to fix that so that like universities or hobbyists can use f-prime and take advantage of its strong integration with modeling without having to go through magic draw I'm guessing that at JPL we'll still be using magic draw for a good long time because that's integrated into our processes but you'll have options and of course as f-prime moves into the future we'll be looking at other missions and other things that we can do with this embedded framework that's my slide on the future is we are open source now we are making improvements we're trying to work with the community in the open source community and with colleges and universities so that we can better our project and make it more accessible to all so my favorite part of the talk question time I've left about 20 minutes so we have plenty of time for questions and so yeah go ahead that's a deeply technical question and I'll say no so a messaging bus is an underlying layer that can connect multiple components one of the technical notes about f-prime is that it's a point to point architecture so you actually have these ports that are either dispatched using a communication layer or using more often function pointers in the underlying implementation that gives you point to point communication the real reason we did this is we wanted small compact quickly dispatching calls between components so it doesn't actually force you to use a messaging bus for those of you familiar with robot operating system ROS they have a message bus architecture so all of their messages hit this bus layer I think it's pub sub so publish subscribe I could be wrong and it broadcast it out and f-prime is different it has actual hard connections between the components implemented quickly if it's all on one node with function pointers and can go through serialization and message passing if you need to get somewhere that a function pointer can't reach it's a framework to build embedded systems you can use it to build a single executable as we've shown here today and you can link it using external communication layers to other executables so as it stands today we're building a single executable which is our flight software controls the system, executes the system so it's not designed to be hey we are the whole topology of 20,000 processors on your robot it's designed to be the for lack of a better term operating system of your spacecraft because Linux itself won't run the whole spacecraft you need extra software to make sure that it's high reliability now one point I should mention and I will get to your question I see you is that fprime does not prevent you from having multiple executables and multiple environments so a lot of time a control that has an operating system level and user land applications command events telemetry but then there are further embedded microcontrollers deeper in the process that runs say the drive loops for your motors or something fprime can actually run on both sides of that fence and communicate between the two so you can build an fprime application here and an fprime application for each of these pieces and use it to build the individual sections of your project now that's where that hub pattern I was talking about comes in hub pattern is what we use in fprime to go through a central hub to communicate to other nodes and you might wrap something like a messaging bus we've had people looking at wrapping something like robot operating system to tie fprime into that for research purposes you had a question right so one of those standard components that I was talking about earlier is the OS layer and the OS layer ties into various operating systems out there so that you can run fprime and take advantage of fprime how would we say interfaces to your operating system so if you want to do something like create a file or open a thread or something like that we have a wrapper for that so that the same fprime would run on multiple different operating systems that's how I can test that GPS code here on this Mac and run it on the raspberry pi because that OS layer prevents the differences in operating systems from breaking my code because we have these standards this is how we're able to test now what I heard you talk about is a simple dispatcher so that's one of the things that fprime is actually bridging in a number of our applications is traditional application development is today all threaded everything is a thread let it run it will run at its own pace your timing goes out the window and for some things in our system that's not very problematic so if you're writing a file and you want to page something to a file does it really matter if it runs now or in 10 seconds most of the time not that's a nice background worker and we can treat it as a background worker and that's where something like a task in the fprime layer would come in you could create a task that is your background worker sending things to a file the rate groups on the other hand the rate groups we like to attach to something like a hardware ping that is your hardware clock and that I like to personally in my projects dispatch things with fprime synchronous call so that we're not running on threaded systems anymore we're going straight in a synchronous loop and as soon as that hardware tick says it's clock time we get that simple dispatcher so now we've seen how fprime can span kind of the traditional embedded system which is clock driven and an operating system that can give you tasks so that you can take advantage and balance the advantages of both worlds does that answer your question yeah so that's what you do the rate groups for so we kind of briefly touched on the rate groups as an important thing in fprime and you're pinpointing exactly why the rate groups of the piece in your software that takes the ping from hardware the ping from the clock and dispatch a set of port calls the run or what we call schedule in port calls at an exact clock tick so that's how we take the time out of some of our deterministic calculations if you hook up a bunch of components to the run port of this active rate group then every 10 milliseconds those ports are going to get dispatched which means the I should jump back into my apologies I was trying to go back where is it give me just a moment I'm going to pull up that code where was that gps application here we are one short there we are this is our gps implementation and this function here is the schedule handler so I wrote this using that same methodology our rate groups are run at in this case every second on my raspberry pie this function is the function that is dispatched on that port call so every second the active rate group is told dispatch your ports dispatch your ports dispatch your ports and what it's doing is going through its list and saying call this function call this function so now I have a generic way of looking at a component design system while still running things on a one second tick so what we traditionally do is tie a hardware driver into or a hardware clock into the rate group driver and ping all of these schedule messages every one second now for the gentleman who asked about messaging bus one of the reasons that we use function pointers is so that this is quick because you need this to run every 10 milliseconds and you don't want to wait for a messaging bus layer to say okay we're going to pack up a message and dispatch it if this is running as a function pointer it's super fast and it can be dispatched every second reliably and that's how you get your rate groups now as we saw a prime inaction here if I can figure out how to forward this video just a little bit I cannot well it'll run a little bit and as we move into the telemetry tab again you'll see at the top there's a little downlink that says these are the times that your rate groups are running and that gives you a feedback for how reliable are rate groups do we have slips and are there problems in the deterministic timing portion of our section I kind of hand waved and said there's some other telemetry from f prime at the top because I didn't think we'd get into a deep discussion of rate groups but that's those top three items are reporting the health and the timing of those rate groups and there's more information available if we were to dive deeper I see another question actually there's a guy back there oh no go ahead these rate loops that I'm talking about are a software construct so if you compile your thing as a user space executable the rate groups are baked inside if you're compiling it as something like vxworks might work where at the unified image the rate loops are inside there if you what do I implement I'm having a hard time comprehending the question so I would say let's chat after the talk and see if there are any other questions in the room because we've done a deep dive into super technical deep levels of f prime and it was a little bit much for an introductory talk so I'm going to pick up some other questions I'm happy to chat with you about this afterwards because I think over time I'll come to the answer you're looking for but go ahead how do you create dashboards for ground operations so this is one of the things that we're looking at for the future of f prime is how can we make this ground system that you saw here more effective and easy to use in just as a general product right now most of the ground systems operating spacecraft are big ground systems developed by whole teams of people for the use of professional operations at JPL so our ground system here is a great ground system for something small like testing in the lab and making sure things run there so what we have here supports our needs you see your telemetry, you see your commands you see your events and that's what we've needed and there are a few other tabs that we've added as we move towards the future we're hoping to find better ways to make that a little bit more extensible so that you don't have to choose take this ground system or take or completely engineer yours from the ground up but it requires more configurable so we're working on that, yes so as it stands now most of our stuff is about handling commands events, telemetry basic health, basic fatal handling of the spacecraft itself this GPS package here is actually the code I wrote as a demo application so it wasn't meant to suggest hey we provide for you GPS it was meant to show you how you might integrate an application for reading tasks so as time goes on that standard library of components that I'm talking about will grow but we support right now what is the fundamental required things to run say your control network of the spacecraft so we don't have full flight dynamics packages and stuff embedded in F prime right now those are on a project by project basis because a lot of those are I think package components from JPL missions so we might wrap for a project a component around one of those packages at JPL I'm just guessing right now because I haven't worked with that sort of thing on my missions themselves I've worked mostly on how do we get commands in and how do we dispatch to this component network to handle these project specific features did I build on, no I don't believe so the Mars helicopter is a very complicated setup because one of the things that the helicopter can't do somebody asked me at the beginning of the talk when we were just kind of chatting have we flown it around the office the answer is no we have not flown it around the office because it will not fly on earth the gravity is too strong and it will just kind of sit there so it's designed to fly in Mars like atmosphere in a Mars like pressure with Mars like gravity so we have tested it in our vacuum chamber but we have not actually flown it because last I heard from the physics guys it will not fly in earth atmosphere yes go ahead sure yes right right right okay so for the test you saw we have mechanisms to actually offload some of the mass from the helicopter so that it mimics a Martian like mass and I can't really speak to all of the crazy physics that goes into trying to fly on Mars because I'm a software engineer not a physics but what I can tell you is the blade design is crucial so in order to get the lift from a smaller atmosphere you have a very specialized blade design I don't know what goes into those blade designs but it's a very specialized blade design and that blade design has does not have the ability to move the earth atmosphere in the same way as to produce the lift required to offset the difference in the size of the gravitational well you're in that is blanking on an answer to that I would guess so but yes I've stood by and witnessed some of those but I have not flown it myself so I love talking helicopter it's a bit outside my talk I am a software engineer so I'm my best to field your questions there but I can't really necessarily answer them all because I'm not a mechanics person I can say to the best of my knowledge that cable is actually part of the mass reduction system so it's not there to provide power but it was there to try to do something so that it's more martian like you had a question I'm having a hard time hearing you are we dealing with the security aspects of our code being open source yeah I'm hearing you so I don't have a great answer for you today unfortunately a lot of the applications we're doing are operating with enough of what I would argue is an air gap that space hackers aren't a huge concern because you've got to punch through some very secured systems to get anywhere near where f prime is running again it's on the embedded system itself how would you secure so yeah hackers could look at the code and what you're doing on the how the spacecraft is managed but if they can't actually talk to the spacecraft then they can't actually do anything because it's running as the embedded system now if you're referring to this ground system here the ground system isn't designed for security and preventing space hackers it's designed to make testing in your laboratory easy so if security of your embedded system is something that's crucial where I would focus your design attention if it were me would be on the ground system and building a reliable ground system that somebody can't say punch into from the outside and then your embedded system is safe because you've you more safe because you've secured the command and telemetry coming on and off the device and not worrying about the device yourself because if the hacker is on the device where f prime would be running you've got much larger problems I would estimate yeah I probably don't have the email address so what I would say is for this software I put up the landing page that brings you to our front page of the repo and you've got the repo here so we we do have an email address it may be on that landing page I'm blanking if it's there but you also have the ability to interact with us through the standard github interaction process issues and such like that so if you are interested in using that software these are the places to hit well maybe not that last one that's today sample application but the top two f prime at github and our landing page which is our github.io basic HTML page and then of course our repo itself so if the email address is not on there most of our external partners interact via github you have a question? Paula in the back is shaking and said yes so yes go ahead so we have an auto coder baked into f prime so it generates the code for you yes love that question I will say that the mac os support and the linux support are really there to enable your testing of basic functionality to the system do we have data flows right do we have passing between components right can we run the unit test that sort of thing so we added a os layer for linux and mac mostly so that you can power those integration tests that I was talking to you about earlier so that comes down to how do you design your real time system which is something that he and I were actually talking about earlier and we got really deep so it's not directly dealing with the real time portions of the linux kernel or mac its design is to give you several options so if you would like to you can figure out the os code and deal with the real time stuff yourself f prime is really there to set up the framework so that you could implement something like that but part of that bigger project question that we talked about I think it's way at the end of my talk was how are you going to deal with the os that as the reason you two are bringing this up one of the most important pieces of f prime on a bigger project is f prime in the os layer so I was really focusing this talk to be kind of how can the community start using this I'm looking towards more of the hobbyist community with this talk because I was hoping people would put it on their raspberry pies and start developing some experience I educate a lot of high schoolers so that's kind of what I was hoping out of this talk was hey people can start picking this up seeing what we do at NASA seeing if they can't get this stuff running there are bigger deeper technical discussions of how to use f prime on this page and that last bit f prime in the os integration that's how you do your real time are you integrating with the real time kernel are you running on a operating system that doesn't have a real time os and just kind of runs and you virtualize real time are you using hardware interrupts and locking out other processes so that right all of these things f prime can do it's a question of how do you set up your system f prime isn't there to tell you how to build your applications it's there to enable you to do it providing certain features for you and leaving other things to your specific application yes so again I pointed those right groups that's how I do real time I if it were me I would hook them up to a hardware spinner on a clock that says hey dispatch dispatch dispatch and I dispatch those right groups in a thread safe interrupt context thus get forcing my application to run on real time as much as the interrupt context can provide that's what I would do personally that's not for every project some projects prefer threads running in parallel and letting the thread scheduler of the operating system Linux or what have you sorted out because their real time constraints aren't as strict as this gentleman over here pointed out you can actually bury some of this stuff into the hardware layer if you have access to the hardware layer so for me I like to run things either in interrupt context or as locked down context as I can and make all of them and avoid the thread schedule all together but that's because my embedded background goes back to we're not even running operating systems we're just on the chip itself and I don't want to deal with the thread scheduler of the operating system but that's a personal preference for a lot of my smaller projects I'm not at the point on many of my missions where I'm making those calls that's up to the project design folks so that's what I can tell you you had a question do I see any limitations designed to prevent you from using sorry I should state it in the microphone the question was do I see any problems using fprime for automotive use or underwater robots and the biggest limitation I could see is what does the community exist that is doing those things already from what I have seen of like 10 years ago field tests of Google's cars that were passing around on the internet they had full blown laptops running these self-driving cars and things so there you're looking at something like a network of CPUs that are all running and I guess based on that architecture and very limited knowledge of the robot operating system I guess it's probably doing something like that so in this case you probably could adapt fprime to these communities but you might be pushing against the research that already exists there and I don't know if that's what you'd want to do so I'm a huge fan of fprime I love using fprime it's a lot of fun and I like to put it in a lot of places that's why I'm putting it on Raspberry Pi and giving inter discussions at scale however I also know enough about project design and working with existing code bases that sometimes you just have to take the flow of where that community is going unless you have a good reason to step outside that flow so this is more of a general software engineering perspective and not an fprime or a NASA perspective but if the community is going in one direction and you're going the other you're going to have a harder fight and have less of your time spent on the thing you care about the code you care about and more setting up all the stuff that your community already figured out using their other technology it's like trying to convince my mom to use Linux instead of Windows she's so used to using Windows that I can spend all of the time I have on our weekly conversation getting her on Linux or I can just boot and say she's using Windows let's talk about how the dog is and how my family is and all that stuff right so I don't see any specific limitations that would prevent you from using fprime in embedded systems anywhere except for the fact that the knowledge share of the community might already be focusing on some other technology and you'll have to devote a lot more work to adapting the technology and not devote your work to whatever research aspect you care about most then it's your question fantastic yes go ahead that is true I don't have any notion of projects that are non-flight that surpass the hobbyist community where people are using fprime right now our work is done on satellites and robots in space because that's our business and the people will come to us to start partnering us through this open source initiative they're also doing satellites and they heard about us because that's what we do and that's where our focus is I like to use fprime on things like raspberry pies because it's fun and it starts expanding the use cases but I don't have any I can talk about today unfortunately so yeah I know most of you aren't launching satellites I was expecting a good number of people who are like we like playing with raspberry pies if you're deeply hoping to use this in the integrated community it's not a bad product but we don't have any to point at and say look they're also using it on XYZ you know network infrastructure or embedded Linux computer that's running I don't know Cisco systems or something we're not at that point any other questions so why was the longitude fluctuating so the longitude was fluctuating a lot because I had gotten a brand new GPS device I had managed to fry my original test model by carrying it around in a backpack without static shielding so yes put your electronics in anti-static wrap if you're going to wander around for months and so the new GPS unit hadn't been used long enough that it could pinpoint its position that was very close to the first turn on and what I learned using this particular model of GPS is it takes some time to get a 100% definitive lock especially when you're operating the thing indoors and it doesn't have a clear view of the sky so hardware instability over time this particular model does seem to get a more fixed lock but I can't say that 100% for certain because it is a very cheap off Amazon GPS you know for hobbyist projects yeah go ahead could be I wouldn't be surprised because I don't remember exactly where I was testing but it was not outside it with a clear view of the sky it was more like sitting at my desk because that's where my computer was to record the demo because I learned at last scale I presented at a few years ago don't do live demos in front of the audience record them it prevents errors well I'd like to thank you all for the time I'm going to shut off my microphone now and if you have further questions I'm happy to answer a few before I have to run so thank you oh hey okay this actually works okay I think it's probably hold it though I'm kind of hoping that maybe there would be another a second worn one anyway testing 1, 2, 3, 4, 5 alright there we go this thing is highly directional obviously I think worst case I could just hold it and like hello hello hello hey there hey hey passivy you know we could push the volume up a little bit that I know how to do but oh but it's yeah eventually coming up after it warms up okay there we go alright it is a microphone microphones are awesome we've established that it works yeah yeah we do okay yeah I actually need to boot the machine too is it plugged in 1, 2, 3, okay it's 6 p.m. so let's get started welcome to accidentally accessible mostly FOSS workflow my name is Sarah and my name is Chris and welcome tonight we're going to start by just going over the agenda for our presentation we're going to start by talking about why you should care about accessibility now rather than later when you feel it might be more convenient next we're going to do introductions of ourselves and of the company I'm going to talk about the communication tools that we use at programmer.com I'm going to talk a little bit about our physical infrastructure because I think that's important for giving context for Chris's talk I'm going to discuss the hiring considerations we had when we were thinking about hiring Chris then Chris is going to go over some user interfaces for the blind some of what he uses day to day he's going to talk next about we have some okay so Chris is going to talk about applications that he uses he's going to talk about pain points that he's encountered during his work then best practices for screen readers and then I'm going to wrap up with a little bit of discussion of other disabilities okay why why should you care about disability now rather than some time later if you think about what it takes if you have like an external client that somebody might want to write for themselves to make an accessible client you're probably looking at an API you're probably looking or you're looking at some kind of message protocol if you have these available you can use them for accessibility but you can also use them for many other sorts of integrations then another thing to consider when you make your organization or you make your software or your project more accessible is that you're opening up your pool of potential candidates or contributors if you think about I think the unemployment rate for people who are college educated right now is 2.2% it's incredibly low if you want to be getting the best people you need to make sure that you can cast your net as wide as possible finally inaccessibility it's technical debt I would say it's also social debt for an organization and the United States if you have 15 or more employees you need to make reasonable accommodations to somebody if they're able to do the job if you wait until somebody comes along and you want to hire that could be very disruptive you might have to think about well we've been using this one chat application and we could go ahead and switch to the other chat application it's pretty much as good but that can be a painful process and there's not really going to be good feelings about that on either side so think about it upfront not just necessarily for in your job but also open source projects if you have an accessible chat program you may get people you wouldn't have otherwise since this is a presentation about people and how people interact with technology just going to spend a little bit of time talking explaining where we came from I grew up very interested in computers I have a degree in computer engineering and I got back into Linux when I had a Nokia N800 which unfortunately I don't have, I can show after the presentation but it is a small internet tablet, it's very smart phone like running basically running Debian and this was before the iPhone and this was super cool and as time went on I decided that I wanted to join Programmer because I saw a lot of opportunities to improve the automation Chris Yeah, so I grew up with an interest in technology I didn't really own a computer until the late 90s but I did start working with Unix earlier than that and I used a terminal and a type device and dialed into shell accounts and I learned the operating system by exploring and reading man pages and teaching myself the system and when I actually owned my own first computer I went looking for a new operating system after dealing with a number of frustrations and I settled on Linux primary things that led me to it were that it was free and it was easy to modify but it could be modified if I learned how to do that if I learned how to program at the time Linux was one of the few operating systems for the PC that could be installed completely without site there's still some truth to that today but the landscape has changed at the time one of the few things you could install without site other than Linux was MS-DOS getting away so I went with Linux I eventually got myself a computer science degree and learned how to program so I could modify my operating system and I haven't looked back it's been quite an adventure and that eventually led me to working at Programmer Sara? Okay, so just in case you aren't familiar with the company because I'm going to go back to what Chris does today I'm going to explain a little bit about Programmer so we provide virtual private servers to the general public generally running Linux or BSD though people can install their own operating system we were founded in 2005 by Luke Crawford if you want to stand up Luke anyway so he started with three BSD jails and then he moved to Zen within a year because jails didn't really do a very good job of controlling resource contention we own all of our own equipment we operate our own network we self-host all of our own infrastructure because as Luke says why would you host a hosting company that doesn't host their own things and we don't have our own data centers we just rent racks and for communications as I said we have everything that we have so for email we run postfix and dovecott for chat we just use IRC we use the NC for the bouncer inspire cd for the IRC server voice is just traditional phone or we also use signal documents, plain text, markdown or media wiki so we have customer documentation in media wiki and some internal documentation in media wiki and then for ticketing we just use request tracker which was originally designed for help desk we experimented a little with having multiple ticketing systems that found that maintaining one was generally easier every single one of those tools I just listed is open source and you can run yourself again just to give a little bit of context for our physical infrastructure our serial we use serial consoles for all of our equipment so it's all accessible via text we have a few specific command line utilities for accessing hardware such as IPMI tool and SNMP we some of the equipment has built in web interfaces which we very occasionally need to use because they have some they expose some features that aren't exposed via the serial console and then for server installs we automate them via kickstart which is for the red hat installer or if that if we have something else that we're installing not the red hat installer we'll just do the install over the serial console and then hiring considerations when we were thinking about hiring somebody who was blind we had absolutely no concern with questions we figured Chris would be able to do the job since Chris is primarily working remote I in fact met him for the first time yesterday data center work doesn't really matter graphs when I was thinking about our traffic graphs that we have for networking that was a potential concern of mine and I think it's a problem that we can solve ourselves it's just that we haven't sat down to do it and then really large blog files seem like that would be a little bit more challenging for Chris but I also had complete confidence that this was something that he would be able to solve and now I'm going to turn it over to Chris to talk about some of the user interfaces for the blind okay so I'll start talking kind of generally about the various output methods that are available to a blind person the two most common ones being text to speech and braille text to speech a either a hardware device or a software program takes computer generated text and converts it into synthetic speech at one time this was done entirely in hardware for many years that was the case because the machines of the time were too primitive lacked the resources CPU, RAM, etc to perform the text to speech function over time machines have gotten more powerful and like many things this task is being done more and more in software I myself use a software speech synthesizer called eSpeak and it is freely available on the internet it is open source software hardware devices are still used a lot of people swear by them especially further the quality of the speech but they're becoming less and less common as time goes on when you use text to speech I would say probably at a minimum at the absolute minimum it reads at 100 words per minute some people run it much faster than that I know people that run text to speech at 900 words per minute I myself run at about 450 we've read that human speech daily human speech is about 180 words per minute so most I would say a lot of people who run text to speech run significantly faster than typical human speech a synthesizer usually provides a few knobs you can twiddle when you have to deal with programming for instance the amount of punctuation spoken that is read is adjustable so if you're reading for information or for pleasure typically you would have no you wouldn't want to hear punctuation characters announced you set your punctuation level to none if you're editing a program or a config file or you're reading command output or dealing with anything that has syntax consisting of punctuation characters then you probably want to set your punctuation level to all and have it all announced not all synthesizers speak every word as it should be spoken out of the box they have their quirks so to get around those quirks often times you can program a dictionary to override the pronunciations of words another thing that's important especially when programming is capitalization because UNIX especially is case sensitive identifiers are case sensitive command names, file names what have you so text to speech makes some allowance for that typically a rise in pitch is used to indicate a capital letter so it's yeah it's very flexible the other common well maybe less common output method is braille I would say probably most readers read at about 60 to 80 words per minute maybe 40 to 80 words per minute braille typically is presented using a hardware device called a refreshable braille display they have one row of text of braille cells compared to a screen which has you know what 25 rows by 80 columns well these things have one row and typically they have anywhere from 20 to 80 cells so very minimal display real estate kind of a minimal bandwidth IO channel the other thing about braille displays is they are mechanical devices and they can be quite costly we're starting to see cheaper units on the market these days like the one for $500 that's a 20 cell display but up until recent memory they all cost several thousand dollars I would say typical prices um you know 2000 to 4000 for some of the well of course there's the $500 model 2000 to 4000 for maybe a 40 cell and then for one of the top of the line units so very expensive a braille is less commonly used in the United States than in Europe um probably because European governments are more willing to afford it for their citizens here in the United States we tend to use text to speech more than braille one thing to note is of course if you're deaf blind then braille or some kind of tactile method is your only option if you have no hearing whatsoever speech won't cut it so a third output method is of course haptic feedback and this is used to some extent today like my phone uses haptic feedback it has a vibrating motor and it vibrates for various kinds of notifications and that conveys limited information one thing we kind of stumbled across while we were researching this was the possibility of using haptic feedback to convey text i.e. having a motor vibrate and Morse code there was at least one project we found on the internet where somebody had done something similar i think it's a possibly a fruitful area for research but it's kind of blue sky territory here the advantage is that it could be very cheap you could make this kind of a system with a raspberry pi for under $100 unfortunately probably a very low bandwidth IO channel maybe 20 words per minute maybe a little more but it could be another option for people who are deafblind who only have tactile methods as their only option something that's very commonly used by both blind people and sighted people with hearing is non-spoken audio feedback typically this takes the form of notifications sometimes the term sound icons is used but there are audio notifications of events for instance a message was received or a task has completed or begun there's not very much information you can really convey with this but it's super useful some general definitions there are two strategies if you will for providing accessible software to the blind and these are screen reading and self voicing applications a screen reader is a program that sits between an application whatever it is you're running web browser, email client what have you it sits between that application and either a text to speech system a speech synthesizer or a braille display and the screen reader interprets the output from the application and converts it into something that can be rendered on the synthesizer or the braille display so the term screen reader can be a bit of a misnomer of course if you're talking about VGA in text mode it does in fact read the screen but if you're talking about some of the modern GUI screen readers they don't necessarily read the screen at all in fact what they do is they hook into accessibility APIs provided by the operating system or system messages or events inside of the GUI and they hook into those events and build a model called an off screen model and they read that model, they read from the model rather than actually reading from video memory so screen reader can be a bit of a misnomer the other strategy for building adaptive software is called self voicing applications and here the spoken feedback is part of an application it's built into an application or baked in often times that's done using a plugin some kind of plugin system an application and somebody writes a self voicing plugin or it's done through an application specific scripting language it's a very powerful paradigm but it's not often used I would say screen reading has won the day and there's a pretty good reason for that it takes more work to write self voicing applications you have to add the self voicing to each application individually and it is application specific so extra work duplication of effort the advantage is that the application has more context about the data that it's processing and the task it is performing than an external screen reader ever will and it can thereby a self voicing system can thereby use that context to provide a more fluent and useful spoken feedback I'm going to display some of that here in the demonstration that I'm going to do and alright so the demonstration that I'm going to do today is I want to demonstrate me essentially writing some Python code and I'm not going to write anything complicated just a very trivial hello world program but I'm going to use two systems I'm going to use a self voicing system that I use for writing Python pretty regularly it's a an extension to the emacs editor called emacs speak and I'm also going to show me writing the same segment of Python with a text editor called nano in a screen reader and it will be a pretty good comparison in contrast of the two systems and so without further ado here I'm going to open a file here we go okay just the standard shebang line here okay so I type def main I'm going to write this as a function because it shows it showcases one of the aspects of emacs that I really appreciate and so I type def main open paren and then close paren and when I hit the close paren emacs speak helpfully told me which what parenthesis the closing paren matched and I'll show that again here it says matches def main very useful here I'm going to introduce an extraneous paren here mismatched parenthesis my system oh so helpfully tells me this proves really useful when editing anything with parenthesis works equally well for braces and curly brackets and all kinds of things so I will um backspace over that and let's go to the next line and emacs oh so helpfully indents my line for me and I'll type print hello hello world okay so now when I read the line I just wrote it says indent for print hello world it tells me how many spaces are at the front of the line very useful especially things like the parenthesis matching and the speaking of indentation and so forth I find it is possible to write python without this and without these features but I much prefer this environment um let's see here so for a little demonstration of nano I will log into a new terminal here and fire up nano yeah you don't want to hear the basic help either do I okay okay there's my shebang line again deaf main of course I hear no nothing around the parenz I don't know what matches with what I mean of course I know that because I just wrote it I know what but my screen reader doesn't try to or try to give me any help here let's go on to the next line and okay so I typed print hello world I'll try and read that line here it doesn't tell me how much indentation I have but of course I can get it to do that which I'll do now okay so it says five and it says print hello world I also have my punctuation settings low which I didn't mean to do but it says five telling me that this line starts at column five so that column five with one base numbering is the first column that contains non-white space you can certainly write python this way I've done it many times we'll do it again but like I said I really love my Emax environment there so let's move on and talk about some of the tools I use from day to day so communications tools the nice thing about open source communications protocols is about standard protocols I should say is you bring your own client email of course I live in Emax so I use something that's built into Emax it's called the news it is actually a news reader it's been around for some years it's a news reader for use net and I use that because at the time I needed something that was both a mail reader and a use net client so I've been using the same thing for 15 years I've customized it pretty heavily and once you've invested quite a bit of time into something you kind of stick with it for IRC again I use something built into Emax called ERC IRC client and again I've customized that quite heavily so let's see for voice we tend to use the plain old phone system and we use signal most of that I do with my cell phone I don't actually use any computer based voice stuff as a rule I use my phone for communicating I think smart phones are a real game changer for people who are blind and the reason is if you go out and you buy an iDevice or you buy an android device today it comes with a built-in screen reader it is accessible out of the box and you just need to enable the feature which is pretty easy to do you can do it as a blind person not hard so yeah I can't speak highly enough of smart phones I'd say for documentation we use a lot of media wiki all of our customer facing documentation is on the wiki I like media wiki because it works with links still the old text mode web browser it's been around for 25 years or more actually I think more media wiki still works with that and so a lot of times I will fire up links point links to use emacs client as my editor and I will edit wiki pages in emacs instead of trying to edit in a text field text area that's pretty convenient for me ticketing systems I think ticketing systems are pretty much a problem in open source making them accessible to the blind it's not hard and I've worked with a number of them that are more or less accessible I've used let's see track bugzilla this isn't open source but github's issue tracker I've certainly used and in fact I wrote my own command line utility to act as a client for github's issue tracker and I could do that because github provides a very complete thorough going API I really appreciate that but at work for my work we use RT and the thing I love about RT is it provides multiple user interfaces it does have a web interface I find the web interface quite usable as web interfaces go it's really good what I typically use those email typically when I'm handling support requests I am sending email messages back and forth there is also a command line tool which is fairly complete a command line tool that goes along with RT and I use that too quite heavily for instance making changes to tickets if I need to update the status on a ticket or change some field like the subject or merge it with another ticket I typically use a command line tool for that that comes with RT but I can't speak enough of that software it's proved to be a really good issue tracker for us pain points I've had a few one of the things I work on as part of my job is we provide distribution images of various Linux distros or BSD operating systems that people can install on their VPS because we sell virtual private servers so I typically produce those distribution images these days I do the do the building and testing and so on and so forth a lot of that testing involves comparing data sets because I run images and I install them and do various things with them and make logs and I compare those logs against logs from previous releases so it's a basically a problem of comparing two large data sets and finding differences you know finding differences that will indicate issues with the release at first found it quite a daunting amount of data to deal with some of the differences there's a lot of repetition you know for instance the logs contain things that are not really useful and in fact throw me off like time stamps the time stamps will totally throw off any comparison I do so I wrote scripts for that help me deal with those kinds of problems I take the data and do some massaging of it with these scripts and then I could compare the massage data and that's worked out really well in practice one kind of diff problem that I haven't found a really good solution for and I would love for somebody to point one out to me if it exists but occasionally two files I need to do not just a line by line diff but I need to go intra line as it were I need to find differences with inline like I will give an example the other day I was comparing two files they had remarkably similar lines but they differed by one character and it happened to be a comma that was buried way down in the middle of the line several dozen probably more than 100 characters into the line I couldn't hear the difference it would be really wonderful if I could find a tool that would let me find those differences easier there's a visual diff tool called meld that's pretty good that can find those kinds of things but I haven't found a textual one other problems I've encountered occasionally I get a request where somebody will send me a screenshot support request and there'll be like my vps doesn't work here's what I see on the screen and I sometimes try to run those through OCR I haven't had much luck with that I'm hopeful but I haven't had a lot of luck with that maybe I just point out what the problem is and ask them if they could please send their output in an alternative format and we work our way through it but it's a little unfortunate that screenshots are so much of a problem kind of an example of something that was fairly easily fixed the BIOS on server motherboards I interact with those when I'm configuring a new server for instance and they're provided over a serial console one issue I had was some of these BIOS were printing ornamentation characters with the high bit set and that was really confusing either my terminal or my screen reader or both I could never quite figure out which we fixed a lot of that by just switching everything to the vt100 terminal type and so it's kind of nice when you can just change a simple setting and have things work yeah I think we touched a little bit on graphs any kind of graph data, time series data there's not really a great solution for representing that other than looking at the actual numbers that's probably the best we can do now it would be nice if some of that could be represented like in an audio form we've talked about different ways to do that perhaps using pitch or loudness of audio or beats per minute to represent changes in state for instance but nothing really concrete here and we would love to know if anybody knows of any work in this field I'd like to talk a little bit about overall some best practices either when looking for software that's accessible or when writing it the one thing I'll say is text is usually accessible out of the box it usually is pretty much 99% of the time so most of what I'm going to say here applies to GUI screen readers and to the web I would say the hands down biggest mistake people make accessibility wise is not labeling controls or user interface elements so for instance you'll be going through different elements and find an unlabeled button you have no idea what it does this is easy to fix and it's the most common accessibility faux pas and probably the easiest to fix I'd say if you're writing GUI applications you should look at the higher level toolkits like GTK and QT5 unfortunately GTK will give you accessibility on Linux it's not GTK apps are not accessible on Windows unfortunately that's really sad how it's often hit or miss choosing technology something we've discovered here recently there's a new technology that's being used to write applications called Electron and Electron apps are essentially web applications that are essentially written as web applications and basically they embed a copy of the Chrome web browser into the application unfortunately at least on Linux these aren't accessible yet I hope that will change but that's how accessibility it often seems like it's hit or miss and it's a moving target but for now I have to avoid anything written with Electron which includes for instance the signal desktop app I'd say web accessibility I'd like to talk a little bit about that it's not really my area of expertise but what I will say about it is there are standards here from the W3C there is a W3C web accessibility initiative sometimes called the ARIA standards and I want to make people aware that these things exist we have links to them on our slides I'd also say that in a lot of cases making your application keyboard accessible gets you most of the way there to making something usable by a blind person with a screen reader keyboard accessibility is probably 90% of it probably the ultimate thing to avoid I've seen a lot of web designers essentially re-implement standard HTML widgets slash controls using JavaScript so for instance I've totally seen people re-implement a select box using JavaScript that usually doesn't turn out well not for accessibility so I can't really say don't use JavaScript I can say please try not to abuse JavaScript try to use it wisely and with that I will send the floor back to Sarah OK so I'm going to talk a little bit about other disabilities just because well in this case we happen to hire somebody who is blind but maybe there had been something else so with dyslexia apparently screen readers can help with dyslexia I did not know this until I started looking but OK that's good to know but there's something out there to help people single handed input it looks like you can have you can buy a special keyboard for one handed input it's about $500 I think that there may be some work to do this with software too just have a separate layout for keyboardless input obviously there's voice input but if for some reason that isn't adequate there are mice for feet which apparently are actually geared towards dentists but you could use them for other purposes there's also a device called a sip and puff which you use with your mouth and you could use those to operate a on-screen keyboard or you could use it with there's something called Dasher which I saw many years ago and you can still install it but I'm not sure if it's very usable that's designed for faster text input then with color blindness I think most people do actually think about this but just in case you forgot use something in addition to color to indicate differences and then low vision I would say is actually you handle very differently from blindness in that you should let people set their own color schemes and then there's also typically built in magnifiers to the operating system and just to summarize if you haven't been thinking about it now is a great time to start thinking about accessibility because the things that you might do to allow an accessible client will help you with your other software and the hiring pool is really tight right now there's people are really busy so open up your pool get more contributors have more potential candidates all standard application tools the flavor of the month may not be compatible but there are things out there you will be able to find something that works programming even something that uses significant white space like python completely doable with a screen reader and if it was you have concerns about something other than blindness there are other affordances out there too so I would like to give a special thanks to Allison who is in the audience because I think she may have kind of helped move this through the people running scale to get us here Kate Diebel who pointed out to me that screen readers are also used with dyslexia and then a special thanks to the FOSC accessibility maintainers who are too numerous for us to list here and any questions so okay I'm just going to actually come down here and let people hold the microphone themselves so thank you I guess this question goes mostly for William as someone who is of course blind and using speak up and all that stuff the nowadays if you run into a website that you do well you can't even most of the time email the webmaster because of the javascript for one thing and second of all the who is that I used to use to get the domain owners address they've disabled most of that now it just gives you go daddy or some ridiculous thing that's not even affiliated with who they are so you can't even access it and then if you do they don't respond anymore back in the 90s it was very common you write them oh I can't use your website it says frame one, frame two oh sure use our no frames version they were very good about it now uh-uh so the other thing is PDF files or disaster whether you send them to Robo Braille to convert them where you convert them yourself the layout just isn't good enough oh those are all really excellent points I don't know if you had a question per say but those are those are really awesome points I appreciate you for making them more questions I'm just curious what it's like reading code as opposed to writing your own code if I read code I don't read it from the beginning serially I would maybe read the function signatures and the function names I wouldn't start just reading say the header files I guess I think when I look at a lot of documents it's very common that serially starting from the beginning and going all the way through to the end is not how I process them I wonder if you're just accustomed to processing really or maybe you have a way of skipping to the main point and browsing around the way people who have that choice to yeah I would say reading code linear like that starting at the beginning and ending at the end is definitely the exception not the rule even for me it's a very non-linear process with plenty of jumping around trying to read in an actual code base even for me that's a really hard question to answer solidly it's like you just do it how do you do it lots of jumping around am I very carefully very carefully I mean you know any screen reader or speech system is going to provide you with navigation commands okay I use I use regular expression search quite a bit I also skim I will listen to part of a thing and just kind of skim my way through it sometimes that ends up shooting me in the foot but I do a lot of skimming I do a lot of a lot of regular expression search so on yeah, other questions I just had a question about a program that's on Windows it's a Dragon naturally speaking and I was just wondering if there's it's used for speech attacks this morning if there's anything available on Linux now that you know like Dragon or something the program that you mentioned I think yeah I know speech to text does exist for Linux as a thing there's a thing called the Sphinx Toolkit I can't really claim much experience I think I did like do some of the packaging for that I helped with some of the packaging for it one time for one of the distros but I've never actually used it so I can't say what shape it's in I think you have to train it with samples from your voice or something but it's not really something you need if you're blind because if you're blind a lot of us learn to touch type but it is certainly something you know that would be indispensable for somebody with some kind of other disability that prevents them from using a keyboard blindness certainly in and of itself does not prevent that if I may add to the gentleman's question there's an open source project called Mycroft AI I forget the name of the speech engine speech recognition engine used in it but I've watched it in debug mode and it is just amazing how accurate it is at translating speech to text so you can start there and kind of drill down and you'll find it and that was actually the question I was going to ask you, William have you used Mycroft AI and if so, do you find it useful in any way? I have not tried Mycroft AI I probably should but no I have not I've heard of it and it's one of those things I just heard of recently in fact so I don't know how new it is yes any other questions in my years as a web developer I used to try to champion accessibility and it used to be easier because it's something that could be sold to the people who are coughing up the money for websites to be redesigned that if we made accessibility a priority then really at the same time we were achieving all the goals of SEO if we decided it was accessible to screen readers then at the same time being made accessible to Googlebot and other search bots but we can't really make that argument anymore because now Googlebot and other bots can parse JavaScript and all this dynamic content and I've tried putting the pressure on by talking about legal requirements in the ADA and the WCAG guidelines which spell out the prescriptive recommendations but you know short enough that a developer can sit down and read them in about an hour it's not like it's just TMI but the main problem is that no one wants to pay for it Fortune 500 companies who have enormous budgets for this do not prioritize it I've found maybe two clients in this category ever in my 10 years of developing who had any kind of consideration and these are companies like HP who have 400 page you know style guides if they don't have a document like that they will not even think about it Do you have any specific questions or just a comment I wish I did have you know some sort of you know ray of hope to offer but it's something that I'm worried about and this gentleman was you know expressing his concern about yeah it's a very good point you know I don't really know what to do about it it's part of why I'm actually standing up here on this platform giving this talk because I'm a really shy kind of retiring person but I want people to be aware that you know that accessibility is a thing and that I can do a job that I can do my job and I want people thinking about it and I don't have any answers but I I know exactly the problem you're dealing with because it's part of my lived experience Hi I was really surprised when you gave the demo of yourself coding it how quickly your screen reader was giving you information like it was definitely too quick for me to parse so I'm curious when you first started using screen readers were you able to use that quickly or did you have to get trained up and also just in general do you have any tips for if there are people who are having to use screen reader technology because of failing vision so that they're new to it do you have any tips for figuring out like how best to use it oh man I'm sorry we had this discussion earlier and I never mentioned and I have in several run-throughs of this talk but I've been blind since birth the first computer I used was an Apple IIe back in the early 90s and so I don't remember really how I learned to use this screen reader but it was a very gradual process with lots of just kind of training on the go yeah I've kind of grown into the technology for somebody who's losing their vision I mean there are tutorials out there but I don't know what this date of those are and that's a really good point especially for Linux I think we in the accessibility community and me because I have worked on some of these things some of these projects maybe we should consider doing some kind of tutorial series for people to help onboard them I think there used to be more of that some of that kind of faded away but yeah I learned through experience if that answers your question at all and maybe the next time we're here or the next video you'll see from us will be a tutorial on how you get started with screen readers so with that I believe that we're a little bit over time thank you so much everybody for coming and listening to us talk about accessibility