 So yeah, then I'll move on to my view of the history of the car and where it might be going. Next, I'll cover some of the challenges that we will face in this transition and some strategies and approaches we can take in order to allow us to do sustained and frequent software updates. And lastly, I'll get into some conclusions and a call to action. Now I am using Automotive as an example in this talk, mostly because currently I'm involved in a project related to Automotive. I also see it as being probably the biggest transformation that's happening. And a lot of what I'll be talking about though can be applied to other industries, whether it be telecommunications or personal devices or other things. So let's jump right into the cell phone. So this is what phones used to look like. You used to have one of these in your pocket, put up your hand, pretty much everybody. Now if you did a software update to one of these devices, put up your hand. So even though it's connected, people always say if as soon as a device is connected you pretty much have to do software updates. We carried these in our pockets for years, they're fully connected and we rarely ever did software updates to them. Moving to modern technology, the smartphone. I don't even probably need to ask, everyone probably has one of these in their pockets and probably everyone does software updates on them regularly. Some of us probably even seek out software updates, looking to see when the next release of iOS or Android might become available. Part of the reason for doing that is that we see value in the software updates. When we buy a device it comes with a certain set of features and unlike most things that we buy in life, when we buy a smartphone we can actually add value after the initial purchase. So moving from feature phones to smartphones was a giant leap for software when it came to what software was on a device. The feature phone had a very small set of software contributors. It was usually the device manufacturer and a small set of partners and suppliers. Whereas on modern day smartphone you have thousands of projects, thousands of developers. It's a much broader ecosystem and coordinating software updates is something that basically makes a modern smartphone what it is today. So this transition from feature phone to smartphone is a good template for what you see with other devices that are currently going through a similar transition. Another reason not only to get new features and functionality is to get your device updated to protect yourself against security. So this chart published by NIST as you can see covers every year that the modern smartphone exists and you can see that in any of those years there was always at least some critical vulnerabilities that happened to be reported. Now a few other notes about this chart probably the most immediate thing that people will ask is why is there a giant jump in 2017? It's actually a fairly unknown phenomenon. You can Google for this and there's lots of debate as to why this happened but there is no consensus. So if you want to submit a request for papers for a talk if you can explain why there's a jump in 2017 you'll probably get your paper accepted. Another thing that you'll notice is this chart only represents the year in which a vulnerability was actually published or reported. Vulnerabilities could exist prior to when they're reported. So despite your best efforts when you ship your software it's already got a vulnerability on day one. So there is a need to update your software and to do it in a fairly immediate fashion or at least to do it regularly. Now this chart does represent vulnerabilities reported in more software packages or projects than you're going to use in any product but essentially you can scale the y-axis according to how many open-source software projects you have. Obviously the kernel is one of the areas where many CVEs are reported so if you're using Linux and you have a kernel in your device you are going to be affected by CVEs and you do have to have a method in which to mitigate against them. Now I did want to try to find data on whether or not people actually update their devices because of security and it was actually a really hard task to do. As these pictures show there's lots of reasons why people update their phones, breaking their screens, they want to get access to faster networks, their better screens or foldable screens, different architectures or better batteries and batteries which charge faster. But even though I can't find data on exactly why people update their devices we can still look at some of the data that is available so we can apply that to the automotive space. So this chart from 2016 shows basically people that are going back to buying a new phone why they're buying a new phone being asked how long they kept their previous device. And as you can see here with Android phones the average falls into one in two years if you really squint it's two years and very few people are keeping their phones for over three years. This has improved since 2016 so I think the chart is slightly out of date but still we're talking about scales less than five years. Similar for iPhone, slightly better in the iPhone case you don't have to squint so much to see that people are definitely keeping their phones on average in the two to three year zone and again only a small fraction of people are keeping their phones longer than three years. So that's a very quick summary of what's gone on in the smartphone world. All of us again use smartphones so I'm sure there was no surprises there but let's see what happens when we apply that to the automotive space and see what kind of parallels we can establish and whether or not there's any differences that we should be aware of. So I'm using a personal example here to demonstrate where cars are coming from. So this is actually a recall notice for my 2014 Santa Fe and as you can see in the part that I highlighted in the box Hyundai will perform a software update to the ECU of your vehicle, the ECU in this case being the engine control unit. So essentially I had to drive my car to the Hyundai dealership where some person had to open up the hood, plug in a device, do a firmware update and after they did that I was able to drive my car home. This isn't the exception. This is the rule right now for the global vehicle fleet even though there are cars that are able to do updates now. For the majority of vehicles on the roads today this would be a similar scenario. So basically the software in your vehicle rarely ever gets updated. Often will be the same software running on the day you get your car and the day it gets crushed. This is a statement found in an article in the Wall Street Journal and you can sort of see from here where some analysts think the cars are going. It's a little bit out of date but anyhow I thought it was still a good quote. So car companies are trying to take a century old business model, make a car, sell it and hope the customer comes back years later to buy another one. So that's where we were and where we're going. They're increasingly developing vehicles as digital devices with the ability to remotely beam new services features to the car that can make it easier and more fun to use. So similar to your phone, when you buy your car it will have a set of features and possibly over the life of the car you might get more features that make driving better. A good example of that is Tesla Full Self Drive. They constantly push out new features and new versions of Full Self Drive and for people that like to have someone else drive their car that's seen as a value add and people will want to get that update and employ it. As I said that quote's a little bit out of date. This is some quotes from an article in 2021 in Reuters, Reuters, sorry. And it's in response to an press release from Ford and so here as you can see, Ford's already got a million vehicles capable of receiving over-the-air updates today and they also announced some in-vehicle technology stack that allows fully connected and updated software enabling automakers to interact with customers after the vehicle sale. So these two slides basically show where analysts thought things were going and Ford being one of the major car manufacturers is showing that we're already going there. Now just to address the elephant in the room. Yes, I know Tesla's been doing this for many years. I decided not to talk about Tesla in this talk because Tesla's very known for verticalization. Therefore I don't have many insights into what they're doing inside their software factory. So keeping this talk rather broad, looking at more of the incumbent auto manufacturers that tend to have a large diversity in suppliers and a large ecosystem in their software deliverables. So again if we look at the CVE, the Critical Vulnerability Chart and we apply that to the model of automobiles having more Linux in it. We have a larger proportion of the CVEs that are going to apply to vehicles. So CVE mitigation is highly important and again we are affected by that shift in when CVEs are reported and when they're actually in the software that we're shipping. Now that's a really important thing to keep in mind because there's different truths. So if you take a CVE in the kernel and you apply it to something like banking and someone is able to use a CVE to break into your bank, as a bank customer you may not ever be affected by that CVE because the bank will use insurance or other means in order to replace any funds that might be stolen. So that CVE even though the bank will see it as important, you as a user may not because you may not ever see it. Another case would be a CVE applying to your cell phone where you might lose some personal data or pictures so it might cause you personal injury to your reputational injury I guess. But again it will have, its total effects are fairly mild. Now a CVE in a vehicle can have life threatening consequences if it's exploited especially if the vehicle is in use. So the one thing that we don't get from the CVE chart is just what the impact of a CVE might be to you. And when you think about the fact that CVEs can cause bodily harm and that those CVEs might exist in your car we pretty much have to supply software updates to vehicles for the life of the car. Unlike a phone where Android 1 stops after 3 years, most iPhones stop getting updates. I think the maximum is 8 years for iPhones, some devices. The vehicle is going to have to have coverage or updates for the entire life of the car. Now speaking of life of the car, we do actually have very good data on that. So this is data from the European Automobile Manufacturers Association. You don't have to read the text in the top. I've highlighted the important bits here. So for a car in Europe, 11.8 years is the average use or lifetime that it's on the road. For a van, 11.9 years. And for a truck or a lorry, that's 14.1 years. So comparing that to your cell phone, where we were sort of maxing out at around 3 years, we're talking significant longer lifespan of the software inside a vehicle. And add that to the fact that we have to keep that vehicle up to date for the life of the car. We're talking significant years of long term support. Compared to even some many embedded devices that I've had the pleasure to work on, this is a long period of support. Linux is, what, 30 years old now? So it's overall for a vehicle. When you think about this, we have to go up to 20 years. It's a long time. So what are some of the challenges we're going to face in doing this? So some of the main software challenges, as you know for something like an automobile, they have to start. It has to be safe. It has to be secure. And again, that has to be from day zero through to the day that the vehicle gets crushed. We must have long term support. We have to deliver new features and services. Basically, this becomes a competition. Proposition, as I say, companies like Tesla are giving their buyers additional functionality and value after they drive the car off the lot. Must run on a diversity of software. So obviously we're not going to be writing software for just a vehicle. So we might even be running on different architectures or different platforms as new technologies because you might be writing software today where the actual device that you're running on may not be invented or produced for another few years. And finally, we are going to be working in a fairly large ecosystem, again with Linux. As soon as you start producing a Linux-based device, you're going to end up using quite a large number of packages. And then the other challenge is the lack of precedence. So again, the phone is an interesting precedent, but it lacks some of the longevity and commitment from the device manufacturers to go for the life of the device. Now, we can change how we view some of these challenges, so must have long-term support. Often when we think about that, we think that we're supporting software from day zero to the last day of the device. However, it just means we have to support the device for the life of the device. We could actually be supporting new software on an old device. So we can approach these things differently. And again, over-the-air updates is a way to achieve this. Some of the other challenges, if for anyone that supported a device over the long term, you end up having different demands on your development team or actually becomes typically a development in a sustaining team. And when you do that, you always have competition between the development or growth and sustaining. Usually there's more interest in developing new features and less interest in sustaining them. You have to worry about open-source software, either the project or release is becoming end-of-life. An interesting example of this, it's fairly recent, was the deprecation of Python 2. This will be something that we'll probably face where Python 3 becomes deprecated within the next 15 years, or possibly it will go on. We just don't know. So you have to be prepared for open-source software to have releases or projects to EOL. And it's constant, you know, like move from IP tables to NF tables. There's always some of these transitions going on. There's also different OSS release models. So you have branch models, like what the Linux kernel uses, where they'll have an LTS that they'll support for anywhere from two to five years, sometimes longer. And then you have what I deem or what I've seen called channeled, where, like Chromium, where basically it is a linear release. They don't have branches, so you end up getting features as well as bug fixes merged in the next release. And then there's what I call value shifting. So basically value shifting is when corporations basically move from wanting to support old devices and instead focus on new devices. So the value shifts from wanting to keep their customers, previous customers happy versus chasing new ones. So those are some of the challenges. Let's look at some of the strategies that we might employ. So design for change is a architecture and software model that I've used throughout my career. I was fortunate enough to have a professor who is well-known for his view on design for change, Dr. David Parnas. And he originally started, and this was before object-oriented programming, he was tasked with writing software for a fighter jet. And they weren't sure what hardware they were going to get for things like radar or other sensors. And they decided what they would do is isolate the code around those changes so that they could, regardless of what radar it was, they would still get the data in the similar format. And this allowed them to swap out one piece of hardware for another. And it resulted in object-oriented design or otherwise known to a certain extent as design or build for change. Now, in order to build for change, you do have to plan early, identify things that might change. You're not going to catch them all, but you can catch many of them. And then you want to isolate your software around the areas that might change. The one rule is that change will always happen. And again, we're talking about 20-year lifespan for vehicles, so that's pretty much a given. And then what we do want to do is write software in the future that we can run on cars that are being released today. So having software target multi-generations of vehicles. Some of the tactics you can employ for design for change, you want to make use of standards. So there's informal standards like the kernel ABI. Linus doesn't like to change the kernel ABI, so it tends not to change very often. Formal standards such as POSIX. Libraries, depending on the library, you might have to evaluate whether or not it's a long-standing library or if it's new what the chances are that that library is going to continue to exist or whether the API that the library is going to expose is going to change or remain constant. Another approach is to decouple modules. So making use of RPC or messaging. In automotive you might be making use of some IP, which allows you to separate functionality and decouple modules. Another approach is to have external configurations. So write a software module that might target various scenarios and that can be controlled via external configurations. Taking advantage of template and observer patterns and finally exploiting the language capabilities depending on which language you're making use of. Another big one, which is something that I've been looking more and more into recently, are declarative systems. Declareative systems being operating systems that are defined by code. So typically if you're downloading and installing Ubuntu you're then going to make modifications to the installation after the fact. A declarative system you're going to have all of the modifications captured in code. NixOS is a fairly well known declarative system but there's quite a few more. The ELISA project just announced one during a conference a couple weeks ago. It's useful in that again it makes you think about change and you will capture that change in the configuration. Most declarative systems will be reproducible and minimal change depends on again what you're doing but because the declarative system requires you to go through a formal code review it's probably going to make you think and really validate what you're changing from release to release. They're easily tracked. Again it's in code so you can put it in git. Minimizes upload bandwidth. That one's a little bit shaky but again you're going to be making changes deliberately and not be affected by changes that are happening out of your control. You can isolate your changes by combining how you define your declarative system. You might separate out the hardware component versus software which has no reliance on hardware. Another thing that it does is it eases rollback. Basically you should be able to roll back just as easily as you roll forward. Another one, robust update frameworks. I was a little bit disappointed being in almost the last time slot for the conference but actually I think it works out quite well. There's been some amazing talks at this conference on update frameworks like ROC and others and I think this category has actually come along a long way in the last year and it will obviously be the bedrock for any update system. It has to be a robust update framework. In automotive some of the challenges might be a little bit larger than the projects presented here. If you've attended any of the discussions about AB updates or ROC most of the presenters have talked a little bit about hypervisors or mixed software where you might have a real-time OS providing safety part of the operating system or safety part of the overall system. The update systems are coming along nicely but they might have to be extended to be fully operational in an automotive environment. Another area where you can put some thought towards is where you're going to target your software. Again the kernel ABI is fairly rigid however the kernel itself can have significant changes so things like drivers or code that you keep inside the kernel is probably more likely to change because it goes to software which you're writing for an SDK with a fairly stable API. Again it depends on the SDK that you're selecting but SDKs tend to be more stable over time than kernel drivers. So you want to target your code to the right area of your system and if you have to split them if you have something that does some kernel work but does user space make sure you do a proper division of the two that way there you can coordinate your software and isolate your change so when you do have to change the changes are limited. So finally we'll jump to some conclusions and a call to action. So I think it's pretty unmistakable that vehicles are becoming digital devices. Every new car I get into I'm surprised by the giant screens and the functionality and that's only on the user facing part of the vehicle. Inside the vehicles it's changing just as much. Again I'm just starting to get more involved in an automotive project and it's amazing what's going on inside the ECUs. If it's a digital device or any of your digital devices they're going to have to remain safe and secure even when you're doing software updates they have to remain safe and secure from day one to the last day of their existence. And delivering new services and features is going to be an important mechanism to ensure that we continue to do software updates. Finally software should not reduce the vehicle lifespan and again software written years from now should be able to run on vehicle ship today. It's the only way if you think about every few years there's going to be new vehicle models there's going to be more old models that you're supporting than new models that you're just about to release so you really do want your software to be portable across generations. Design and build for change can assist by allowing modules to be reused time and place so obviously time it would be across multiple generations and place could be in vehicles with different SOCs or other hardware. Reducing code change when it happens so if you've isolated an area correctly that changes when that change does happen hopefully it results in less code to be modified you reduce testing those modules that don't require updating when change happens usually only require a lighter testing regime than the areas that do change selecting standards to assist in design building for change so again standards and libraries any interface that doesn't change is a good approach to not having to change a software and robust update mechanisms again an area that I think we're in good shape but there is some evolution that still needs to be done there and all of the above will make software updates easier and possible in the future so finally a call to action so as drivers passengers, pedestrians we all have an interest in vehicles remaining safe and secure as attendees of this open source summit we all have an interest in open source software how can we all combine those two interests to ensure that this endeavor is successful and with that I'll open the floor to any questions yes it's not guaranteed that software is going to get bigger and bigger with the declarative systems you do have control over well first a lot of software that we use has been around for 30 years right so like grep for instance that has been in existence since the early days of Unix so the 1970s right a lot of software will be in that sort of vein and I don't believe that grows or shrinks I've been working with Linux since late 1990s my systems have definitely gotten bigger but most of that's been related to desktop Linux most embedded devices now have significantly grown in the amount of memory that they do have and I would be surprised that you would have enough growth to outgrow RAM or storage using some languages like Golang if you were to build it including libc for instance yes if you run more software you are going to need more space but it's you can still reuse components even if you're running low on space on an old device that say that's released today you don't have to run all of the new software that's becoming available you can still limit it to the size of your system I guess what I'm saying is you're not going to go wild and say that a system 15 years from now every component you can run on a system that's released today you'll have to have some discrimination as to which software you're able to bring back versus software that you're not going to allow so when you had android and you had 16 gigs of memory on your phone you were fairly keen to move to 32 gigs probably but when you went from 32 to 64 was that it became less important as memory became cheap enough and more available so I guess what I'm saying is when android was moving up from 2010 to 2015 there were significant jumps in the size of storage those jumps aren't as important any longer they're still important but they're not as significant again that's just my view thanks for addressing this topic I think it's important that the user and also the developer to move on I hear a bit of a problem from like the phone analogies that a company may not have interest to support all devices since it only produces code for the company and people don't buy new products do you have any ideas about that or are you good at it it's tough as I was writing this talk I thought when I wrote this synopsis I thought I could talk about this and I can for like 8 hours 9 hours the hard part was getting it down to 40 minutes so what I was trying to convey is as developers if we can if we can show value in doing updates then the corporations will continue to do updates if value disappears from doing updates then they're going to want to stop and I think there's a few things that can help with that one is legislation again the CVE in a vehicle can potentially be life-threatening so I think regulation and legislation will have to play a role we can't have like Samsung stopping doing updates after 3 years you know I carry a phone which no longer gets security updates and I know it bugs me but 99% of the population it won't but I think there is a need for us as technology people involved in technology to help push for rethinking and getting corporations to commit to their devices long term I actually had an interesting debate with a colleague that basically said with so much technology going into cars we'll throw them out after 4 years and that just my heart just breaks thinking about it especially seeing as I think they had a publication the average price that people are spending on a car for a long time was around $25,000 and it jumped during the pandemic to $40,000 I can't afford $40,000 that's $40,000 a year for a vehicle plus all the other another approach and again I could talk about this a long time another approach would be put a car on a road and instead of it being in your parking spot for 90% of the time use it as much as you can and wear it out in 4 years that would be okay but not the way we use vehicles today so yeah it is a big challenge I would be nervous if I was buying an EV spending $60,000 knowing how much technology is in it and whether or not that's going to receive updates 15 years from now sorry Joseph you were first I'll get you next Joseph you mentioned your vision for declarative systems so nerd question now wouldn't you consider the opening beta slash Yachto stuff a declarative system? it is, I think it has some shortcomings though some? many much configuration is the tough one so Yachto does a great job I can define exactly what's going into my image in Yachto doing configuration like adding users or adding other setting up a container runtime with default containers and stuff like that so Yachto has a ways to go but I have thought at some point again the declarative systems is something that I've only been thinking of recently so for a long time I was more of the mind of like CoroS having a mutable operating system and then adding in via containers or other mechanisms but I think that approach is a little bit it is and it's difficult to work with whereas a declarative system gives you the opportunity to make modifications but at least you get a lot of the benefits of a mutable system but you do have the flexibility to make the changes when you need to but yeah Yachto is I think it's a really good start to look at having an alternative mechanism instead of local.conf and bblayers and doing that in another format I'll say YAML if people want to beat me up or Jason but I think that would be an interesting add on to Yachto to be able to do that so that you can have fragments and then assemble them and then use that to control your build and your output it's it is relatively new I'm still mulling through how best to work with it and eventually get to something that I could propose to Richard and get into Yachto before I run out of time I better ask yeah so again so I'm a architect and a technologist that is just being bought out by an automotive company so so my expertise is on the operating system side in the presentation you'll see that I didn't separate applications in OS it was a decision I had to I opted for this route I know it's not ideal if the talk was twice as long I could have split them better but yeah now that we will be owned we're just waiting for regulatory approval so we were bought by Aptiv and so I am getting insight into how automotive industry works and that will accelerate as soon as the deal is complete and I hope as as an operating system vendor to be able to take my whole goal is to take any technology that has more than one user and put that into a product if there's a technology that a single user is doing that's sort of bespoke there's not a huge value in getting that into an operating system that we sell to many customers but as soon as something's done twice then I like okay that's something that we can put in the product and there's value so I'm hoping that our relationship with our new owners will give me more insight into that and I can help expand our product and contribute to Yachto from an automotive perspective we are working with Ford already so have a little bit of insight there but I'll get more insight with when I'm working with Aptiv directly like we got one minute so one more question or we're done and people can go to the closing game alright, thank you very much