 Hello, everyone. My name is Tim Bird. I'm a Principal Software Engineer at Sony Electronics. And welcome to this session of Embedded Linux Conference. I'm going to be talking about issues with open source license compliance in consumer electronics. So I usually like to put the abstract inside the slides in case people come along afterwards, but you can see that online. And this is the agenda that I'm going to be talking about today. I'm going to go through a little bit of the legal disclaimer. It's important for a talk like this. I'll talk about aspects of compliance and give a little bit of the license history. Talk about some of the gotchas that you may encounter as you're dealing with compliance and then give some of my own personal recommendations. So let's start with this disclaimer. So first, it's important to recognize that the law is way different than code. And there's a lot of armchair lawyers out in open source, but it's important to remember, well, so one, I'm a software developer. I'm not a lawyer and I'm specifically not your lawyer. So there are a lot of unexpected things in law that unless you have legal experience, you're not going to know about. So don't trust what you read on the internet or even what you hear from me. People will make strong statements about what's required by licenses, but it's just their opinion. The same goes for this presentation. There are actually very few legal court decisions that have been made about GPL and the language of the GPL in a couple of places, it's a little bit ambiguous. And I don't know of any open source license that includes a choice of law, which means that it might be interpreted differently in different legal venues. And I know for a fact that German law in terms of licenses and contracts is a little bit different than US law. And so there's unexpected things that can occur in different jurisdictions. So you should always check with your own company legal counsel when you're making decisions that you think might have some legal risk. And it's really important for me to let you know these are my views. I am not speaking on behalf of Sony Corporation, even though I've got the logo on my slides. These are my viewpoints. And so let me get started. Part of the reason for this talk is because I do have some pet peeves. I've been doing Linux for over 25 years now, and I hear this a lot on forums and stuff. It's trivial to comply with the license. Just post the source code. What's the big deal? Well, it's actually not as trivial as it sounds. So that works when you have an in-house product and you're in complete control of all the source top to bottom. And that's true for some products, but there's a lot of other products where you're getting components from other places and just doesn't in particular with large products like a mobile phone. I think the last time we, there's one product at Sony that I measured the code on. We had 58 million lines of code that came from multiple vendors. In case of a mobile phone, you're talking about code from Google, MediaTek or Qualcomm and Sony. And there are a bunch of others in the supply chain providing software for drivers and sometimes black box components that go inside the product. So it's not as simple as having all of the code just in-house. And there are some business units that do black box or what we call white label products. I'm aware of some old security cameras that were just relabeled by the vendor and they'd come from a completely separate vendor. And some business units are not in the software business per se. So Sony B2B is a business unit that does IP and AV suites for stadium packages for stadium suites. And so they do not have the same level of expertise as like system level developers inside some of the other Sony product teams. So it's much more complicated than people kind of assert. But having said that, let's talk about what is required. So these are the aspects of license compliance. You have to know what's in your product. I'm going to talk about what types of things you have to provide and some of the requirements around those. So let's get into that. The first thing, and this is really important is you got to know what's in your product. If you're dealing with a large product, by large, I mean lots of lines of code, lots of components, then you can expect to maintain a spreadsheet possibly with thousands of lines of rows. And most people will look at a package, the list of packages on a website and say, well, there's not that many packages. It's like, well, but there are that many files and the license granularity is not necessarily at the granularity of the package. So you have to know what's inside your product. There are tools to assist with this, both commercial and open source. And you can go look at those. There's lots of resources to find out about those and how to use those. I have found, in my experience, that most often the problem comes from suppliers. So there are some suppliers out there who do not take their responsibility seriously. And there is a working group in the Linux Foundation called Open Chain and they focus on education and certification. So there's a standard and it's actually an ISO standard. It's either it's already been approved or it's on track to be approved, but for open source processes within a company. And this is really important because as products become more complex, you're getting software and hardware for more and more suppliers. And all it takes is one supplier to kind of screw it up in your supply chain. And so we're, as Sony, we're a platinum member of Open Chain. We're very interested in seeing that this education and these processes and methods that improve open source compliance are used throughout the industry. I think it helps not just us, but it helps helps the entire industry. So the first thing is you got to know what's in your product for it. In order to do this, it's very helpful if you can actually build what you get from your supplier. So that is interesting. Sometimes the software is embedded. And so deeply embedded. And so it's not, you can't do this 100% of the time, but it's useful if you can validate the source that's provided to you and that that matches the binaries on your component. So some of the, one of the things that you have to do is you have to provide what we call attribution. Most licenses, even, even permissive ones like MIT or BSD require that you give someone notice about the presence of that software and possibly the copyright or the authors somewhere in the product text. And some, some licenses require this to only accompany the source and others require that accompany the product. Well, and so it used to be that we just put stuff on a paper manual. But as we've been used more and more open source, it's gotten way too long for that. So here's some of the requirements for different licenses. Different licenses like MIT or BSD requires stuff either to accompany the source or they require that it accompany the binaries, which means the product BSD for actually required that you give notice in the advertising materials. That is a super obnoxious clause that the, actually, the whole industry kind of worked hard to eliminate that. I don't think you'll find much BSD for licensed software out there, but that's an example of, you know, there's a big variety of places where this these requirements required you to put place, put things. And so in order to simplify things, most companies will just gather everything, the licenses, the attributions, the copyrights licenses, and they'll put it on a single page in the device. And that's accessible by a menu item. And so if you look on your own phone, I have an Xperia phone here. So if you look under about and legal information on your phone, you'll see this huge long screen. In fact, the scroll bar I think is only one pixel big is or two pixels. It's hard actually to get to the scroll bar, but and, but you have all this information it's included with the product. Now, so what happens if you have a product that has no display. And there are some products that have no paper docs either. And there are even some products that have no display no paper docs and no packaging. So what people do and I'm not, I'm not going to make any judgment about whether this is sufficient or not but what people have seen people do is they put a web link, either on the product packaging if it has packaging or if there's no product packaging I've heard of people describing a web link on the product itself. So, I think in that case that's about as well as you can do. And then of course at that link whatever the website is you have to keep that website up and you have to have the information, all the relevant information there. The other thing that you have to provide is you have to provide source code, or an offer for source. So the GPL gives two options for this so let's let's talk about those two options. So, one of the things you can do is provide the actual source code with the product itself so the GPL says accompany it with the complete corresponding machine readable source code on a medium customarily used for software interchange. Some of your options here you could include physical media like a USB stick or CD ROM and I've actually seen examples of this and, and those are fine some some products that you can actually put it on the internal storage in the product itself. And this is an interesting option it does solve some issues. It makes certain other things easier so you don't have to create a download site you don't have to keep it populated for a period of time. You don't have to do physical fulfillment after the fact you once you've done it in the product you don't have to. Yeah, I don't have to keep doing it in the in the future and you don't have to give the software to anyone who asked for it you only have to give it to your product recipients is kind of baked into your product delivery. The reason that this is not used very often there's a couple reasons one. There are products that don't have sufficient internal storage. Right, we are talking about embedded products here deeply embedded. But even, even then, if you're going to do firmware updates, you're going to do those over the air. You can't be shipping a USB stick or a new CD ROM with every firmware update their size issues, even if you wanted to kind of include the source with the CD update so ultimately, ultimately, this is a solution that is not used very often in the industry so the other solution is to provide an offer for source instead of the source itself you provide an offer. And the wording of the GPL here is a company with a written offer valid for at least three years to give any third party for a charge no more than your cost of physically performing source solution, a complete machine real copy of the corresponding source code on a medium customarily used for software interchange. And that last part is kind of key. So most community members feel like an internet download site is sufficient. However, I've talked to a fair number of lawyers and most lawyers believe that this should be interpreted that language should be interpreted as the medium used for software interchange in 1999 when the license was written. So offering only on the internet is not sufficient. So an offer of a CD or something up maybe a mag tape. I don't know that be rude is is required. And then I want to take a little segue here. So it, but it varies so different venues may interpret this differently. So the I already said the licenses don't include a choice of law. So something that may be made a decision in one country may not apply in another country. But if you're dealing with a large multinational corporation which I am with Sony. You have to look at all the different countries. The license can be interpreted in different countries actually in Germany there is case law there was a court case said you must provide the option to obtain physical media. So there's a legal precedent there and German courts have taken the GPL seriously, including details about the source offer so you have to be careful. I just want to take just a tiny bit of a segue here or a detour certain effects of or features of German law were used by one litigate to gain a lot of money with a particular legal strategy. So, and I'm not going to go into the details of that. I have a resource page at the end of these slides that has information about that case it was the litigate was his name was Patrick McCarty. And the bottom line there is don't sign any consent agreements or compliance agreements in Germany without consulting a lawyer and reading up on the strategy there. Anyway, I'll just leave it at that. So, there are places where you need to be careful with your interpretation and your fulfillment of the license requirements. And then, even within a single country, you can have different contributors to a project who interpret the license differently. So, so it's, it's, it's not as clear cut, you know, fulfilling all the requirements is not as clear cut as a lot of people say. In terms of duration, so how long do you have to offer the source code well, it's not three. So you have to offer it from three years for the last shipment of the binary. So you're distributing a binary to someone and it's three years from from that date. But that's not the last shipment of the product if you do over there updates you're going to be providing updated firmware, and that's going to be another distribution of the binary and it and it doesn't mean three years from the date that you last put the firmware on your website. It's three years from the last from the time someone last downloaded the firmware so the firmware could be up there really long time even after the lifetime of the product. You have to keep providing software so that people can obtain it within three years after so this is usually however after having said all that. It's usually not a big problem. There's virtually no incentive for companies to take software off their sites and stuff just hangs around a long time so technically you could be removing stuff. But most people don't do it so the duration is not that big of a deal. And then I'll just make one observation. If someone is asking you for physical media. They are most likely a person who has more interest in your compliance and how you're doing with that then they do with the source because if your source is out on the internet site that's easier for people to obtain. So I just a red flag handle requests for source promptly and carefully. So now to the source itself what must the source include so for the GPL license which is the main license that requires people to distribute source says for an executable work complete source code means all the source code for all modules it contains plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. So there's a term used elsewhere in the license and by most lawyers there's an acronym that's used complete and corresponding source code or CCS you'll hear that for short. So, the GPL also says the source code for work means the preferred form of the work for making modifications for it so that means you cannot deliver obfuscated code. And it needs to be something that actually can be worked with by human in terms of modifying it and rebuilding it. And it's unfortunate but I actually have seen examples of supplier providing obfuscated code. I'm not going to name names because they, they came clean and and and provided an obfuscated code later so we're, we treat that as an anomaly. But you, you should not do it should be something that some developer can actually use now about rebuilding so you does the definition of CCS require that a recipient can reproduce binaries exactly. And I would argue no. So it's very difficult to read reproduce binary in exact form in fact there's a whole, there's a whole. Not industry but there's a whole movement to try to do reproduce builds and this has been a source subject for decades so I think that's too high of a bar to require of a vendor. Does it does the definition of CCS require distribution of compilers linkers ID user proprietary tools. No, it does not you don't have to provide the compiler. You don't have to provide the any proprietary tools. It does raise an interesting question and in my opinion and I'm not going to talk about this a lot but is it compliant to use proprietary tools to build open source software so can I use a commercial tool chain. But you can keep doing this thought experiment for longer and say well is it compliant to use tools that are unavailable to the public can I make my own tool and use that. And I'm not even going to get into that I don't know. It's never been tested to my knowledge and people don't actually even talk about it. But having said all that my view is that what is required is the scripts used to control compilation and installation. And this means auto tools configure scripts config files and makes files and that's sufficient for most packages. Now another issue that comes up sometimes is reinstalling. Because the license requires scripts used to control that installation. It has that phrase in it some people interpret that to mean that you need to be able to install the software on a particular device. So license does not actually say that doesn't mention the act of installation or the target of installation it just says the script used for installation. And I'll talk a little bit about that so industry practice. In fact fully supports the use of GPL software in read only memory or ROM. So you can put GPL software in a ROM where it cannot be reinstalled in place on the same device that it was delivered on. And that's been a practice that's been supported and and there are even quotes about GPL reinstallation not applying in that case. So I don't think the license requires reinstalling. So if you go back and look at the license history. It's pretty interesting so BSD licenses kind of came into effect around 1988. GPL actually v version one came out in 89 and it was generalization of the Emacs license that Richard Solomon had been applying to some of the software in the GNU. Sweet or GNU project. And the history is pretty interesting. GPL v2 was released in 91 and was adopted by Linus Torvalds and then GPL v3. There were some perceived flaws and in GPL v2 and Richard Solomon and others worked on a new release of GPL that came out in 2007. There was a long discussion period on that several years. And for very a variety of reasons, including not really agreeing with some of the intent behind the license. GPL v3 was not ever adopted by the Linux kernel. And so I want to talk here about interpreting the license. I think it's important because at this point people will start talking about, well, what is the what is the intent of the license and first the overriding legal principle is that the intent only matters if the license is unclear. So license does not have its own tent. It is not a sentient being license is an expression of the intent of the copyright owners of the software and it's not. It may reflect the intent of the author of the license, but that's not the person who is making the agreement. It's the copyright owners of the software who are making the agreement. So the problem comes up with Linux and with other projects that have multiple authors as well who whose intent matters. Is it the leaders of the project? Is it anyone? The Linux kernel now has about 14,000 contributors. So I'm sure that there's probably a bell curve of different intents for how the license should be interpreted. And but I will I will say this. So Linus Torvald's intent for the kernel is very clear. He has said on numerous times that the intent is tit for tat and that expression means that I give you something you give me the same thing back. So if I give you source code, you give me source code back. And again, that's been Linus has said that a number of times and a number of different venues. And I think that's so I believe that that is his intent. I've seen other leading kernel developers agree with that interpretation. So what are the license requirements? Again, this is my view. I'm not speaking on behalf of Sony Corporation. But my view is you give people the source, all of the source, including make files, configure scripts, config files. Customers should be able to build the software, but there's no requirement to exactly reproduce what you've built. And there's no requirement to reinstall on a particular device. So if we have if if a vendor has made fixes to bash and you think those fixes are good, you can apply them to your version of bash and use it on your desktop or anywhere else. If we've put a driver in for for a chipset or a feature on a chip, and then you can take that feature and use it on the phone of your own making. So vendors are not required, in my opinion, to produce hardware that can run binaries that are produced by third parties. So bottom line, I believe secure product lockdown is allowed by GPL v2. And I don't think I'm alone in that requirement, interpretation of the requirement. So GPL v2, GPL v3 came about because of because of that interpretation, I believe. So if you look at GPL v3, really the requirements there sought to impede lockdown and change the terms. And GPL v3 does actually require keys to install the software on certain products. And most CE vendors, including Sony, strongly avoid GPL v3 for this reason. So there are a couple of different views about that. One is that the GPL v2 had that intent all along and the changes in GPL v3 are just changing the wording to strengthen that. But another view is, well, if the GPL v2 incorporated that requirement, you wouldn't need to change the wording. So the GPL v3 is a different license. And there are some people, very clearly some people who have been vocal that that was not their intent to disallow lockdown. So take that for what it's worth. And the final thing here is that many projects adopted the Linux kernel and GPL software based on the wording of GPL v2 and not of GPL v3. So you can't hold them. I don't think it's fair to hold them to the standard of GPL v3 or to the requirements of GPL v3. And even Richard Stallman has said it's evil to change the terms of the deal and expect people to abide by a different license than what they agreed to. So now I want to talk a little bit about gotchas. There are some nuances that can come in and bite you. So what happens is languages change over time and projects change over time. And these are two sources of things that you really have to watch out for. So MPL stands for Mozilla Public License and over time it changed its wording slightly. And so as a vendor, if you're using MPL software, you need to pay attention to the version that's on the license that's on the version of the code that you're using, the version of the license on the version of the code you're using. MPL 2.0 has a complete and corresponding source code wording that's more similar to the GPL and MPL 1.0. Also, well, a similar issue with GPL v2 to GPL v3. So there are packages out there that are widely used packages. And these are, this particular one is owned by the Free Software Foundation that was originally written under GPL v2 but then switched license to GPL v3. And it's not the packages themselves, you can go back and find those old versions of the packages. But the interesting issue comes up with what about patches to that project? So for instance, there's a very interesting and useful patch to plug a security vulnerability in Bash called Shellshock. And the patch for that was submitted simultaneously by the same person or same organization to version 4.3 of Bash, which was GPL v3. And version 3.2 of Bash, which was GPL v2. So what is the license of that patch? Well, I would say it's the license of the project it was contributed to, but it gets more complicated than that. So you can envision places where people are extracting patches from versions of the software that have a different license and are they, you know, can they be accepted in back ports. So there's a lot of factors to that. That's something that you really need to look out for. So just be really, really careful in handling of any packages that are GPL v2 that you know have switched to a different license. And then there's another example. There are other examples where it's not, it's not trivial to figure out the license for all of the outputs from a package. So for instance, in Util Linux NG, it's mostly GPL v2. And if you look at the copying file that's at the top of the source code directory that says GPL v2, I think it may now say GPL v2 and some other stuff. But you have to go look at the headers that are on individual files because there are a couple of files that are actually GPL v3. So you have to be paying attention and looking closely at this stuff to make sure you don't get caught. So this is, this is the part of the presentation. Well, I'll just unlock ability in my opinion is not required by GPL v2, but I think it's a topic that a lot of companies, the E companies are interested in. And there's a lot of discussion in the industry about it. So I'm going to talk about it anyway at at my own legal and reputational peril. So you can have, there's a spectrum of unlockability, right? So you can have a fully unlocked product, a not unlockable product, which is a lockdown product to put in more positive terms with a double negative. Or you can have something that you can unlock and then reinstall the OSS only. So let me talk about those. A fully unlocked product is something where, okay, and I'm talking about kind of a specific class of products that have the ability to stream content and or and or the ability to connect to a back end service. So I'm not talking about some little iot device that doesn't have a whole bunch of other considerations that's like I got 100% open source stack that's that's not what I'm talking about. I'm talking about a product with a mixed stack consisting of proprietary and open source software. So a fully unlocked product would mean that that you, you can reinstall all the software and retain all all the keys and connect the back end services. Okay, so first, this is not required by any open source license, including gpl v3. So open source licenses only cover software that is open source. It doesn't cover the proprietary software or the or the streaming keys or the back end keys. Those are not, those are not part of the open source. And for some products, not for all of them, but for some products, it's simply not supportable as a business proposition. And so the example I'll give out here is video game consoles and network play. So if you can unlock the product and you can alter the operating system, then trust me, you can figure out a way to cheat in online gameplay. And that's kind of the reason that people do this. And so that is an example of a product where is the the ecosystem relies on there being a protection of the capabilities of the individual nodes on the network. Or to make it fair for other players. And so that's part of the reason people buy the software is when they do network play, they don't want people to be playing unfairly. So it's actually kind of baked into the whole business model and and what customers want. So a fully unlocked product, I think is unrealistic. It only works in a world of perfect and honest actors and that is unfortunately not the world we live in. So let's talk about the opposite end of the spectrum, which is mostly companies locked down the product and just don't look back. And so you can argue whether that's good or bad, but the security for this type of thing is often done in multiple layers. You've got encryption for the update images themselves, the things that are transferred from the back end over the air to the product for updating you've got on the device you've got trusted execution rings and trusted protection modules. They're holding the secret keys and stuff you've got encrypted file systems. And then you've got kernel module application signing so everything's verified before it starts. So these products are really locked down hard, because the industry has shown itself well certain as sectors of the consumers of these products have shown themselves to be pretty agile at breaking in and and doing stuff. But there are counter examples right so I know that there are some experienced phones that are unlockable and some Google Pixel phones not all of the phones but some of them. So it is possible to do some level of unlocking you don't have to lock everything up. And so this is a product. I'm going to describe here that allows you to unlock the product and reinstall only the open source software. And a couple of notes about this so this does give consumers the ability to modify install modified open source software. While at the same time protects the interests of the content industry and consumers who want to stream content. If you if you anyway so there's no obligation to make proprietary software work with open source software that's not in any license. So what that means is may require the removal or disabling of proprietary software. What that ends up doing is so the device once you've unlocked it like this you may lose quite a bit of functionality, depending on the stack it's heavily dependent on what items are in your stack. So the customer the bottom line is customer demand for this is very low this is not your average consumer that's trying to break into their TV set or their phone or, or whatever or their streaming device. But because the customer demand is low it's a very hard sell to your business and so as there's just not enough. You know customers to make it interesting to do the difficult engineering work involved. This can be a one way action. That's most often how it's done is that once you unlock the product you can't relock it again and get back those proprietary software those content keys. That is actually technically very very difficult once you've removed the content keys there are some places in those layers of security where right once operations are performed so you can only. You can't actually rewrite the encryption keys for some flash devices for example. So, okay, so having said all that. Let me let me just dive into what I recommend. So this is kind of the summary wrap up portion here. These are the kind of main things that you need to think about. And what I recommend you do so first in terms of communication. Please please please communicate with your upstream suppliers and do it in advance don't do it as an afterthought. Get your suppliers involved with open chain the open chain project make sure they have processes in place. I is sad but I'm aware of several major vendors who have had what I call OCRUD moments where they found out a supplier was not giving them the right stuff, or was not giving them any stuff and and and that's a really awkward and difficult situation for a vendor to get into. So some of the ideas I have are. You can put penalties in place in your procurement contracts again this is for kind of large suppliers who have procurement contracts for you can put penalties for failure to disclose open source software or you can require permission in advance to deliver components with open source software there's a couple different ways to deal with that. I think it's good if you can get a license manifest that talks about all the files on your device and try and get the license for every single file. So don't do don't don't do this as an afterthought don't have this be during your QA of your product you're assembling all this information try to get it up front. Another thing that you can try. I don't think very many companies do this, and I'm not going to assert that Sony does it in all cases, but you can try to build what you get from your supplier. So I guess that your source be accompanied with a script that doesn't start to finish build. And one of the things you find is that products have a lifetime of product has life in the five to 10 years the build environment changes substantially over that time or what's the kind of the current distro. And so it becomes obsolete and very hard to reproduce so I recommend Docker file to capture things like the host OS the tool chains and other programs. But if you have an internal team if you could have another team besides that team kind of test out the build. This is highly aspirational I'm not saying anybody does this now but I think this would be a good thing for the industry to move towards. And then in source availability this is pretty obvious make an internet download site where people can obtain the source for your products. And there are some people who sometimes play games with only trying to allow just customers to get to it. But I don't I don't see a reason for that just make the source code available to all. And then you do need an offer for physical media. You won't get much traffic you won't have to find yourself burning a lot of CDs but you should make sure you have one and that it works in terms of unlock ability. In my view, unlocking your product is not required by gpl v2. So, but it does make community members and customers happy. And so you may decide that you want to do that there there are good reasons to make an unlockable product. And so independent of what your view is of the license requirements. If you support unlocking, then if you handle back end services or streaming content anything protected you'll have to figure out a way to deal with that and it's more difficult than it sounds. And if you don't support reinstallation, you must avoid gpl v3 soccer that's my opinion. So, final statements. So first, I hope I haven't scared you with all this talk. It sounds like a lot of work and and there are there are some eyes to dot and teased across. But it is possible so people people have been doing this for many years, a compliance with the license is required it's not like it's an optional thing it's part of. The payment that you're making for the open source that you received you've gotten an awful lot of value as a vendor from open source software from the developers to put time and effort into it and giving back to them. What they asked for is is appropriate and and legal. It's not trivial, but it is achievable. And then I, again, I'm going to make my disclaimer at the end here due to legal uncertainty. Make sure you make decisions based on kind of knowledge of the risk and with your own company's legal team. And then my last thing is a plug for the open chain prime really should check out the open chain project and I have a little four leaf clover here on on here so I hope that the information is presentation has been useful to you and that you'll benefit from it and good luck in your compliance activities and hope to see you around the industry so with that I will go ahead and take some questions.