 Some of you may be aware of Dr. Wheeler. He shows up to one or two working group calls a day. No, I think David is on pretty much tries to make the rounds of every working group very busy in trying to help us in open source security, and also a topic that's very near and dear to my heart, education. David Wheeler is the Director of Supply Chain Security for the Linux Foundation and is also a passionate educator. He's written several books and papers on security and he lectures on developing secure software at George Mason University. David helped lead the development of the CII Best Practices Badge, which is now branded the... The Best Practices Badge? That's right. And he has helped numerous open source projects operate more securely. He's the author of developing secure software training course, which is available through edX or the LF training for free. It is my pleasure to turn the stage over to my friend Mr. Wheeler. Thank you, sir. You're going to talk about an awesome, amazing topic. Thank you. Thank you very much. Okay. Yeah, I hope. Well, thank you. Yeah, so technically I am a doctor, but my experience has been whenever someone calls me Dr. Wheeler, that means I'm getting set up and clearly it's no different today. Okay. I actually think we're starting slightly early. Is that okay? We're a little five minutes ahead. Are we okay with that? I'm getting the thumbs up. Okay. So for those who didn't come early, they're going to miss out. At least I'm going to hopefully provide some information that they wanted to see. All right. So as Crow mentioned, I'm David A. Wheeler. I'm going to talk about education and training. And hopefully this will be more than just education, good, ignorance, bad, although I actually believe that that's true. But we want to delve in because education, I think most of us will agree, is pretty important. But let me drill down a little bit further. If you read the abstract for this talk, here's what I said I'm going to do, and hopefully I'll be able to do that. I'm going to try to claim that many software developers are receiving an adequate education and training on how to develop and distribute secure software. I'm really thinking the whole mix, development, building, distribution, operations, and so on. I'm going to discuss both why it's a problem and some steps that we're taking to correct that. But I can't say that we've got everything all done or even all the steps all completely laid out. What you're seeing is the bridge being built. So first, I want to try to argue and convince you that we have a, more generally, a serious cybersecurity skill and education gap, and that includes the knowledge about how to develop secure software within the development world. And I'm a big believer in trying to get actual data where you can. So here's some interesting data points I think kind of justify the claim. So CSO Online 2016 said that 46% of organizations said they had a problematic shortage of cybersecurity skills. Five years later, you can see it's going in the direction, now 57% of organizations are having a problem. Great. Okay, now granted the questions weren't worded exactly identical, but I don't think the overall impression is wrong. I think what you're seeing right now is there's already a skills gap and we have more demand. When you have more demand and not enough supply, the results are not good if you're the one trying to get the supply. What did economy claims? 4 million unfilled cybersecurity positions. There's a lot of quibbles you can say about this kind of data, but I think other people have looked have agreed. You know, you can quibble about how you measure it, but there's clearly a gap. Poneman basically has a report in 2020 that 53% of developing organizations don't ensure that they have training on what they call secure coding. By the way, it may seem strange, but I'm going to use the phrase secure coding in several places because that's what my sources use, but I'm actually not a fan of that phrase. Coding is a very narrow part of software development. And when you're doing software development, it's not just typing in text into a screen. There's a whole lot more that goes on. You have to figure out what to do. But that's okay. This is what my sources are using. 2019, there's no top 40 US, what they call coding, or top five non-US computer science school requiring secure coding. Now, hey, in 2022, we had a slight improvement. Looking at all of the top 24, we now have one. Anybody know who it is? Sorry? Name names. Okay, it's UC San Diego. So good for you, UC San Diego. Shame on you for everybody else. Shame on you. And this report in the bottom here, I mean, you can see the report, but the interesting phrase I pulled out is, universities don't train computer science students in security. And isn't that wrong? I think it's terribly wrong. I have an engineering degree, and they didn't let me out with some other things you might not think of as engineering, things like ethics and economy, you know, things that, well, that didn't involve designing circuits. Yeah, but you still need to know that. And I think security nowadays is a critical part of knowing about software development. All right, LF does survey. And what we found was that when people were queried, they were asked, hey, which of these things would be helpful for improving open source software security? And now we're narrowing much more specifically on open source software and security, okay? And people were allowed to answer with multiple answers. So you're not going to see a total of 100%. The top three, well, the first one was best practices. I'm sure Crobe is very unhappy to hear that. Number two was tools, okay? But number three, in terms of agreement, was more training in secure and memory safe programming for the open source software community. So there's clearly a widespread view that training would be really, really helpful. Okay, and you can look at some of these numbers and so, but there's widespread agreement. Now, when I went out and looked and, hey, what's the data that statistics tell me, there was one study that at first seemed to contradict this. But in fact, I think what it's telling is a story of efforts trying but not succeeding to deal with the problem, okay? This is 2022, so this year. Secure Code Warrior Survey reports that 89% of their survey takers say that they've received sufficient training and secure coding skills. 89% that is completely different from what everyone else is saying. Now, whenever you do a survey, it's always possible that the respondents set is very different from everybody else's set, okay? And that's a notorious, it's always a challenge whenever you're doing research, okay? And that is, of course, possible for this. But I'm going for the moment just to accept it at face value because indeed, that's a problem for all surveys. So what does this say? 89% sounds like we're done. Well, wait a minute. Mission accomplished. Yeah, let's stand in front of that aircraft carrier. All right, when you look at that next sub-bullet, let's look at what they mean. They say we've had sufficient training, but most of them are not familiar with the common vulnerabilities. They don't know how to avoid them and they don't know how they can be exploited. Why do you think that's sufficient? In what world would that be sufficient? If I was a bridge builder and I didn't know the common ways that bridge building fell down, the one thing I know is you should not be hiring me to build a bridge, okay? If you don't know what you're doing, if you don't know how to do the thing that you're being trained to do, there's a problem, right? It's more than this. I mean, 92%, they need more training on security frameworks. Wait a minute, you said thought, and granted, maybe this is just a very narrow understanding of coding. But guess what? Offer development is much more than typing text or drawing arrows, okay? 81% said they do regularly apply what they did learn. Now that's impressive, because it means that whatever they're getting, they're at least finding useful. So the whole idea of hey, no one will use this stuff, that's obviously not the case, okay? 86% said that it was challenging to practice what they called secure coding. I'm going, why? Why is that hard? You know, buffer overflows. Don't write past the buffer, don't read past the buffer. SQL injection, there's prepared statements and other kinds of things you can use. They're not hard. If you're having that much trouble, you didn't, maybe you got a course, but you didn't learn what you needed to know. Now it's not true that managers don't care. Nowadays, managers are saying they're a lot more likely to hire people with these kinds of skills. What does this mean? As you can see from my slide, I think that when they say, hey, I'm getting sufficient training, it's not because it's sufficient, it's because they have ridiculously low expectations. I think what's happening, now, I can't from just the survey figure out what's happening. I can simply look at the data and the other data sets and try to figure out what's happening. But I have a story that I think, I'll call this hypothesis, that I think explains it. For the most part, those are not teaching the fundamentals that they need to know. Our educational systems are failing at education. So what is industry doing? They're trying to quickly ramp up their developers by giving them some quick little courses, maybe an hour or so, maybe a nice cute video. And they're finding even that small snippet helpful. They are using it, but it's not enough. They don't know what the common vulnerabilities are, for example. And so Mary Simpkins has this wonderful phrase, I'm gonna steal it and I mean, I give her credit too, whether she wants it or not. But entertainment is not education, okay? It's okay. Oh, thank you. I'm not against entertainment, not at all. The more interesting we can make our educational and training resources, the better. But that should be the secondary purpose. The first is aiding learning. They did say that one of the top ways to improve secure coding training was get a certificate. Basically, they'd like to give credit. Now again, this is what this suggests to me, is that a lot of organizations are trying to create little in-house programs but with very short resources. So making short little resources or hiring out for a very short course and frankly what they're getting isn't what they fully need. How do they prefer to get this? They want codes. They want to see code samples. They want to see annotations. They want itself paced, okay? Articles, books, okay? Multimedia, if you can. But that's the kind of thing they're looking for, okay? And so this is the, huh, this wasn't what, all the other data says X. This seems to say not X. And I think what this gives us is a more nuanced understanding. There is a problem. People are trying to fix it within their organizations but it's not working yet. But there is some hope because even these, what frankly seem to be really terrible courses, they're still finding useful. So let's do better, okay? But you could say, and although most people don't really fight that hard about education, there are some folks who have kind of questioned it and it's fair, you know, questioning your assumptions. Aren't there alternatives? Because educating developers, that takes time. That's work. Can't we do some other things? Well, we've definitely tried other things. Certain large organizations, who will remain nameless, you know, you can probably guess them, have tried the, we're just gonna take some insecure software and configure it. You know, we'll turn the knobs and run some stigs and we're gonna suddenly make this insecure software secure. And what they've found out is that you can't do knob twiddling to turn insecure software into secure software, okay? Yes, you can turn on some more secure configurations. Yes, you can disable some very insecure protocols. But in the end, the software has to actually be designed to be secure. You can't just twiddle some knobs to make it secure. What about training the end users? Well, here's a problem. End users actually have work to do and it isn't babysitting insecure software. Okay? I'm not against education in general and you're sure there are cases where you can at least try. But frankly, you're never gonna get a lot of satisfaction of trying to train everybody who uses a computer. That's just, that's too large. The ocean is too big, okay? And frankly, user training for the most part is pretty ineffective. If you look at the, click through after you've had, you know, I've trained on, you know, in an email, oh, you don't think they'll do it tomorrow? You know they will and they'll do it tomorrow. And you know what? Here's the dirty secret. If you can't click on a link from an email, you design some software badly. That's what links are for, okay? To click on them, that's what they're for. So if you can't do that safely, then the software is wrong, not the user. All right, why not use tools? Now here, we are getting some, actually a fairly good retort, okay? Because I actually do believe tools are absolutely necessary, but they're not sufficient, okay? False pauses are required human judgment, okay? Almost every tool is gonna report things that are an issue from the point of view of the tool, but are not really security issues. Conversely, pretty much all tools are going to miss some things, okay? And this means we have to have human judgment, which means we need to have users and understand what the tools are saying or not saying, okay? There's an old phrase, a fool with a tool is still a fool. Years back, one of the things that I've done, I implemented years ago, is a little static analysis tool. Somebody scans a program looking for vulnerabilities. It found some vulnerabilities, but don't worry they inserted some comments above each line to disable the report. That fixed the vulnerabilities. No it did not. There are two CVEs with the name MyTool in there, because it found the vulnerabilities and they didn't know what the tool was telling them. And then it's not that the tool's hard to use, it's that you have to, the tool on the other end of the eyeballs needs to be ready. All right, so why not just use secure by default APIs? Now here, we are in total agreement, we should as much as possible make things secure by default, both for end users and for developers. But the problem here is, who do you think makes these APIs? They do not fall from the heavens. Some developer had to figure out the APIs. And if a developer has to figure out an API, they have to know what security is so they can make an API that's secure by default. So we're back to, it's not that this is wrong, but it's insufficient. We need education. And really, yes, we do need tools. We need secure by default APIs, but we also need education in order for those other aspects to work effectively. They work together, not in opposition. And really, I would also say, prepare for the future. Attackers are intelligent adversaries. If you block off one path, they're going to look for another. And so we need to have intelligent defenders to deal with intelligent adversaries. All right, so with that, as you probably know, one of the first things the OpenSSF did was release a course on how to develop secure software. You can blame me. I'm the primary author of it. But with lots and lots of help and review, I thank every single one of you. It's basically approximately, I think, 16 hours to go through, and that includes clicking on some other links. And it covers, you can break into three areas. Requirements design and reuse, implementation, and then verification and other specialized topics. And the idea is that it's supposed to teach the fundamentals of developing secure software. And it's not specific to open-source software. It turns out it's open-source software. So the necessary issues in developing secure software are true for all software. So it covers things like design principles. They're basic principles that are many decades old now from Sultz and Schroeder. They're still true. They're going to be true next 10 years, 20 years, next 30 years too. So developers should know them. Using accept lists, not deny lists. You should know what the most common kinds of vulnerabilities are and how to prevent them from happening. If your software is going to have vulnerabilities, at least make it a whole new kind no one's ever seen before. The vast, vast, vast majority, you'll see different numbers, but anywhere from 90% to 95% are the same old things. There's reasons why they're the same old things. Once you're aware of them, it's not hard at all to prevent them. The course is built from small modules and, in fact, the text is available under CC by, so you can go and use it in all sorts of ways. And if you're interested in taking the course, there's a link right there. Okay. Now, initial is available on edX 2020. Now, there, the course is free, but there is a time limit, and if you want to get a certification of completion that does cost money, this is basically how edX works, and that's fine. But a lot of people are saying that, hey, we're concerned. You know, if those cost of certifications are potentially a blocker, we want to encourage people to be able to prove that they took the course. So, on March this year, we made the material also available on the LF training and certification platform. Okay. It has a slightly different name, but it's exactly the same material. And here now, on this platform, the course is free, but with no time limit, and the certifications themselves are also free. And, you know, and once you complete it, it's good for two years. And we're going to continue to make it available also on edX, because a lot of people like the edX platform. Many organizations have special agreements with edX. We want this information to get out to all sorts of folks. So, we plan to support both platforms. And I have a cool new announcement. Ooh, okay. So some of you already know about this. Okay. But, all right. So, as of today, I want to announce that we're going to add yet another way people can get access to this course material. It's something called SCORM Connect. Those who are really into learning systems already know what that is. So, it turns out that a lot of educational institutions think universities, colleges, and so on. And a lot of large organizations have their own systems what they call learning management systems. There's a lot of these in the market. And they basically enable you to track your employees or your students in terms of what courses they take, make sure they get certain things, and track and so on. We're now making the developing secure software course available via SCORM Connect so it can be directly incorporated into those learning management systems. So, if you're in a large organization or a university, you can just click on it through their system. It looks like it's embedded as part of theirs, but in fact, it's the material that we've already developed. This is going to be, this is free for open SSF Premier members. If you're not an open SSF Premier member, but a member, you can talk to us later. And we're also going to make it free to accredited educational institutions because we want to get this out. Other organizations can absolutely include it. We would love for lots of folks to incorporate. We're trying to get this information out in as many different forms and ways as we can. It is going to be an annual fee for other organizations because it actually turns out to be costing a non-trivial amount for this SCORTA stuff. So, this is as far as I could get people to swallow paying our costs. But that said, we don't feel like, if you don't fit those categories, you can't do this. Absolutely, come talk to us, okay? Again, this provides another mechanism so developers can learn about this. It enables any integration. The course is going to continue to be free via LF's training certification and the edX platforms. So if you're an individual or a small business, you don't have one of these systems, don't worry about it, just take it through the mechanisms we already have. We're not removing anything, we're adding something. Okay? Quick technical explanation for those of you who are really into this. In the educational world, SCORM is a common format for training materials. It's been around for a long time. We put the learning materials, quizzes, everything else in that kind of form. SCORM Connect is a weird form of the SCORM file. It's a standard format and it omits all the content. Now how does that work? Well, what it has is a little redirect that says, hey, to get the content, go over here to this website. Okay? Now why do that? Because that way we can update the material and instead of trying to convince people to update their material, we can rapidly update material as new information or as fixes become available. Okay? All right. That's not my only means, the only educational resources. OWASP has the Security Knowledge Framework. The Open SSF has actually provided some funding to that group. And this, SKF is a different approach, although they actually embed the course within their materials, the other course I just mentioned. But what's interesting about them is they do hands-on code experience. The pro, of course, is when you do hands-on, you learn better. The con is that that takes a lot longer. So, you know, there's pros and cons to all sorts of approaches, but, you know, that's a pro if you're willing to put in the extra time. A safe code is some very nice training materials. There's very expensive materials. I should note related that the LF plans to soon release other courses on Sigstore and DevSecOps. I can't say more yet. But it's coming, it's coming. Now, earlier, Brian mentioned the various stream, the 10 streams in the Mobilization Plan. Stream 1 was specifically about development education and certification. Okay? So, what I just talked about, the course is a part, a support for that broader goal. And here's if you look at the stream, here's some of the things that it mentions. Basically, relatively few software developers get serious formal training. You know, that survey notwithstanding, I think those don't count for many cases. If you don't know what the common vulnerabilities are, you've got a problem. You know, a small amount and ideally more. And there are some modules, stream modules available free, like ours and the OWASP one. And basically, we want to make things better. So, there's three parts to it. The first part is collecting and curating content. Let's get what's already available, okay? And make it easy to find, easy to use, work it together. We very much intend to build on the course I just mentioned, the Secure Software Development Fundamentals courses. Absolutely add and do gap analysis. Second part is to expand training, okay? Where we want to deliver training across industry. We want to partner. We are not everywhere, nobody's everywhere. So we want to partner with everybody, lots of different kinds of organizations to get the word out. Have unified certification approaches because if everybody understands what something is, what more likely people will want to get those kinds of certifications. And finally, reward, incentivize, developers. Working with major hubs for open source software development, GitHub, GitLab, and so on. Try to offer some financial incentives. Working with job boards. Well, it would be very cool. We don't have this yet, but I think it would be very cool if, for example, you went to Indeed or LinkedIn and you could say, oh, that developer has that certificate. Okay? Or you went to GitLab and GitHub and looked on their page and, oh, look, they know how to do this. I think that would be awesome to be able to start to see that. And creating some sort of tiered badging system with escalating rewards of some kind. When we first publicly released the mobilization plan, we intentionally had breakouts to discuss each of the streams to get feedback. Thank you. On those. And one of the ones that we had was a discussion about stream one. Lots of great comments. One that particularly struck me, I'm not sure how to deal with it, but I find it very intriguing, was really a push to say, hey, wait a minute, most software developers learn a lot of how to develop software before they even hit college or universities, if they go do them at all. And there are various numbers, but a vast number, some around half of software developers don't go to universities, new colleges, at least for learning how to develop secure software. Develop software may even be much larger, but it's a significantly large number. And whether it's a majority or a minority, doesn't matter. There's a lot of folks, you won't hit, say, the computer science and software engineering and those kinds of degrees. So push the education back to K through 12 and really especially at the high school level and trade schools. Covering universities and code boot camps and existing practitioners increasing the demand for these courses. Maybe we can find ways for governments to incentivize this. And in fact, I mentioned some of you earlier, translations for international consumption. I learned a little French and the main thing I learned is that nobody wants to hear my attempts at French. So we really want to make this international and part of that is going to need to be figuring out how to get translations. So if you are interested in improving the world of having software developers educated and trained in how to actually write secure software, what do you do? Well, hey, if you develop software, there's an easy one. Take a course, at least one. Software development is a, as long as you're going to do software development, it's a lifelong commitment to learning. That's not necessarily a bad thing. But if you're going to develop software, you need to learn how to develop secure software. You know, go to that link, take the course. You want to do the OSSKF, that's great. There's others too. Obviously, I developed one of the courses, so I like my course, but you know what? I'm way more concerned about taking a course. I don't care if you take that particular one. I'm much more concerned, do you know what you need to know? Work with and oversee, if you work with and oversee those developing software, encourage them to learn. You manage other software developers, you probably already know that this is an issue. Well, you know, yes, it's a pain to stop for the moment and learn something, but it pays dividends over and over and over again. Improve existing training materials, okay? So for example, if you want to improve the course I just talked about, it's on GitHub, okay? It's CC by, file an issue, or even better, a pull request, okay? And in fact, one interesting thing, we've actually been talking about for several months, about how to make some interesting little pictures to jazz it up and maybe help explain better. So I just got permission this morning from OpenAI to use Dolly to create some graphical images that we hope will clarify some of the concepts that are taught in the course. Will it work? I don't know, but we're going to do an experiment and find out. And I think that's really key, is let's look for ways to try to make things better. Repeat, if the experiment doesn't work out, okay? You learn something, but if it does work out, you've made an improvement, and if you do enough of those, the results are going to be better for everyone. If your organization has a learning management system, consider adoring the OpenSSF course, okay? You don't have to be an accredited university or an OpenSSF Premier member, but if you are, then hey, it's free, and if you're not, come talk to us anyway, okay? Because we would like to see more and more developers no understand this material, this kind of material. And finally, OpenSSF Best Practices Working Group is creating within it an education SIG. And the idea is to convert these higher-level ideas into a plan, and more importantly, implement the plan. There are way too many people who think that planning, do you create the plan and then you're done? Please no, okay? In fact, whatever you end up with is probably going to be different than your plan, and that's all right. We need to implement. And join us, and if you're interested in the OpenSSF Best Practices Working Group, there's the link. So, education is a key part of improving tomorrow's security, and here's some of the ways we're trying to get there. So, with that, thank you very much. Do we have time for one question? All right, one question. I may give you, and I don't know, by the way. Yes. There probably is. This course wasn't particularly designed for, although it does depend on the kind of management you're doing. There's a whole lot of managers who also write code, and for those folks, I think the course on developing secure software is still absolutely apropos. That course really does assume that you know how to develop an A programming language, though. So, I imagine that there are such courses, but I don't know them offhand. There's also opportunity, since we have both the Working Group and this new SIG, if someone's interested in collaborating on making that material, that's absolutely a work stream we could help facilitate. Yeah, yeah. So, I don't know. I would say step one is let's go find out research, and if it doesn't, that does sound like something that should exist. Four minutes? Okay. I hate to take away all the time, but the extra time we got. So, oh wait, I see a hand in the back, I think. My eyesight's terrible. Ah, does the training suggest the use of S-bombs? Yes! All right. That was easy. Yeah, it talks about S-bombs, and not just, and more importantly, what they are and why. I will quickly answer. I see Mr. Freeman. I see Alan's very happy about that. I will quickly note though that just having an S-bomb does not suddenly make things secure, and I'm sure you'd agree with that, too. You've got to actually, an S-bomb simply gives you a list of materials. When you receive it, you're going to have to do something with that information, like look at it, and run a tool and say, hey, are there known vulnerabilities? Should I worry about those? Are some of these components hideously old? Does this suggest IRS? S-bombs don't tell you the risks. They give you the information necessary to help you identify potential risks. Risk, yes, sir. Ah! Actually, let me repeat that. SB does not stand for silver bullet. Totally agree, yes. The challenge, more generally, it's very, very difficult to capture metrics. People don't want to report a lot of things, so we don't have, like, before and after analysis, that kind of thing. It wouldn't be a bad idea, but I'll be honest, right now, the state of education is so woefully bad that, or actually, basically nonexistent for most developers, or at least, you know, it's on the level of I took a course, but I don't know what the common problems are, and I don't know how to deal with them. Given that, I don't think, right now, we're very worried about it. The state is so bad that anything isn't going to be an improvement, and so we haven't focused on that. But I do think, as we go on forward, it would be great to try to measure, you know, AB testing. We do it this way, do it that way, which is more effective. But we haven't made any plans for that yet. Okay. I think we're... We're going to exit off so somebody can do the sync with their thing. So, thank you very much. Thank you everyone. David Wheeler.