 Can you, does it work well? Okay. Okay. So, now that we set the stage for the introduction, what this meetup is really supposed to be about is DevSecOps. But what is DevSecOps? What is even DevOps, right? These are still very young or relatively young terms that some people have a certain understanding of what this means and some other people have a different understanding. So, what we want to do now is actually set the stage of what DevSecOps is and what it actually entails. So, to really paint a broad picture and cover things like culture, processes and technologies on a high level. With this in mind, we can then further iterate through, if you will, and figure out other topics that fall into these three categories as a basis for future meetups and talks. So, my name is Stefan. Don't worry about my last name, you will never have to pronounce it. I work with a local company called Vantage Point, where I'm focusing on initiating and developing and maturing enterprise security programs and also helping companies integrate security into the development life cycle. And what I really care about is actually achieving security at speeds. And this is also what triggered my fans, if you will, for DevSecOps. But the reason why I'm really here today is DevSecOps Singapore 2015. Who has been at that event? I know at least one. Anybody else that has been there? Two, okay. This was really great because DevOps, it was the first time that they have DevOps days in Singapore. It's a conference that just came up a couple of years ago, actually 2009, and it spread across the globe like wildfire. And I'm not sure if you can read it here, but what they are following is this open space concept. So, you can actually take a post-it and write down whatever question is on your mind, post it on the wall, and if people want to hear about this, you can have a round table, if you will, and people just discuss about the topic. So, I was going there and just tried my luck and see what happens if I write, how do you integrate security into agile and DevOps, and it really got a lot of attention. So much in fact that this was selected as one of the topics, and we had over 50 people that came, joined this open space session, and wanted to know about how do we actually do that? What works for you, what doesn't work for you, et cetera. The funny thing was that even though we had a good group going, there were more questions raised and actually answered. And in the sense of DevOps, that's actually why I started DevSecOps, because there were so many questions, there was so much interest, and there were no security people there. It was only developers and architects and testers that want to know how we can integrate security into DevOps. And that's why we founded this to make sure we can give back to the community, maybe even start and kickstart the community, and find a way to answer these questions over time. So, let's start with a brief history of DevOps. Who knows what DevOps is? Okay, a few people, good. Let's start at the beginning. At the beginning we had waterfall. Everybody knows what waterfall is, right? Waterfall has tremendously long release cycles. You can even think about having one release every nine months. You have a lot of so-called work in progress, which means stuff or things you've started on working, but they're not going to be completed until you actually release. So you have a lot of resources, money, in this waterfall process that doesn't return money for the business. You have a lot of functional silos because you have the business at the very beginning that comes up with the case for these products, they write up their requirements, throw it over the wall to the business analysts, they take it, look at it, define the requirements, throw it back to the architects, but once they hand it over, they forget about it. They don't care about it until they see what happens, and then they're mostly upset. So it's a really incredibly rigid approach to developing. So something needed to change. And that's when agile came along. Who here uses agile methodologies at work? Yes, yes, quite a few, fantastic. What is agile? Agile is supposed to be the complete opposite of waterfall. You have shorter release cycles, and in fact, they're getting shorter and shorter every month now as companies evolve. You have much smaller batch sizes, meaning you don't want to ship the whole product at once, but you actually want to start with an MVP and slowly iterate it and see what works and what doesn't. You have cross-functional teams, which means you don't have different people in silos working together, but you actually have people on the same team working towards the same goal. So all of this allows you to be incredibly agile and actually ship software or at least develop software rapidly. So what happened then? Now you have development teams that actually can create software that runs within weeks or even months. Suddenly, ops became the bottleneck. Suddenly, they were getting more releases, but they couldn't really release it because the infrastructure was not there. They didn't have the methodologies because they were used to the nine-month release cycles. We actually have lead times of deployment of months or even quarters. So this is where ops was suddenly becoming a bottleneck. And at this time, two major trends emerged. One is called agile operations, which is based on the agile manifesto but mapped to infrastructure and infrastructure operations. And the other one was collaboration between developers and ops people. So because of these two trends, they ultimately led to this first DevOps days conference, which took place in 2009 in Belgium. And as you can see, DevOps days or short DevOps, this is where arguably the term DevOps comes from. And since that time in 2009, this has spread across the world and is now in many different cities. And that's why we're even here because it's what works. It really allows you to become agile, not just in development, but from the whole lifecycle end to end. OK, so what is DevOps? To summarize it very briefly, it's a set of principles and practices for efficient collaboration and communication. So let's call this culture. A very key part of it is also the automated deployment pipeline. Let's call this processes. And we also have a supporting tool chain or technologies. So we're going to explore this in the rest of this introduction. But first, there has to be one differentiation made. We talk about DevOps, but it's more than just developers and operations. And I want to quote Gene Kim here, who is one of the foremost figures of the DevOps movement at the moment. And he says, it seems as though the problems are just between developers and operations, but test is there and you also have security. And all of these are actually top level objectives for management. And they're now becoming a part of the development or the DevOps picture. In other words, when you hear today DevOps, you should probably be thinking DevOps QA test infosec. But clearly that's not very catchy. So let's just stick with DevSecOps for now because the focus of this group is really about the security aspect and how it translates to DevOps. OK, so far so good. The target state for DevSecOps is to enable organizations to deliver inherently secure software at DevOps speed. And DevOps speed means almost instantly. It's quite clear that companies that are embracing DevOps, they are facing a lot of challenges. And if you look at this, let's start with the typical enterprise what we covered before. A typical release cycle, deploy frequency is once every nine months. The deploy lead time might be months or even quarters until this release is ready to be deployed for production. Then you have the reliability of these deploys that are low to medium, sometimes really poor, because all hell breaks loose when somebody is shipping software. You need your green zone. You need everything prepared, a specific time window to make that happen. And if you're getting over this time window, you screw essentially. And obviously, the customer responsiveness is low to medium. If you get a feedback from a customer and they say, change this and you say, sure, can do. Just give us nine months, then they're going to do their business somewhere else. But if you look at companies like Amazon, Google, Netflix, Facebook, or Twitter, these are the ones that are the forefront, the examples of how DevOps can actually be done properly. They are deploying at least a couple of times a week. If you look at Amazon, 23,000 times a day, that's pretty much insane, right? So if you have this and all of them have a few things in common, actually, because it doesn't matter if you deploy 23,000 times a day or, let's say, a few times a week, if you have a deploy lead time of minutes, you're doing really well. And you will be able to do this reliably. And you can respond to customer needs almost instantly, if you will. Fantastic, right? So what's the problem? If you know how security works or how security works traditionally, in a typical enterprise, using waterfall, you have maybe a two-week window at the end of the release as a part of unit testing or after unit testing, we have a two-week window to do, let's say, a security test. Then you take the results and feed them back to the development team to fix it and then ship it. How do you do this at Amazon exactly? How do you do this if you even have one deploy every week? It just doesn't work. The traditional security mechanisms and the way security is done traditionally cannot keep up with this. And the exciting thing really is that DevSecOps now is really still in its infancy. I mean, I think if you look at Google Trends, it doesn't even register yet as a part of a trend. If you look at DevOps, it goes like this. But DevSecOps is still in its infancy, so that's why most of the things we're doing right now, we see where or what we see other companies doing, is experiments that kind of work most of the time, but we're still figuring this out as a community. So this is an exciting opportunity to really push the envelope and make sure that we can get security integrated into DevOps. OK, so DevSecOps, let's look at the three key categories, culture, processes, and technologies. For culture, one of the key aspects that is critically important is communication and transparency. Dev and ops or DevSec and ops. We have people from different disciplines and different backgrounds that try to understand each other. They try, hey, what are you trying to solve here? What's your objective? This is ours. This is yours. Great. So how can we make it work together? It's not about, hey, we have our objectives, you have yours. You do yours, we do ours. It's like, hey, we have the same objective. We have a business goal. We want to ship software. The software has to be performant, reliable, and secure. Let's make that happen as quickly as possible. Another thing is a high-trust environment. So the folks at Etsy, they're also a really very interesting DevOps shop that embrace DevSecOps as well. And what is really important to them is that they need this high-trust environment that allows people to fail. Because all of this is still in its infancy. If you are not able to fail, then you won't try. And then nothing will change. So by having this environment, and the term is, for example, blameless post-mortem, people are actually looking at finding mistakes or finding issues and coming up with fixes in a very rapid manner. And this also leads to continuous improvement. Because if you know all these little things, the little paper cuts that actually produce waste in your value stream that delay you, that cost you this little time, if you identify them and continuously improve upon it, you end up over time with a really good process. Another important aspect is that everyone is responsible for security. It's not just the security folks' job. It's the job of the architects, the business analysts, the developer, the tester, everyone. Automate as much as possible. This is gonna be covered more in detail later, but because resources nowadays are really sparse. You know, finding good infosec people or finding good people, there's so much demand for them. How can you make do? How can you actually handle all this demand? And this is where automation comes into place. And also everything is code. And we are not talking about code code, we're talking about infrastructure is code, security is code, and what we talked about earlier, compliance is code, or things like this. And this is important because if you actually stop having documents and are able to codify a few things, you can enforce that over and over again. Okay, so this slide is not content now, but this is supposed to give you some topics for the open space or some ideas for topics. So it could be interesting to have a session on how are you guys doing DevOps or DevSecOps in your companies? What are the challenges you have faced? What are the problems? What has not worked at all? And just share what's going on inside your organizations. Are you, any of you doing, for example, continuous improvements, like Kaizen, is this something that you've experienced with that worked for you or not? And what kind of security tasks or anything are you automating already? Okay, the next category is processes. Here we differentiate between the secure software development lifecycle and security pipelines. A secure SDLC is really just an overlay over the existing software development lifecycle, which allows you to embed security activities into the lifecycle or if you will touch points and even gates where you enforce certain activities or at least check for certain activities as an application goes through a lifecycle. And the key here is that this is not supposed to only work for waterfall or only work for agile. This is the security as part of every project going on and it has to fit into each of these methodologies. So if you talked about everybody is being responsible for security, training is a very good example for that. If people are not aware of what security means to them, it should mean for their role. There's no way that it can actually contribute. Requirements, there are things where you can actually embed security requirements in organ, like in the, let's put it this way. Requirements are typically or security requirements are typically considered to be non-functional requirements, which is not the case. But this is really a very immature area in software security at the moment where business analysts don't necessarily know what is the security requirement? What are our requirements that we have to follow? Then we have architecture and design, making sure that the design that you come up with is actually safe and sound and secure and not working and can be broken easily. And there are other things like coding, testing, deployment and post deployment. And for all of these, the different activities that can be done. And this is especially important because most of the things or at least a few of the things, they cannot be done by tools and they cannot be automated. So in order to actually achieve velocity for DevOps and making sure that a software can be shipped quickly and securely, you need to find a way to get the people involved into that. The next one is security pipelines. Has anybody heard about the Absec pipelines before? Okay, that's actually a very recent innovation. An Absec pipeline takes the principles of DevOps and lean and applies it to an enterprise security program. So what does that mean? If you look at the critical resources in every security, in every company right now when it comes to security, it's typically the security team. Like there are normally like 1.25 people for every hundred developers. That's really hard to scale because there's a lot of work coming in. So how can you make sure that the people that work in a security team can focus on the things that are relevant and not look at every code change? So this is where the security pipeline comes into play because it really reduces the friction between developers, operations and security people to allow kind of a self-service approach to security and facilitate automatic assessments as well. This also at the same time increases visibility because many times organizations are not yet aware of how many projects do we actually have? How many go through our SDLC? How many are we covering in terms of security? And importantly, that each step of this pipeline is repeatable. Okay, this is one of the recent OWASP projects. It's called the Absec pipeline. The folks at Pearson, they are doing this day in, day out because they had accepted a problem that they had a team of, let's say five people, three of them left and now they had to do the same work that five people had to do before, which they couldn't even do before because five people is still not enough for a team of hundreds of, like tens of thousands of developers. So what they did is they became creative. So they figured out something like this and templated it on OWASP where they modeled a pipeline for application security and honestly, for this topic alone, we can probably have the next six months to just talk about how they're doing this and how you can actually implement this at your company and what the benefits are. But yeah, this is a big picture slide deck so we're not gonna dive into details. Okay, so for processes, some ideas for future discussions could be how are you managing your security requirements? How are you building security into the SDLC and what I would be really exciting to see is how many of you actually have seen EPSAC pipelines in the wild? Is anybody using any pipeline for application security already at the moment or are working on something like that to make it happen? And another interesting thing is chat sec ops. Anybody familiar with chat ops? Yeah, so this is a way to actually use your Slack or Hipjet or whatever you use internally for chatting to make it a self-service portal for your project managers or devs to actually request a fortify scan or request something like that. Just a simple chat command and then it's done. You don't even have to do anything. So it would be really interesting to see if there's something already used or maybe even have a workshop on what chat ops is and how you can get security involved there. Okay, the last category we talk about is technologies. And dev ops and dev sec ops is not supposed to be about tools. Big, big word of warning because many times we talk to people as well and say, hey, we are dev ops shops. We have Jenkins. But that's not at all what this is supposed to be. So especially when it comes to security as well, just having a source like aesthetic colonizers running on your every release doesn't mean that that's it. We're now a dev sec option, not a goal. Still critically important because without the technologies you cannot enforce the processes with an automaton. Yes, technologies and tools are really important to ensure that you can facilitate the processes and have the culture and direction grow. But tools alone not gonna solve the problems. Okay, so when we talk about dev sec ops technologies, you will see there are similar phases as we have discussed in the processes for SDLC and for the pipeline. But really this is about, so if you're already doing security requirements, how can you scale them across? Not just one team where you have a security champion, but how can you push them out across the organization? This is about when your developers codes, how can you give them instant feedback, be it by using ID plugins or maybe even integrate Sust into a static application security tools into your continuous integration pipeline. It's about figuring out the way to give this feedback loop to your engineers. Then when it comes to testing, you have things like gauntlets, which is a dedicated test suite that allows you to run security tests. Just typically in the test phase, if you look at the QA folks, they run unit tests, but they typically don't cover much of security. So there's a lot of room for improvement here. Of course, there are other things like dynamic and interactive AST. For example, Web Inspect or Contrast Security has an interactive application AST solution where we're not gonna dive into any details here, but there's a lot of tools that we can explore and see how well they work, if they work, et cetera. An important aspect is also configuration management. For example, when it comes to hardening, hardening should not be a problem anymore. We know exactly how to harden the systems. We know exactly what is required to make it work. So why not just codify it and make sure you have chef cookbooks or recipes where you can actually deploy an infrastructure and instances out of the box that are fully hardened without you having to do anything. Patch management is another interesting aspect. So right now, if you have to install a patch in a traditional environment, it's a nightmare. But there's actually interesting strategies out there. For example, Phoenix deployment strategy, if you're familiar with that, where you're actually not upgrading the systems, the instances that you have, but you're deploying completely new instances that happen to be patched and you destroy the existing ones that are not patched. So you roll out a new green field of instances and eliminate any configuration rifts or even technical depth. And also the ability to monitor these is quite interesting because if you codify your infrastructure, your security, you can also audit it at the press of a button. And another thing is, for example, RASP, runtime application self-protection, which is also one of the latest crazes. And I would be interesting to see if people are using this already, if it works for them or not. Oh yeah, and one word of warning about the security tools out there. They're often misunderstood to be like silver bullets. They can help a lot. They can automate a lot of the pain away. But the point is that these tools typically only cover half of the vulnerabilities that exist, what we call the implementation bugs. You know, if you have a SQL query and you didn't parametrize it, if you have a cross-site scripting because you didn't output encode the outputs, this is what the tools are good at. But when it comes to business logic or even authorization, authentication, almost impossible at this stage that they can find it. So yes, tools by all means, but understand what they can contribute, what they can actually address and what the other areas are that you still have to consider. All right, so for the technology's open space ideas, I would be interested to see how you are scaling, for example, security requirements. If any of you is using test-driven development or general using security in a testing environment, which of these star AST technologies are you using? What works for you? What doesn't? Have you made any experience with IDE plugins, commercial, non-commercial, open source, whatever? Yeah, so we have a few ideas we can also explore. So to summarize, in order to make DevSecOps work and enable organization to actually deliver inherently secure software at DevOps speeds, you have to make sure that culture, processes and technologies actually come together as one. That's the same goal. And then good things will happen. So with that, I say thank you very much. And open the floor for any questions. Yeah, so what do you think? Did I do it justice or did I miss out a whole lot of things? Regarding the test-driven development, do you mean that because of the test-driven development or the application of both? Actually both. Traditionally, it's more of the operating system at the stack for the Phoenix, but you can do it for application as well. What's the number in conventional? It takes a lot of time. Especially for Windows, you know, it's stuck. Yeah, exactly. So the tool use is for Phoenix. So what's the recommended tool? For this, I think recommend it to you. For this, Phoenix is just a strategy on how to do deployments. But yeah, you can use different tools, for example, puppets to make it work. Any other questions at this stage? Anything you would like to ask? Was this interesting at all or what you expected? Yeah, all right. Well then, what I would suggest is the following. We have the whiteboard over there. Maybe we can get it in there.