 To those of you online, welcome. And of course, those of you that'll watch this as a recording, also welcome. And I'm Mike Nash. I'm our VP of Information and Cloud Security at LightBend. And I'm going to talk a little bit today about cloud security and how you can write and develop and deploy secure web applications or any other kind of application using our product Calix. And Hugh McKee, our developer advocate, is with me today. He'll jump in if my slightly less than normal internet gives us any trouble. Please feel free to add questions. And there's a Q&A tab now in Zoom, because we're in webinar mode. So feel free to add questions in there. And we'll try to address them either during the presentation or definitely at the end. I'm just pausing for a moment to let anyone else that's going to join jump in. All righty. Without further ado, let's dive in here. I'm going to share my screen. I have a quick run through a deck that I've prepared to give us some talking points. I may go a little off script, just as there are things that are always changing, of course. But welcome again, and welcome to this presentation about information security and privacy with Calix. So the things we're going to talk about today are building secure applications with Calix. Calix is, of course, a hosted service by LightBend. So security is a big deal to us and a very important priority. How Calix is built and our own internal InfoSec program will cover, how you architecture and develop your application to take advantage of security features will cover, and how your Calix environment and its services should be configured to work best in a secure environment will be covered. We'll also talk a little bit about compliance and how your company's compliance can make use of what we do with Calix. OK, so let's quickly talk about LightBend's information security program, a topic near and dear to my heart, and also the foundation of how we build and run Calix in a secure way. So we have an information security program, of course, at LightBend. And Calix is built within an environment that is designed to promote security from the ground up. So from the beginning of how people are vetted to make sure that they should be a part of the team to how our development environments are created, to all of the dependencies of Calix, these are all part and parcel of our InfoSec program, and these are all about both how Calix is built and how it's operated. And we have an extensive set of policies that make sure that we're all on the same page internally as to how these things should be done, where the boundaries are, what tools should be used, and so forth. So our policies not only count against how Calix is built, as I mentioned, but also how it's operated. Although it is zero ops from the customer's point of view, you don't have to worry about operations or Kubernetes or any of these things for Calix. It obviously isn't for us, we're the ones who are actually operating it. And our team follow a series of very secure practices to allow us to be able to guarantee your data security, integrity, and availability. So it's not just a matter of security, but of course you must be able to get to your data. Our information security program is built around a series of policies that start from the AICPA SOC2 standards. Security and organization control standards, I believe is what that stands for. So our software development and our operations are guided by these policies and our team are continually trained in the latest standards and techniques for building secure systems. In fact, that's one of our policies is that our team is always up to date on what the latest and greatest discoveries are in the world of information security. And then what we have is this is our compliance and our attestation that we follow these policies. We also have an external annual audit and we're in the middle of setting up the very first one of those that confirms our opinion that we are following these standards. So we have an independent firm that validates and gives you the SOC2 certification, which basically says we've looked and we attest that LightBend is in fact following these practices during the period that we've observed and that observation period is over a period of three months and we do that every year. So not only are we meeting these standards at a point in time, we're meeting them on an ongoing basis. So the software development is where it begins, essentially with Kalex and how Kalex can be used in your environments to build secure applications. Our software development environment begins with of course our people and we've talked about how they're validated via background checks and so on and then exposed to ongoing training so that they always have the opportunity to learn what are the vectors that we should be concerned about from an operations point of view and also how should we build Kalex to allow you to build applications that can remain secure operationally. Now, of course software supply chain is a big deal nowadays and everything we use to build Kalex is validated and scanned. Now, one of the core elements of Kalex is of course our ACA framework, which LightBend also built but then ACA has dependencies and then of course some of those dependencies have dependencies all the way down. We validate that entire supply chain both with manual scans periodically so that we can investigate the complexities of that chain but also with automated scanning. So if any new dependency is added before it's actually built into any of our products there is an automated system that is part of the CI CD process for us that verifies that there are no known vulnerabilities or no security warnings about adding that new package and then that's validated by a human whenever a new change is made. So our change control process also allows us to make sure that multiple eyes are on every change that could have any bearing at all on the security of Kalex itself and therefore your data. So any vulnerabilities that are discovered any CVs that are reported of course by our system are immediately addressed before a new release and any CVs that are discovered after release are also immediately addressed. So let's say a discovery of a CV in a library that we thought was secure is made after a release of Kalex. That of course becomes a very high priority for us and our customers are automatically notified to say, hey, this has been discovered after release, here's what we're doing about it. Here's how you can avoid any harm from this potential vulnerability. So our process of course is very secure. So we develop and store our code on secure systems of course with all of the various state of the art protections that you could imagine. We report and investigate all security incidents transparently so our customers know exactly what we've done how we're going to make sure that problem didn't occur again. Honestly, not many do occur but every now and then we'll find a warning coming through from one of our providers that we have to investigate we make sure that your team is very informed of that. And of course Kalex's own source code is scanned with tools that we helped build. So there's actually a tool called Fortify and we built a plugin for the language Scala that we built Kalex with. It allows us to find patterns that are potential security holes and to address them and address them might mean in this situation that's perfectly safe or no, we should actually change that practice and do something else for their source code. And then of course every change is reviewed by two of our team so that it's not possible for one person to make a mistake and no one else to see it before it makes its way into the product. So on top of our development process when we deploy onto the major cloud providers, we're working right now on Azure, we have already deployed onto Google Cloud and to AWS. And we of course take full advantage of the fact that they have very robust security programs. So we're standing on the shoulders of giants so to speak to say, all right, we will utilize all of the security mechanisms encrypted disk, physical security, all of the things that these providers give to us we will make full use of as we build Calix and put the little piece on top that we do that's Calix that allows you to utilize our service. The links you see in this presentation are actually the links to the cloud providers own InfoSec process. And you can learn about how deep and wide and of course they are as you can imagine quite extensive from each of the cloud providers. So another element of InfoSec is that is very important in a highly international environment which is what we deploy to is data sovereignty. Where is your data physically? And in what domain does it reside? And therefore what legal protections does it have? What legal framework is it under? So with what we call a Calix dedicated instance which is a specific cluster set up for your use as a customer, as opposed to a multi-tenant cluster which is used by many customers. Calix allows us in this modality when we're doing a dedicated cluster to specify which region you'd like to run in. So if you specifically need to run in the EU we can run in the Frankfurt data center in the Stockholm data center wherever you need. We currently have dedicated Calix instances running in many of them in the United States some in Europe and so far one in Australia. So the data and all of its processing occur only within that region. So not only within the sovereignty but within that specific region across multiple zones though of course for availability but within a specific region. So data sovereignty is a big thing for us. Now assurance and compliance many companies virtually all companies that are out there selling services on the web nowadays have their own compliance program and standards that they must comply with. And Calix being a vendor to them means that we must in fact adhere to a certain level of compliance in order for your program to be complete. So we adhere to the standards for secure development again by adhering to the SOC2 standard. There's actually several others. The GDPR standard for example and we also implement some of the ISO 27001 controls. Excuse me. That also involves a lot of the things I've already talked about. So extensive review of any security related change there's always review of every change but there is more extensive review and more extensive testing of course of anything that may actually affect security credentials anything of that nature a new login mechanism and so forth. Extensive testing at many layers. So the testing is not isolated to any one level of testing. The whole system is tested on a completely separate development environment that it's performance tested on a staging environment to make sure availability isn't impacted and all of this is an automated process that happens before the new version of Calix even runs to production. And of course only as needed access control to all admin functions on our side. So our team is, you know unless there is a necessity or a request we are not actually accessing the running Calix instances at all we're managing the cluster externally but not dealing with your data. And of course automated validation of deployment signatures to make sure that the image that we're deploying is in fact the image that is getting deployed. And then on top of that of course extensive automated observability. So we have monitoring we have tools like Grafana cloud and so on that allow us to keep an eye on the operational cluster and then some security specific tools that will alert us to anything that's out of band. So performance is impacted or a potential DDOS attack all of these trigger and alert process on our end and our incident management process which we'll talk about later. So how do you develop securely with Calix? Because of course the nature of the platform is we're running a platform but your teams are building the code and deploying to that platform. So you must of course make sure that access to those applications that you've deployed is secure and Calix provides a number of facilities directly for you to do that. So a very quick and hand wavy level architecture diagram of Calix talks about its data stores talks about a client application and a quick assortment of some of the mechanisms these are always being extended. We just actually added a new one recently for multifactor authentication. But to give you an idea what the mechanisms are that are provided that your application can take advantage of. So typically there's some manner of client application either another service or perhaps a web-based application that is communicating over an HTTPS or some similar link to a network routing as soon as it touches the Calix network the beginnings of security start there. So for example only selected ports are available. You have some options to enable other ports but by default there is not even an external web route to the service that you deploy on Calix. As for example it may be only used by other services within your application in which case there's no need for an external web route. So external web routes have to be explicitly granted such that access is under your control. So therefore you've got secure by default you have to actually take action to allow the rest of the internet to be able to access these routes and you have full control of that. There is also the support for client side certificates and support for JWT for the JW tokens. We also have mechanism for ACLs. I would recommend having a look at our documentation it's freely available online to see exactly how that works but this allows you to do some simple security mechanisms right away and say this service is only available in this situation and from this service and so on. And all of the services in Calix because they're dealing via a proxy with the remainder of the Calix infrastructure don't have direct access to the underlying data store. Therefore it's actually much more difficult for any penetration to happen at the service level for something to be able to access your data. So there are as with all good security programs there's not just one big wall there are many different layers of a good security program and it allows you to build applications in a zero trust mechanism. Oh, and of course all data both at rest and in motion is continually encrypted. So the only time there's decrypted data is of course during the actual processing. So some of the things to consider again around using these mechanisms for you to be able to develop a Calix application that is secure is a local Docker execution which you can do via CI CD pipeline if running Docker locally is problematic and sometimes there will be security constraints in your organization that actually forbid local Docker execution. So that's an issue. Our command line utility the Calix CLI is commercially licensed and must ideally be available on your development machine. We have actually helped some clients use a web environment and a web IDE to be able to overcome constraints where they could not actually run that executable locally or at least they could eventually but it would take along security validation process. So that CLI is an issue around making sure that you can develop locally and then of course the Calix specific SDK all of the different SDKs for each of the languages that Calix supports and the different frameworks it supports because for example there's more than one SDK for Java our open source and allow you essentially to more easily use the APIs of Calix. They are not binary SDKs in the sense that they don't connect in a binary fashion to Calix. It's a network protocol. So although Calix is running in the same environment as your application, you're actually communicating with it and all of its APIs via an internal network protocol. So from that point of view there's an additional level of isolation there. So certain attacks are not possible in that environment. Be each of these SDKs of course also goes through the same scrutiny that I described for Calix development in general that we consider them a part of Calix and their full bill of material software, bill of materials, their full supply chain is also validated and all of their dependencies validated. And of course any contributions to those SDKs still go through our process. So even if one of our users contributes a new language support for example, it still goes through our process before we will validate it. And as I mentioned, if local development is problematic which it is in some highly secure environments and banks and financial institutions then we have validated that Calix will work and you can develop quite nicely with cloud IDs and be completely isolated from your own internal network if necessary. So you still have to of course build your applications with security in mind. It's entirely possible for you to build an application that would allow some data to be publicly available and deploy it to Calix. If that's your choice to make that data publicly available you have to actually do it intentionally because as I said, you have to add routes to your application. So keeping in mind all of the OWASP standards is still essential for your development team. We do have tooling for example that can check for common security issues. I mentioned Fortify and the Scala plugin. There are many other language plugins for Fortify. Dependencies of your application must of course also be analyzed for vulnerabilities. So even though you're running in a very safe and secure environment you could be using a dependency log4j anyone that is known to have a potential backdoor and you need to be able to scan for that. So we do recommend the same kind of standards that we apply to developing Calix itself need to be applied when you're building your applications to deploy on Calix. And we do all of this internally for Calix. And of course dependencies also need to be checked for acceptable license terms for your use. Some organizations don't permit GPL applications for example. Okay so some of the facilities that I talked about before just to re-summarize and talk about the how they fit with the encryption of data at rest and the data encryption of data in motion are the transport layer security of course communication with clients access control list which I mentioned there's a link here for all of our documentation on this the ability to do client certificates if it's service to service communication looking for that's a good way to ensure the origin of requests and of course JSON web tokens which are extremely popular with web applications. And then of course on top of that port limiting such that no other ports could be used by an external party to potentially attack your application. So these are of course in addition to the built in encryption of data at rest and data in motion that I've mentioned a couple of times. So that doesn't require any effort on your part to have your data encrypted when it's actually stored with Kalex. We talked about routes we talked about the fact that routes are not automatically created. This is actually a security feature in and of itself because when you deploy a new service you must explicitly say here's the URL that can actually access that service. TLS is always required. Kalex can generate a cert for you which is very convenient when you're first starting up and trying to deploy your first few services or you can supply your own cert of course which typically would want to do in a full deploy in a production deploy. You can update, rotate your cert at any time. Client side certificates as I mentioned before are supported and there's again a detailed link to the documentation as to how to do all of these things. I'm just eyeballing the Q and A link there. Okay, I don't see anybody raising their hand so far and I'm trusting you to stop me if my satellite goes behind the cloud here. So let's talk for a moment about JWTs. So JWTs give you the ability to provide authorization and authentication for your service requests. We support both the symmetric and asymmetric keys, many different algorithms. Again, we can generate the key for you which is very convenient getting started or you can supply your own keys and you can utilize multiple keys for one service if that's appropriate. So a single deployment unit in Kalex can have multiple endpoints and those could be secured independently that the security doesn't have to apply all for one and no exceptions for that one service. We also support both the validation and signing of messages and all of this could be validated locally. So this is one of the advantages of Kalex is you can actually run what we call a local proxy and validate all of your services before you even deploy into the cloud environment to make sure that you're getting the results you expect. There's also a good blog post here and again we'll share these links that allows you to see how JWT and OfficeZero were integrated. So this is one of the experiments one of our team used. Client certificates is a great way again, often used in the situation where a service is calling another service so some external service is calling a backend Kalex service perhaps some other application that is running locally or in your own private cloud. This is more for course drain control. So this external party can in fact communicate with this entire service. Again, reuse an existing cert or use an entirely different certificate authority and routes can be individually assigned to require specific certs. So you can actually say, these routes are available to this set of certs these routes are available to that set of certs with configuration. And again, certificates can be rotated or replaced. I see a typo. Again, detailed documentation here for all of the, how do you actually do that? And then ACLs, which we brushed over very quickly before control which services and sources of request may access your services. So service to service calls are automatically encrypted. So the mutual TLS within the cluster of Kalex itself and principles are defined, you do callers and then allowed or denied to each service endpoint. So on an endpoint and by endpoint basis you can control with ACLs access to each of your services. And we do encourage this. Now it's entirely possible to bypass this and do your own security but this gives you a convenient way to A get started and also do a high level of security if you're using other controls on top of this. We also just recently added the idea of a service token. The service token is associated with a project and allows the bearer to exchange the service token for an access token to actually get access to that project. And therefore allows for the easy automation of things like CI CD pipelines. So for example, if your CI pipeline wants to actually deploy a new version of a service into Kalex and you don't want to have to stop and have a human do that well, they need some way to authenticate themselves against Kalex to say, hey, am I actually allowed to deploy a service? Generally there would not be, of course. This gives you a mechanism if you choose to use it to say, all right we actually do want to allow the CI CD process with the appropriate token to access our deploying environment and deploy a new version of this service. And then of course you can also set up multiple environments within your Kalex workspace to be able to differentiate between a development environment and a staging environment and a production environment and do the appropriate upgrades or migration from each of those environments according to your own development standards. Also I mentioned at this point somewhat related interactive developer logins can also be secured via multi-factor authentication. Something we've also just added fairly recent. Okay, that's actually the end of my formal presentation that cooked through pretty quickly. Does anybody have questions? I am opening my Q&A tab right now. I'll stop sharing and I will await if anyone has some questions. You can confirm for me that this thing. Yeah, it went, everything went well. Okay. Good job with that. It's great to have a much more comprehensive security offering now than we've had before, especially with Kalex. Yeah, nope. And realizes that from a point of view of a hosted service security becomes front and center and we're taking it extremely seriously as you know. Yeah, plus I like from the standpoint of the, for developers, the things that we can do. And that was actually, there's no questions but I think you and I put together some questions that we could just kind of cover that we thought might be interesting. And one of the questions is, what are some of the common use cases? And like I was just saying, like common use cases from the perspective of developers implementing their services and exposing them out. You mentioned JWTs, Jason Web tokens, right? And certificates and ACLs. Is that really kind of the main thing that developers can do is to control security or? Well, there's really a couple of use cases in there. One of them, like I mentioned, client-side certs are very popular for service-to-service communication, where you wanna say, okay, unless that other service is authenticating itself appropriately, then deny access to my Kalex service. Whereas JWT is much more common for, it can be used for service-to-service communication, but it's much more common when you're building like a single-page web application, something of that nature. JWT in order to validate login or have a login procedure. And I see there's actually a question in the question answer and user auth would be handled outside. Unfortunately, it's the usual answer. It depends. Not necessarily outside in the sense that if you wanted to authenticate a user via let's say an SSO mechanism and then pass in a token to say, okay, JWT or something that says, all right, this user is this person with these authorizations and therefore they should be allowed to access services. That is certainly one way to do it. We've seen that happen. There's nothing stopping you running a login mechanism. In fact, we've got a couple of samples of this within Kalex itself. We actually can handle the back and forth and the flow even of an SSO style authentication and user authorization by building Kalex services to do that, that's possible. So either, I guess this is the correct answer to your question. I hope that answered you. But the ultimate authentication that would be external though, right? So to answer this specific question it's a great question that, I mean, we don't do any... Do we do authentication or do we depend on an external thing to do that? Well, we give you the tools to do authentication. Let's say that. So there is no mechanism, there's no like built-in login page that you could just use within Kalex. You could build one by building the appropriate Kalex services. And then by using the ACLs and using the JWT support you could get that kind of authentication or you could do it entirely external to Kalex. For example, integration with a tool like Octa or one of the other SSO providers like Google and then simply having the user authentication not happen at all within Kalex then you just write your services. It depends really what you need in terms of an application. I would say in terms of how common is it to write something that's doing end user off in Kalex fairly uncommon. You're right in that sense that it's usually easier, usually easier to do it external to the Kalex services. Cool, okay. Just cause there's some great tooling out there. And of course we integrate very easily because of course being a cloud native environment it's very straightforward to get things to go back and forth to Kalex services as required. Okay. Yeah, another one that, and you kind of alluded to this it's just how do we make sure that we can meet the, you know, that the customer can meet their compliance requirements. You know, is there a document or documentation? And I know there's documentation and security but is there something say that the teams that are doing the development can give to their security organizations to show how their Kalex application is, you know, compliant? Yes, definitely. Your compliance and security being slightly tangential in the sense that you need security but at the same time from a compliance point of view and potentially even a legal point of view for regulatory purposes, you need compliance and you need to know that your vendor supply chain including flight ban who would be one of your vendors in this situation is compliant with a set standard with the appropriate standard for our country. And in the US that's basically the SOC2 standard. I say basically simply because it also incorporates things like GDPR which we're also compliant with and some elements of the ISO 27001 standard as well. What we have is a comprehensive set of documents that define our policies. So often an internal audit or an internal security group will want to see those. And with a relationship with the customer we can of course pass on the appropriate ones. It's a pretty beefy document so we don't just sort of hand it over but selectively organizations will want to see particularly financial services organizations will want to see who is validating that we are in fact following our standards. So that would be the audit report that we're doing right now where an external party says, okay for these trust criteria within SOC2 we will validate that light band actually meets those criteria. Typically that's a good starting point. Often we will also participate in very frequently do what's called a vendor questionnaire. Essentially your security department will ask us a series or usually we'll have a portal whereby they can ask us a whole series of questions and provide proof and documentation in some cases diagrams and other data to validate their concerns to go, okay are we doing this? Are we handling physical security? Are we handling background checks? Are we doing, you know, they will have a whole series of questions that they need for their own internal assurance program. And that's part of what we do as when companies sign up with Kalex if they need to go through that process we'll facilitate it. Usually it's pretty quick sometimes it can be, you know as many as weeks before we actually get through the whole process. But that's part parcel of what we do. So how do the customers reach out to us to begin that process? What's the typical way that they would connect to us? Yeah, the easiest way is to actually ask a question via our support mechanism within Kalex. So there are links within Kalex to go to our online support. If you say, hey, I need to do a security questionnaire or I need to, you know have answers around compliance and, you know, my own internal compliance needs then we'll get back to you immediately that way or you can reach out to us via email. Okay, great. Yeah, it's not an unusual request. We're very happy to get those and very happy to handle them. Yeah, excellent. Next question was about where's the data actually stored? That's kind of an interesting story with Kalex, right? Yeah, absolutely. The data is stored eventually does end up in a database even though you don't need to deal with the database as a customer of Kalex and where that database physically is goes back to that discussion about data sovereignty. So if you do not specify then we will allocate you in an appropriate region based on where you are as a customer and you will be in a multi-tenant cluster but in your own independent database. So the databases are always completely segregated. There's no opportunity or risk of you ever overlapping with someone else's data. If that's not sufficient for your own internal compliance and sometimes it's not or if you have specific data sovereignty needs, e.g. you have to have your data in Switzerland, for example, which was one request from one customer, not just the EU but Switzerland specifically, then we need what we call a dedicated cluster and this is a value add that allows us to say, okay, this cluster is set up in this data center or in this region of AWS or GCP or Azure, again, your choice in this particular place and typically what they will do is have a multi-zone region such that we have the redundancy where there are actually multiple data centers so that one data center fire can't take you offline kind of thing but then you have control over selecting where that region is going to be. That allows you to say, all right, my data, my backups, all of my processing are happening at this specific place. So you get to choose, I guess is the answer to the question that I love. Okay, all right. Well, no more other questions. So I think we could wrap it up here. One last thing though is where is the best place to go and look at some of the security documentation. Aha, that would be the Calix documentation. So if you go to calix.io and then go to the documentation, there is a section under services called securing services and we have a link in the presentation as well when we ship that out. And that securing services is the keyword you wanna search for in the Calix doc and it will show you all of the sections that we just talked about. Yeah, and if you go to the Calix homepage, like you said, Calix.io, there's a developer link at the top. You click the developer link and then right kind of in the middle of the developer page is that link to the documentation. So it's kind of Calix.io developer documentation and then services and securing services which is in the left panel of that page. That's the path. That's the one. All right. Always like- If you could answer the question at the time then please do drop us a line afterwards. Yeah, definitely. And really appreciate, again, I always learn a little bit more every time I hear you do this presentations but two or three times. I just love the story because I feel like we're covered, fine that we've got it covered end to end. That's certainly the plan. Which is a nice place to be. Yeah, excellent. Thanks a lot, Mike. Well, thank you and thanks everybody for coming.