 Hi everybody. Thanks for attending the session. I think it's the last one for the day, so we'll have to try to make it a little bit light so people can go after this to take beers and have fun. I'm Paramon Kluz. I'm the city of Plumgrid. Plumgrid is a company that works on creating a cloud networking stack for OpenStack. I'll discuss a little bit about that, but the topic about the presentation is not necessarily about Plumgrid. I'm going to talk about some concepts for network security and policy and where these things should be enforced and how it should work in the OpenStack community and cloud in general, but the main idea is to steer some discussion. After the talk we can have questions and answers, but it's not about specific things on how most of the policy discussions that are happening in other sessions should be shaped, but rather than to elevate the discussion and see why are we doing the things that we are doing and are we doing it right or not. The first thing, as I proposed this title about putting network security and policy in the same subject, it took me a while to somehow start thinking about how to explain the concept. And the idea is that there's a lot of confusion in terms of what network security and policy means. And before jumping into the topic, I had to think and scratch back some of the presentations and some of the discussions we had in the past in OpenStack. So the first thing I did is I found a presentation that I did two years ago in San Diego in OpenStack Summit. So before I jump into the topic of today, I just wanted to go back a little bit on the concepts that we discussed back then and see if we have done a lot of progress, if we got things right, wrong, what's going on in the industry and if we are getting to the right place. Oops, something is going wrong. So those slides are like word-to-word exactly what was presented two years ago. So I'll try to go fast through them and after that we'll continue the discussion. The idea about networking is always the tension between enabling applications to communicate and at the same time when you live in a multi-tenant cloud or in a network environment where multiple people have access to it, there's a notion of how to prevent communication. And the notion of security versus connectivity has been in the industry for a long time. We fix it a lot of times putting policy enforcement points like firewalls and things like that. But at the end of the day it's a blur line between what's connectivity, what's security policy. In the sense that at the end of the day if you have a great scale of QoS from zero to infinite, zero could be like security policy, allow deny versus connectivity could be enable connectivity. The things are a bit complicated for two reasons. First is because somehow we build these complex networks and this is just a diagram coming from the recommendation designs from the Cisco safe guidelines. And somehow connectivity grows organically. We keep putting more switches and routers and things that enable connectivity. And then after the fact we tend to think about where do I apply security? Where do I apply policy? So somehow we tend to bake security and policy in a topology form. We tend to think in kind of a bidimensional graph. We understand where the workloads are, where the computers are, where do I have to apply my policies. But it's harder than connectivity because somehow connectivity is like enable everybody to communicate to everybody. Routing protocols will kind of enable all these IP addresses to be advertised. Security is the opposite. I have to understand where things are in order to enforce security. If you think though, this is kind of a mentality of after the fact. So people think about designing network security where the network has been designed. Now I have to put certain set of policy boxes, elements to make sure that it's secure. But somehow the approach to design security is slightly different. We should design secure networks in the sense that you understand why did we create the network? What were the business values that we are trying to deliver? What's the fundamental value of this application that we are going to deploy in the cloud? Which brings to the point that in reality it's hard because it's not about just securing our applications through security policies. It's about understanding what the business needs for those applications were to start with. For example, we are running an application in the cloud and the question is what connectivity features do we have to provide to let this application fulfill its business needs? And based on that then you say, okay, what kind of data leakage or what kind of security risks I'm going to impose to my users in a way that then I can translate it to policies and then I can build a security system from scratch. So somehow this linear thinking of what am I trying to achieve? Let's do risk analysis. Let's define the policies that are going to secure that and now let's implement it into a system, gets lost and most of the times we say, oh, somehow through some sort of network services we are going to solve the problem. So most of the times after the fact we think, let's put security policies or let's put firewalls or let's put other type of networking service devices. And we see that even in the OpenStack community that somehow we started with a notion of we are going to create a network, we are going to have network other translations, we are going to have switches and routers and security was an afterthought. Firewall as a service came a little bit later and now we are having this notion that certain components like networks and routers are thought as devices and firewall as a service is thought as a service that you apply to your application. So there is some tension in terms of the afterthought thinking security. The other reason why it's hard in cloud is because it's not only like the business needs from an application point of view. There's a cloud provider that is creating a service and this service has to grant these certain availability, certain security requirements, certain functionality, sorry. And somehow the business needs of that service provider is to offer a service that get translated into set of risk analysis, security policies, and implementation of the features that now when somebody uses the cloud on top, if the business needs that will translate to the security policies of that specific tenant, don't match to the services that the service provider offers, there's a mismatch. So that's why in cloud it's even more complicated than in your own enterprise. There's a set of things that are not under the control. On top of that the multi-tenancy aspect has two components. One is we need to provide the isolation between the tenants and then of course a strong isolation with a cloud provider but we want everything to be self-provision. The multi-tenancy aspect that now we want is like everybody with a credit card will be able to self-provision that which means that we have an access to the infrastructure that is controlling the cloud. That access can be used at the same time to take over the cloud and essentially appears across the isolation that the cloud provider is trying to to propose. So and then we start thinking about the application and we create an open stack project, we create a set of networks, we create a set of VMs and now we have to decide a set of enforcement points with Firewall as a service security policies which somehow they tried to mimic the way we were thinking in the physical world. In the physical world we baked security policies and QS policies in a topological bidimensional graph and somehow when we abstract the whole thing we tend to carry these concepts in the virtual world. Now the question is when we have workloads like VMs and applications that live in the cloud that they could move from one data center to the next data center but they could extend across the data centers is the notion of applying security policies to specific points in a network topology the right way to think because somehow this network now is an abstraction that our application experience in the cloud. So the idea is when we think isolation and SLAs now the question is who offers it, why, what are the users and we always have like two dimensions. The first one is what the tenant, the user of this cloud needs and this would be kind of the isolation, separate separation, security rules from an application point of view. The other is what the provider owns and wants to provide because differentiation between service providers between cloud providers is going to come by the fact that somebody will have secure physical placement, will have data leakage prevention, will have better security frameworks that can offer to the customer. So these things somehow they have to be transformed in two dimensions. The first one is what kind of enforcement points do I have and when I have the tension between the tenant and the service provider or the cloud provider do I commonly join these enforcement points or do I keep them separated? The second is how do I define those policies especially when they are going to have conflicting policies between the applications on top of the cloud with the policies that the cloud provider wants to offer. So somehow these were some of the thoughts that I discussed a couple of years ago but the problem was that that's not it. It's not about just security policies and policy enforcement. If we think how to create a security aspect for a cloud the idea is how do we go to the next level. When something happens how am I going to monitor the system? How am I going to detect that an attack is being carried right now? The next is how do I make sure that I can ensure my customers that they have set of compliance checks that when their applications get deployed into my cloud they are going to fulfill the SLAs and the business needs that I was granting them. And of course then there's incident response and forensics and SLAs and all that stuff. So this was just like to explain that when we think security and security policies we always tend to focus on the action of preventing something but when we will secure clouds we have to elevate a little bit the thinking and we have to tight monitoring incident response and things like that as part of the process and always with this dual role that in a cloud environment there's two types of users the service provider the cloud provider that offers the service and the tenant sitting on top. So based on that and the two years have passed from a security point of view now there's a lot of discussions going on and the question is if they are converging or not but at least things are moving. The question is why so hard? So now I'm going to jump to the presentation for today. The idea is what is cloud? I mean we tend to always focus about we are running this application and it's running on open stack and we have this notion of a project and we create the networks and we just make sure that everything works but when we think cloud this is not a cloud this is a data center. When we have a single location that is running a service that's a data center that may be multi-tenant maybe offered as a service but it doesn't have the availability grantees the geographical grantees that a cloud service will have. So we have to start thinking that we don't enforce policies into a place we don't apply a specific workload running in a place. When somebody is trying to create a cloud service around the world now the thing is like how do I distribute it? How do I make it resiliency to failures of a specific location? So we have to start thinking that this is a service the service may run into multiple locations and now the question is how do I abstract it? Next is the famous phrase that networking is in the way and the question is why is in the way? If we go from the previous slide that the cloud before was a place within data centers and we think how the network was designed was engineered was kind of a hop-by-hop concept. You go from one edge for one compute node to the top of the rack switch to a router to a firewall to somewhere else so somehow the application logic in terms of security policies, scalability policies has been baked into the fundamental topology of the network. Now the question is why is that? Because we could think that the notion of securing an application or scaling an application are properties that should be more attached to the application itself, not to the physical infrastructure or to the physical network, otherwise we defeat the notion that cloud is a service, the portability aspect. So the idea is we have to start thinking the same way. Networking is not a place either because if networking was a place how could we support an application on top of a global service? So the thinking has to be slightly different was has to go towards what abstraction do we provide for the network that is going to be able to run my workloads in a way that they are portable across clouds or across data centers. So following the same way we have to elevate the thinking in terms of what do we put under our VMs in a way that then when we have plugins for Neutron or Nova or any policy framework that we may want to create how those will map into a way that the application or the project is self-contained and it's portable to other environments and the policies that express the security and the scalability that surround that application they have to be attached to the application, not to the physical infrastructure. So if we think like that the question is then what's the perfect abstraction for networking? And if you think OpenStack did an incremental step towards not falling for the notion of network elements and links, if you think the way traditional networking where is this going so fast? Networking was defined was I create an element, a router, a switch, I put another one, I link them with a link and I start building this topology focus on devices and links and interfaces. When OpenStack created the notion of the network on the notion of a create a subnet I can assign directly the subnet, the default gateway, the DHCP address space in a way that now network is a concept. Still carries the notion of create the network connected to a router and so on but somehow the network properties that the VMs needed in order to boot and have address and connectivity were abstracted in something that was elevated from the fundamental physical fabric in a way that now it would not be explicit if I map it to VLANs or if I map it to overlays or if I map it to Linux bridges would be abstracted from that. Now the question is when we try to extend that now we want to add the routers and the load balancers and the firewalls and the intrusion prevention on the VPN service. We are starting to fall towards like creating and engineering this topology graph into a virtual wall that may defeat the portability aspect. So the question is how do we provide this abstraction that gives us the multi-tenancy that we need and multi-tenancy defined as kind of a security location and fault isolation. I have to be able to have this abstraction that encompasses an application and if the application has an issue it doesn't propagate to other applications in the cloud. The other aspect it has to scale. I mean some of the designs that we are exploring right now in terms of putting virtual appliances or elements in the network that become chalk points may work for a small scale environments but when we start creating cloud environments that scale may have bottlenecks that have to be addressed. And the other aspect it has to be this container or this abstraction that we are creating has to be kind of future proof. Today what we are doing is we are trying to elevate the physical constructs of switches, routers and firewalls into the virtual wall but what if we didn't have to express them like that? What if we could create a policy framework that we say connectivity, security, availability, things like that are policies and then an engine renders and translates it to different framework that maybe doesn't reflect the old way of thinking from networking. The other aspect of this abstraction would be the notion of simplifying the operational model. We have to be able to create an envelope that can be spawn, shut down, move, created, clone, templatized similar to what happened to compute. In compute if you think there was the aspect of what's a virtual machine. Every time we create a virtual machine for the first time you allocate 4 gigabytes of RAM to virtual CPUs, an IO, a virtual disk, a software image that attached to this disk and we call that a VM. So essentially a VM is nothing else than a container of elements that will give you a sort of compute abstraction. Now the next time you create a VM you may want to create it from scratch or you may want to clone that VM or replicate it 10 times. So essentially if you think the VM is a notion of a template or abstraction of compute that can be replicated as many times as you want. So if now we go to the networking aspect why not something similar? Why not the ability to create a network definition that you create let's say a 3-tier topology validated design, a policy defined network validated design, a single subnet validated design and now you say I'm going to attach an application to it and I'm going to spawn it 25,000 times. And how do I move 5% of them to another data center, to another location? So sort of coming with the right abstraction from a networking point of view applications will not be able to enjoy the mobility and the properties that compute has today. So the question is what would be this network abstraction? And of course has to be a service because in the cloud wall we cannot just assume that somebody will put a purchase order by a piece of hardware, deliver it, install it every time that the customer on application gets created. So sometime back we had those discussions and essentially at Plumgrid we started to use a terminology that is just a concept of a virtual network domain. The idea was that regardless of what's the network model you believe in, is it a traditional model, is it a policy defined network model, is it an open flow based model, is it something else, doesn't matter. There has to be an envelope that contains that network definition and then based on the identity or based on the properties of the VMs and workloads that have to boot into that system, they have to be associated around time when they get created. So we came with this notion of virtual network domains and then we explored the possibility of okay let's bring traditional networking models in those virtual network domains or let's bring more advanced models with policy defined networks. And these were concepts that we were developing a couple of years ago and we were always discussing what would be the natural mapping to the physical world and this is where the divergence of solutions start to come. There's always two approaches to solve networking problems. The first one would be I have this abstraction, this virtual domain, regardless on how I define networking on top of it and now I have to map it to let's say my physical infrastructure. In my physical infrastructure I would have let's say villains, virtual routers, firewalls, things like that. That would be a model to fulfill those virtual domains on the man. Of course that depends completely on the physical characteristics of the network and their ability to scale let's say beyond 4,000 villains or a number of virtual routers, things like that. The other model is well if we have to reinvent networking and we have to make it in a way that satisfies the cloud needs, why don't we rethink again what was networking about? And if you think that working was always two components, the first one was connectivity. It's how do I provide connectivity? The other is how do I structure such connectivity? Do I give you let's say a thank you with link or do I give you a villain that curves this thank you with link in multiple tenants or multiple users? So there's always these two components that the network has solved. So the idea was well we can do it provisioning the physical infrastructure or we could go kind of with an overlay based model that we create that infrastructure. And this is where the notion of provisioning an infrastructure blends because now in the virtual world in the overlay world we can create those concepts on top of a connectivity matrix. And this is where now these competing views essentially create kind of a fundamental disagreement where you see the world in two different perspectives. The first one is what we are discussing about how do I map it to the hardware. And this is what now a lot of people are calling hardware defined networks and they are some vendors that they are marketing and advertising it like that in a way that there's a proudness of saying I can map all these concepts to my physical network and my physical network is going to solve all these issues for you. And the other model is like well no you should have fundamentally let's say a layer 3 leave a spine architecture, configure it once, never touch it again and create all these concepts on top of that physical infrastructure. And you can use any physical infrastructure. So the idea was like what are the pros and the cons of these two approaches. On the hardware side of course you have this notion that if you provision the physical you have to buy a physical infrastructure that has the ability to perform those tasks. So most of the times impliers green field deployment where you have to get all the physical hardware. The other aspect of course is the supply chain management boxes and things like that. On the other side you have the SDN wall, overlay business SDN wall that somehow is nascent. There's multiple technologies there's five or six startups working on this space that give you the notion of I can create it instantaneously now. As you need I can create these environments. If I mean you can read about the details or the things the pros and cons from different but somehow if you think what opens like is is a software cloud infrastructure. And the idea is that when you think about platforms and platforms that are portable across environments you can have like a cloud provider rack space using open slide you can have a private cloud you can have red hat mirror and this piston you have multiple distributions. Essentially everybody is creating from a software infrastructure multiple clouds. And then you start thinking as a user of the cloud how do I have portability across clouds. It's not possible to force essentially a public cloud provider an enterprise and a third party to use exactly the same hardware infrastructure because you violate the notion that these cloud applications have to be portable across multiple locations because as we said cloud is not a place it's a service. So the idea that I'm trying to convey is that there's multiple ways to solve the networking problem. The first is how to settle in an abstraction that can satisfy the application needs. The second is how do we map it to reality and there's two paths that one ties it to a specific location the other ties it to a specific service that would be kind of the high-level concept. Now having said that what's happening is that there's some agreement in terms of what the solutions that are being proposed which going from this notion of what's a network abstraction and do we think it in terms of topologies do we think it in terms of policies doesn't matter. Doesn't matter in the sense that at the end of the day what's happening is that this industry is not being driven by networking issues. When somebody creates a cloud it's not asking do you have the best network or the worst network? I'm sorry. What's happening is like you built a cloud because you have a specific application or a specific use case in mind from creating a generic infrastructure service or a platform as a service cloud to creating a private or public cloud for a specific application like running a VDI solution, CRM, things like that. When you think like that you start from the top. For us it's like what application do I have to run in my cloud? Once the use case is clear you say okay so what infrastructure can provide me the framework or the platform that allows me to run this application? Is it an all enterprise application that requires enterprise type hypervisor to run because there's a certification process or this is a new cloud application that is going to scale and be portable at cloud clouds? Then you say okay based on the application and the uses that you'll have what cloud management system do you want to use? Open stack, something else, let's say open stack and now the next step is okay so what container in terms of networking storage and compute do I give to my application? So the thinking is going top down. It's not about how do we build these three pillars of infrastructure about compute storage and networking and applications on top but rather the other way around which is what applications and what use cases am I going to have in my cloud and then coming down the stack and at the end of the day there has to be this network infrastructure abstraction that we were discussing before that will bring it to the specific implementation of am I using a physical network? Am I using a virtual network? But from an application point of view has to be transparent. So coming back to policy the idea was like why is policy so complicated in terms of discussion? I mean there's like a few policy discussions from some of them from a networking site, some of them were from a global set of resource storage networking compute placement things like that. But the idea is that reasoning about policy is kind of a tool that allows you to define what kind of objectives do you have or what kind of enforcement do you have for your application and this somehow it's an abstraction by itself. If you think in terms of policies and this could be security placement others now the notion of where to put this VM or where to have let's say an over API that puts this VM or puts a security policy this becomes kind of abstracted by the policy framework where you just define what you want and the rest it will happen automatically. This is where you have a lot of different ways of being the wall from let's say heat as an orchestration framework that will drive through a policy infrastructure what has to happen or through direct APIs or other competing proposals. But at the end of the day what happens is that there's two aspects to a policy framework. The first one is what do you define in terms of what do you want to accomplish but the second is who's going to enforce and what are the capabilities of the entities that have to enforce those actions. And this is where we have to think hard in terms of to make an implementation happen those two aspects one is what can be done from the open stack community point of view the other is what elements do we need in the data planes in the Linux kernel in KVM in ESX in Hyper-V in the networking aspects in the storage aspect in a way that the policies framework on top can control these elements. So which brings back the notion of certain things like resource scheduling policy distribution things like that they are completely at the control of open stack. The other is what data plane are we going to use for networking what capabilities is it a hardware ASIC is it an OVS is it something else what approach do we follow how do we bring the functionality that we think that this policy framework has to complete. So somehow the the discussions get entangled into what's the perfect model the perfect concept and sometimes we tend to fall into specifics but one has to fit into the other and this brings down again the discussion about what abstractions what what goals and what enforcement do expect from these network containers the storage containers or compute containers in a way that when we think about policies when we reason about policies we have the right abstractions to control whatever infrastructure is going to be underneath and this is where open stack by itself as a cloud management system has to step into the details of certain elements or certain functionality needed into the compute nodes or into the storage things like that to define what elements are needed to create these policy frameworks. So at Plumgrid and this I'm just going to talk two seconds about what we do and how we focus on creating a cloud networking suite for open stack we took this concept from the beginning. The idea was if we had to reinvent networking from a data center point of view or from a cloud point of view the question was more about what's going to be preserved when everything gets cloudified and when things move into the public or private cloud environments and the idea was applications will stay and applications will have network definitions that will be embodied into this notion of virtual domains that will have to be able to satisfy the network applications or the network needs that the applications had. So what we did is we created this notion of iobizor which we created an extension to OBS that essentially gives you the ability to programmatically from a framework insert runtime additional functionality into the data plane. So we have an element that is part of this Plumgrid cloud networking suite that allows us to publish in a running system multiple network definitions and this would be from distributed switches, routers, network others translation security policies and so on. And we accomplished this through an SDK that would abstract that data plane with a compiler a compiler that would expose a definition language a data plane definition language that we could create new applications on that framework. So what we did essentially is like to create these applications and those network applications would be fully distributed in nature like OBS you have a distributed virtual switch the extensions that we add with our iobizor would allow you to create a distributed router distributed NAT distributed security policy in a way that when you think in terms of virtual domains and the topologies defined in that virtual domain you don't have to worry anymore about the policy mapping to the physical wall of before there's no notion of service stitching I have to go from point A to point B to insert a firewall and then point C to go to somewhere else because the idea was like how do you distribute into the compute nodes through an overlay concept all the network functionalities that the applications may need. And of course this had to support the insertion of additional features additional network functions from the ecosystem partners and all the network vendors that are part of the open site ecosystem. So with this I just wanted to say that when we think security and policy again is not a place it's a journey in the sense that we have to start thinking about how do we bring the notion of the networking features that the applications need the networking features that the providers need in order to fulfill the security requirements that those two users of the cloud have and not only that to tie it to the previous discussion about what other elements in terms of forehand fix and remediation and things like that that have to be part of this and this is not going to be something that is going to be accomplished just by a policy framework or by just by a framework that does the policy enforcement or just by the orchestration aspect all the three elements all the aspects have to go together in a way that a proper security solution has to be delivered to make cloud a reality and this is where we have to start thinking in terms of how do we operationalize open stack not only from the point of view of being able to insert certain features that are needed in order to secure the cloud but not only that but how to report orchestrate and accomplish the kind of the angle so just to finish the talk the question was like open stack again is not a place is a destination that we want to go that networking has to provide the proper abstraction in order to enable this glue that will bring the multiple locations of these cloud services together so with this I'll open four questions if you have specific questions about how to implement these things in distributed data planes and how do we turn those concepts from a product point of view and how do we implement it inserting an open stack I can discuss a bit more that or if you want to discuss more questions in the in the sense of what's going on with policy frameworks and why is it important or not and how it will conflict with the neutron APIs or with the network now IPS and things like that we can we can take that too any any question hi Nils Wart I have a question in the neutron group we're looking at things like group-based policy in order to describe relationships inside of the network in open daylight there is some activity around group-based policy and affinities and more recently there's policy activity here in open stack named congress what are some of your thoughts on the community-based definition of policy yes I mean I think let's start with congress congress I think is taking an interesting approach which is go beyond networking which brings a notion of what's networking right if you start thinking I'm going to have a set of resources like storage resources computer resources locations places SLAs networking and everything and you start defining a policy framework as a way to work astray the whole cloud that's kind of a very ambitious goal which makes complete sense and the question is how do you integrate this policy view that puts all the pieces together into an implementation that can offer the functionality what we are saying before about the policy enforcement points and the policy rendering versus the definition I think the ultimate goal were like congress wants to go make sense now the question is how to map it to the tools that we have available today then going back to open daylight and neutron maybe that is more bottom-up approach in the sense that let's start with neutron started more with the notion of I have an application that I have a type of machines web application and I create a set of security policies so it's more the traditional networking policy thinking in a way that now rather than I explicitly put rules from like a firewall service point of view I just defined type of workloads identities roles and now I'll let a policy rendering engine translate it to whatever policy enforcement so definitely I see that the neutron kind of approach more practical short term more grounded in something that you can accomplish with an original set of implementations underneath and then the open daylight it's in between right you can spin it towards networking and then plugging open daylight into neutron or some other way to interface it with an open stack but has the ability to start thinking in terms of location services and things like that so to me the biggest issue is not the fact that what do we think about wine versus the other versus the other it's more the fact that why do we have so many competing views of policy and how do we bring them together and open daylight is somehow trying to be part of this framework that is going to be inserted as a way to express policy but at the same time open stack has to continue with shown definitions so I think the issue is more not it's more how to express it in a way that then orchestration layers like heat or TOSCA or whatever it's going to be on top they can use them in a way that we don't get stuck again with the specific details of different implementations and we finish with abstractions that they cannot be portable between themselves that would be my major concern but I think that all of them have valid points and the more the merrier provided that they are interoperatable and somehow the elements underneath like the Linux kernels and the KVMs and the dockers and the Linux containers and the OBSs or the iobisers all the elements underneath they can fulfill the functionality that those frameworks require any other question okay then thanks for attending and have a good evening