 there we go well they want it for the video right low audience at home boy welcome to the apply rinse repeat get location agnostic session thank you for joining us at the last session we're we're gonna be we can be flexible with this I think we're tweaking a little bit of what the content is we'll go through exactly what that those changes are but the focus is going to be on building physical infrastructure cloud infrastructure container infrastructure and uniting it with software to find networking so big topic a lot of things my name is Rob Hirschfeld I'm on the OpenStack Foundation board I'm also CEO and co-founder of a company called rack N and we focus on physical infrastructure automation so big scale automation you might know me I used to be at Dell and I'm the founder of the crowbar project and we'll talk about that and I'm on Z-acal online and on IRC so if you might see me that way also so I'm Paranthap Lahiri you work for Juniper Networks specifically I take care of the solution engineering team for contrail and the part of that role I engage with customers in their hybrid cloud as well as on premises cloud deployment from a background point of view I used to work for Windows Azure in Microsoft online services and before that I was part of a unit network engineering so just before we dive in how many people are familiar with contrail open contrail not that many okay so we'll talk about that a little bit so and and then how many people have deployed OpenStack so that's good okay and how many people are familiar with Kubernetes and how that works okay so we'll it just if you want to ask questions about how those things work ask questions about how those things work we have backup material for it and we're gonna we'll talk through through those things this is the last session so we can be casual and we're we're here to talk about two technology sets that are adjacencies to the OpenStack project neither of them are OpenStack projects per se but they are both in and around the ecosystem quite a bit one is digital rebar which is the rebranding of the open crowbar project it's really v3 of open crowbar if you're if you're used to that and it's it's providing a complete provisioning system going all the way from physical infrastructure out of the box all the way through building a Kubernetes type infrastructure and then recently we've been changing it to make it so that it can run not just physical but cloud infrastructure also so you can do a Kubernetes deploy against an OpenStack cluster and we'll talk about that so we are going to talk about open control today as you some of you might know already that it's a virtual overlay network system which goes beyond a single orchestration system so you can have OpenStack as well as Kubernetes as well as VMware and you can have any orchestration system spawning either container VM or even bare metal and open control can tie them together in a single virtual network specifically but you can also do policy between those virtual network and every container or VM gets its own specific IP so essentially the virtual networking needs are all solved by control so we are going to talk today about how digital rebar and open control works together and and what I'm really excited about in in the combination here is that we're really talking about building end-to-end infrastructure right making cloud infrastructure truly agnostic and that's a lot of what what we see in market what we see people wanting to try and do they don't want to be locked in right OpenStack's very much about not having lock-in but that lock-in extends to other cloud infrastructures like Amazon, Azure, Google, VMware where people want to be able to create portability between those infrastructures so we want hybrid clouds and as anybody's been in industry knows that's really really hard we need open platforms we have those we need distributed overlay networking we have that we have consistent scale operations we've got that and one of the things that we need to be able to pull all this together is learn faster right these are complex infrastructures it's been really hard if this is a this you know a very simple description just showing the layers right it's from metal your first network your cloud network your container network your container infrastructure we haven't even gotten to your own application yet so it's a lot of stacks a lot of things that have to work together a lot of pieces that have to fit and then you multiply this across a hundred nodes or a thousand nodes yep it's a big deal and so when we look at the head their infrastructure needs what we're saying is that you really are looking at heterogeneous infrastructure right the reason we're on stage together is that you might say well I just want to run metal put sestian on top of that and then run my application on metal right that we can do yep you might want to do metal put in sdn networking in kubernetes and then you might want to then build a whole stack you might want to say well I'm going to do open open stack with software to find networking in neutron and then on top of that build kubernetes and why would you want to do all three of those things what you're trying to do is to create a consistent networking topology right so you want your application to be able to mix and match different types of infrastructure together so your kubernetes application might be able to talk to a Cassandra database or a Hadoop database that's running on metal and then still have a single network unified yeah absolutely and what we see is people want to consume services which is running in a traditional oracle database or in a oracle bare metal and it only exposes a single IP endpoint which may be an application that is written on kubernetes wants to access and they want to put it in a single virtual network so all those kind of use cases gets enabled in this approach so if you can do all these things then you've created a private infrastructure right so the difference between this previous slide this is working in your own infrastructure where you're you own the metal but here we're talking about something that where you're mixing and matching not just different types of infrastructure that you own but different types of infrastructure that you might access without owning it so public clouds from that that case because do you want to talk about some of the capabilities and how how you actually unify this type of network right right absolutely so essentially what you see here is there is an on-premise network which could be having one infrastructure at the service layer which exposes all the VMs actually yep so yeah this is a good one so if you look at this particular slide on the left hand side you see the on-premises private network which has a physical underlay then you can have a infrastructure in the service layer which gives you VMs and be it spawned by VMware on ESXi or OpenStack on KVM and then on the top of it you have containers that are our pods that are sitting inside those VMs so those pods might be in its own virtual network now if you look at the true networking need that pans across public as well as on-premises cloud it actually connects in each of these three layers and the first underlay could be a direct connection like you have Amazon direct connect and I think Express Connect is there from Azure or you can do it over internet and then you need a IPsec tunnel to connect the VPC in AWS with your on-premises network and then you have the public cloud which runs the Kubernetes clusters inside it and that has its own network which has to extend to the private premises so that is the complexity we are talking about here and we do have technology which does this but the packaging part and how do you and that's the digital rebar comes in where you can rinse and repeat the whole infrastructure so this makes sense so you know if things can communicate that doesn't always mean that the application see it as a unified network so when you're building an application that spans clouds you need it to look like a unified network you need your routing you need load balancing you need you know different ways to approach the technology that don't present it as three completely segmented networks exactly exactly and so one of the reasons this is so hard is there's just a lot of moving pieces there's a lot of configuration that you have to get exactly right there's a lot of learning that you have to do you have to account for you cloud infrastructures and physical infrastructures and we've been talking about something called the fidelity gap so fidelity in this case means that each one of these environments is so different that when you learn something in one it's a development environment the learnings from that don't translate into the next environment so if you go if you've got it working in DevStack then that that does not translate at all into your test environment that doesn't translate into proof of concept that certainly doesn't translate into production and those gaps cost a lot of time they cost a lot of money and what they're doing is they're breaking automation so when you try to move between these different infrastructures from your desktop to an internal cloud to a public cloud those steps end up costing you a lot of time they end up costing you a lot of communications gaps right if the tester finds something that broke the developer doesn't have a good way to replicate that fix that bug in the future those that's what that's what we're talking about with this and then it's even worse fidelity gap with you know in this graph it's traditionally saying oh this is just a single path through but if I'm building a hybrid architecture then I actually have a fidelity gap between my physical deployment my cloud deployment and now I'm going to constantly struggle with my deployment because I'm literally sitting in two completely different environments that don't have fidelity so our goal here is to address the fidelity gap by creating a smoother way to do automation between them right faithful operations in every environment physical virtual desktop and then that would translate into a faster and faster deployment cycle so this is one of the things that we think is really important is that if you want to do this type of complex environment you know we're not trying to hide that it's you know that we've we've gotten a big easy button that's going to deploy heterogeneous clouds and distributed architectures and build software to find networks between them yeah the thing that that's going to you're going to need to do is learn iterate improve and go through that cycle the faster you can go through that cycle the more the more chance you have of succeeding in doing it right if you can even if you could do it once you might not be able you're not going to be able to maintain it right and that's where digital rebar comes in and what we've been doing with digital rebar is actually building that up and I have some slides that that support that but this is the type of hybrid infrastructure that we're talking about building I love this slide just because it's complex go ahead did you want to say something on this is exactly actually it kind of is a logical plus physical slide kind of combined so if you look at it you have the blue site which is on premises which could be VM as well as bare metal and same for the public cloud because your soft layers of the world which provides you the bare metal servers right now on the top of it you use digital rebar to build this clusters and the clusters come alive with control built into it and I use the control capability to create all this virtual networks on the top which includes service chaining as well as policy-based interconnect between virtual networks so the combination makes the whole thing pretty powerful because you can typically put in the people who have done enterprise networking they know that the network segmentation is a very prime need for all enterprise network because you have to put firewalls there you have to capture logs and it gets pretty complex and you can get all those features turned on by control when it gets built by a digital rebar and this is a little bit you you probably want you might want to drill in on this so this is the same this is the top layer if you will have that it's probably worth describing some of right and so we talk about overlay network and this is a quick slide to show what overlay network actually means so essentially on the bottom side you see a physical environment where you have the hypervisor with VMs or containers sitting on the top of it with the V router sitting there and on the other side you have a similar V router sitting on the other side now what overlay network does is essentially using the control plane that control has it understands the endpoint encapsulates the traffic to the endpoint with whatever information is needed in the header and then it essentially it kind of delivers it over a pure IP network so you don't have to redo your IP network put L2 domains into it and do all the interesting stuff in the underlay so it is completely underlay agnostic and you pretty much have whatever you have on the top here which is if you have three separate networks you connected through a policy which is part of control or you can do service chaining which connects the blue network and the yellow network with a virtualized firewall all it could be a bump in the wire IPS idea service so all those capabilities kind of come together I'll pause for questions right this is this is a challenge to do in part because every time you add a new component for there's your infrastructure the infrastructure needs to be aware of what's in the infrastructure so there's a circular cluster that it's like building a cluster right when you build a cluster you don't get to randomly have things show up in it you actually have to know what the scope of that cluster is and coordinate those activities so it's not just you know oh I added a couple more servers in and things just magically work you out you do actually need to coordinate those activities so that you build that that the extent of your network that makes sense this is one of things that if you've tried to build any of the networking technologies you know one of the things that makes it really hard is understanding how those bridges work in the routers and all the networking topologies it's it's the significant challenge is not necessarily making the software work the software does what it needs to do is configuring all the pieces so that it has the data to work correctly all right yep to just to add to it like if you have a physical network firewall that is sitting in there and the corporate policy is to take all the traffic and make it go through that physical appliance then redoing the entire under the infrastructure and putting all the VLANs and making sure all the traffic goes through the whatever physical appliance you have it's pretty hard and whatever agility you have in your cloud kind of goes out of the window the moment you try to do it in a quicker way so that's where it all comes together this this is part of the we have a couple more slides ago I'm going to pause for a second but part of the apply rinse repeat the title of this talk and we're still building on that to an extent or through it is that none of these configurations are going to work the first time and this is this has been something that that I know from having done deployments and trying to help people do deployments if you start with the idea that you're going to know the extents and bounds of your deployment and get it right because you wrote it all down you're going to find that that's that's a very hard way to do a deployment it you're you will always find things in doing a deployment that you didn't realize were in in process or in situ or on the site and the number one thing number one piece of advice I give in any deployment is be prepared to iterate iterate iterate the faster you can turn through that the more likely you are going to be for it to be successful right if you show up thinking oh I'm going to install the servers and get it right the first time and and plumb my networks and get it right the first time you don't budget for going back and tuning it and tuning it and tuning it then you know right you're gonna be in trouble you're gonna discover firewall in lack of ports open right you want to talk to this slide to sure sure so essentially like if you look at a modern day infrastructure that we are kind of looking at you have the on-premise resource pool by resource pool I mean there is storage and CPU and whatnot and then you have the public resource pool now the type of computers that you have there are containers and pods and then you have bare metal there and have virtual machines and then each of them could be running different applications on it or it could be running application from different tenant right now if you want to bring them together in separate network segments which is the top rack there where each tenant actually has one or more virtual network that is actually holding them together and making the interaction work with each other inside that virtual network so it is a complex problem and we cannot trivialize it or be little it and that's where you have to go through the iterative process and that's where digital river becomes really important where you can make it learn and at one time you reach it and more so like think of dev and staging and park everything spread around all of them so it kind of becomes really really important and complex for orchestration and is really not an optional thing at any level of this this orchestration if you're gonna build this type of infrastructure you have to have the right sequence of events you have to be able to repeat them you have to be able to span different data different elements of your infrastructure and say all right I'm gonna talk to my controller now I'm gonna talk to my DNS server now I'm gonna talk to my actual node right so it's a very much a coordination exercise and this is one of the things that leads to the fidelity gap if I'm doing a development system and I've got to sit two machines I don't even think about the fact that I'm tweaking one than the other or I'm on my laptop and like yeah I always have to kick this this action or repeat it or do these in the right sequence because I'm just I'm just developing and so I know those steps if you're gonna do a hundred node deployment it all of a sudden becomes much more meaningful if the nodes boot out of order right or I bring up the nicks in the wrong sequence or I don't put my IPs in correctly so those those elements the repetition here is that you can actually orchestrate things repeatedly and then do it in smaller scale because what what we're really talking about being able to do is get things in the right sequence just looking at I mean this is a simplified version of it but if I was gonna build this infrastructure I would actually have to go through and do it in the correct sequence so Kubernetes if your Kubernetes uses at CD is a database at at CD has to be built into a cluster if you're gonna do HA and then when you go turn on the Kubernetes infrastructure part of what you have to do is plum in the networking plumbing in the networking means actually doing a kernel patch getting that applied then you have to retell Kubernetes to use the networking every time you bring up a new container you have to crosstalk and tell that container which network to use and then identify the network traffic maps from each container coming up into the infrastructure right it's a ton of work some of that's handled by Kubernetes thank goodness but all of those steps have to be done in the correct sequence repeatedly and so what we're really talking about doing is understanding that if you want to be successful with this you have to start automated oh okay yeah so and so if you look at what this cluster is I was gonna throw a demo in this if you look at what this cluster is we're back to this slide but and I want to spend some time on this in Q&A but you know it's it's multiple clusters it's multiple pieces all of these things trying to work together and build this connected infrastructure so it makes sense yeah so pretty much that is the last slide right and we have we have some more we're doing great on time so we have some time for we have a lot of time for Q&A and I have a microphone for it and this links to more more information that was fast yeah so we we we wanted to not overwhelm you with technical details we could have we actually I think the first pass of this deck was tutorials on Kubernetes tutorials on digital rebar tutorials on Juniper and up in contrail and we thought we'd leave it for questions to figure out what the audience wanted to hear from that perspective questions but how many how many people have seen Kubernetes and how Kubernetes operates would people be interested in hearing a little bit more about what Kubernetes does and what the architecture looks like for Kubernetes is that a yes that's me so Kubernetes uses Docker and then so that the thing with with Kubernetes is that its job is to take individual Docker containers and build clusters from Docker containers so what you have is a YAML file that describes here is the list of containers I need to I need to create for my map my application it puts them into pods which are sort of these these tenant clusters and then it tells you just like in a Docker we use Docker composed for digital rebar it tells you map this port to it this is how to direct traffic this is its life cycle this is things it's connected to so there's a lot of information describing the applications architecture in that in that file that that configuration file including where to go get the the containers themselves and if a container changes there's ways to go and say oh well update this container to a new version and take care of all of that house keeping so how many people have used Docker to actually package an application or run an application in Docker wow okay so we'll step a little step even let me step back even further that's fine this is I love having these conversations because this is this is how we you know we people need to understand how these things work and that's why we didn't want to dump a whole bunch of here you here's how contrail ties into because it would it wouldn't make much sense if we jumped at the end so the reason why Docker is really interesting is there's a couple of reasons but what Docker is going to do is it's going to create a portable operating operational environment so you can install your application the way you want so all the packaging installs configuration pieces like that and then the infrastructure like Kubernetes is going to start that that container up and then expose it to the network so Docker by itself you can put things in it you can start them and then you can play with the application it's pretty handy as an individual thing no applications run as individual things they they run in concert with other services and a lot of what the what you'll hear with Docker development patterns is microservices so you're really taking your your big application and breaking it into lots and lots of small pieces and then the small pieces you need to coordinate their activities so that if my app so I'll go but let me go up to rebar and explain it so if if my application needs these services then that will translate into actually building the full application so you decompose your application then Kubernetes helps you build that application back up digital rebar uses compose which is sort of like a lightweight version of Kubernetes so we have to bootstrap so we install Kubernetes it's not there yet so we have to bootstrap with compose what we've done in digital rebar is we actually took a full data center provisioning architecture and collapsed it into it's about 12 containers so that means that are the digital rebar application itself has a web server it has a orchestration engine it has a database server and it uses something called console which is a service registration bus a service registration distributed lock managers what we're calling it in OpenStack world right now so it needs those things to run and then to run a data center you need DNS and DHCP and a provisioner you need a half dozen services to actually run your data center right you can't you those are the minimum sets right I have needed time synchronization and names and I have to have a way to install an OS so what digital rebar does is it'll bring up containers with all of those services in it and then because we're using compose it actually brings them up in the right services and then console connects all those pieces together I'm not in a position to crank up a demo that fast there's I have some videos online you can watch how this stuff works but literally what happens is we bring up all of these containers and then they register with each other and then we wire all that work together so when you booted it when you boot up a new physical infrastructure or VM or attach something from the cloud it comes in and says okay I know who you are and then it starts putting you through the paces of all the data center infrastructure okay the fact that we do that in an automated way means that you now have the base you have to start at the base to start doing it if automated provisioning of all the pieces so you can say oh I know how to SSH into this I know how to install Docker I know how to install Kubernetes I when I first I have to before I can install Kubernetes I have to build an SED cluster then I have to install the master infrastructure then I have to and then you need DNS entries for those so you can build HA clusters and then you have to build the minions which attach to the master you have to do all this work in sequence if I'm going to test my open-contrail networking on top of that which is what we're doing then you actually have to inject open-contrail steps in the middle of those sequences so when I'm sitting testing open-contrail Kubernetes deployments I'm going to go through that cycle a hundred times and the sequence matters I'm going to spend a lot more time testing the sequence than I want to test installing the operating system or the radar bios or you know we do all that stuff it's it's work it was reliable what we're doing in building Kubernetes deployments is we're learning how to make all that stuff work so we're doing that so you can start but then you're going to show up on your site and you're gonna you know what my my IT department says I have to integrate through the firewall and that means these network rules are this and I want to do some work in Amazon and that means I have to have these paths to the internet those variations are doing you you're not going to know what how to fix them the first time it's going to take you learnings to do it so the whole infrastructure is designed to then go through and say alright this didn't work tear it all down go through and do it again and the thing that were showing here is that you're you're literally building up all these little services wiring them together and you take the next level build a whole bunch of servers and services and wire them together that's the that's the design pattern that we're talking about and for us the thing that was really fun to me is when we when we rebranded to digital rebar we also switched to this microservices approach where we're literally that that services oriented mentality goes all the way down to the very fabric of how things are structured okay oh here's the console pieces so that's so Kubernetes is taking that same approach it does it for your application if you want to build an application uses Kubernetes you're going to take your bigger application and break it into smaller really progressively smaller and smaller pieces and that's something that people experiment their way through but what's nice is that you can say Kubernetes keeps something if you have 10 nodes of scale capacity and it'll it'll make sure that if a server dies it'll bring up another couple nodes do you want to talk about the integration sure yeah let me let me talk about using the same slide so essentially in Kubernetes you have the concept of pods and each part has multiple containers inside and is that better yeah we can go from there so now where control comes in is you can take a pod and give it and put it as a part of a virtual network and each pod is going to give a complete specific IP address with the IPAM that control enables and essentially a set of pod would be a separate virtual network that now you can use to connect it to a VPC in AWS or make sure the traffic coming to that virtual network or coming out of that virtual network goes through a set of firewall be it virtualized firewall be it appliance hardware firewall and all those iterate that and you are going to find those out partly when you when you are trying to design the service and partly when you are actually going to run the service so all the flexibilities are not obvious upfront so now you created and now maybe you figure out that hey you need three more virtual networks because there are three more policy level we have no idea and this three more policy level is actually dictated by the operator of the eye who runs IT and this is that you know what this particular service has to go through IPS scrubbing and now suddenly you cannot redo the entire thing you needed a capability where you can bring in the dynamic way of putting all those pods in a separate virtual network and make it go through a different service chain right so those are the capabilities that get enabled with control so I want to highlight what pods are right okay so and I think you and I both add different things to this definition it's it's fundamentally it's pretty simple if I'm bringing up an application Kubernetes is going to put it put my application into a pod so it's a domain boundary what what we're saying is that in that pod the networking is is going to appear as if all of the networking is local within that pod even if it's distributed right Kubernetes is going to provide that capability because that it's it's a technology out of Google but it basically makes the assumption that if I have a pod all of the elements within the pod can talk to each other and if I want to now have a pod that spans layer two boundaries or data centers now I've got to deal with external rules which is what you were describing right right so so if you can imagine you know I've got I want a pod that spans Amazon and private infrastructure or an open-stack cloud private infrastructure then that's going to go through a firewall which means your IT department's gonna actually care about that route and so you need something actually builds that route and it's not for persistent route it's not a VLAN when that pod is created you're actually in every containers attached to it you actually have to build the networking between that pot that that container and the other containers in the pot right it's it's that dynamic right so I I say oh I want a Kubernetes scale up it adds 50 containers those 50 containers go up on 10 different hosts maybe in two different data centers those those network connections actually get built for you dynamically so that you now have built this infrastructure that's I mean this is complex stuff but once you've built it it's really powerful no you're absolutely right and just to dig little deeper into how Kubernetes handles pods and the nodes so you have the cube minions that was mentioned there now node is basically a single server or a single VM which has a single IP address from the infrastructure but that can have n number of pods inside it and now in that in the simpler way of handling the networking people have provided a separate overlay private address space for each of this cube minions or cube nodes and now all the pods ends up getting an IP address from that particular block now you are in a situation where you cannot actually impose a network segment in that kind of situation because in the IT world each of the pod needs to be part of a different network segment so you can make it go through a different policy chain because that's what IT and Enterprise lives through and of course in some multi-tenant cloud infrastructure they have the same infrastructure same implications and same kind of boundary implications so that's where a single IP address of a node and getting translated into multiple disparate IPs of each of the pod and then that's where you need a overlay layer which is smart enough to provide a pod a particular IP based on where it belongs to and what we do inside Kubernetes is we use the label concept inside Kubernetes and you can give a name it's a key value pair you give a name and give whatever unique name and that becomes a virtual network and essentially you get an IP address from that virtual network so if that sounds complicated it is it is the thing that we're advocating here because we think it also is incredibly valuable to build this type of infrastructure what we found is if you want to be successful doing that in production which is where everybody should want to be successful the closer you can get to doing that in your development environments and your testing environments the more successful you'll be right you're gonna find out these issues you're gonna build it so what we what we don't want to see people do you can do it if you want I'll just I'll pat you on the back when it when it hurts what we what we think that is not good is if you start and do development and you're using Kubernetes yay but you're doing it in an environment that doesn't have robust networking you're you're not gonna find issues in your application deployment until you get into the next cascade over the next cascade over and so what we're really trying to do is make sure that you can do real deployments in very small environments and then grow them gracefully up so you start that learning process and you actually get into the the cycle because what you don't want to do so I'll step back the value proposition for Docker and Kubernetes and this type of model is that your your development environment is as close to production as you can make it that's one of the big things that people like about Docker I packaged my application it's immutable it's going to translate all the way through into production that will that falls down if you're if you're then to be if your test environment your Dev environments are not true enough to production that the networking breaks when you go between those environments right that that would be a failure right that that breaks the whole model so we're trying to make it so you can you can get really small in these infrastructures and then when they go to production you can actually have that faithful experience of all the way through that's right that's where the fidelity concept comes from okay and we're just about out of time yeah maybe we should open it up for questions any last questions we've been sort of taking hand polls for questions but no everybody's ready for happy hour and closing the summit down yeah yeah I'm in favor of that yeah I'm in favor of that yeah I think I'll we're gonna upload these slides I'm I'll I'll tweet them and post them out of out of doesn't have my Twitter handle so I'm vehicle online just you can look for that more you'll these will be off my slide share so yeah for a purpose when the signal to noise ratio changes maybe I will spoken like a network person thank you all for coming and this closing so if I hope you had a great summit wonderful thank you all right thank you