 Well, good morning, all. Everybody's wide awake this morning? No? You didn't get your coffee or sugar before you came in the room? Well, we're going to go ahead and kick into the dive into the discussion. There's three of us that are going to walk through some material today. I'm Billy Cox with Intel. I have Shua from Huawei and Kapil from Canonical. We're going to go through some topics around provisioning and managing OpenStack from a software-defined infrastructure perspective. So let's see. Let me dive in. So for those of you who actually read the agenda description, I'm not DOS ChemHout. There's no way I can possibly replace him. He missed connections on his way over here and had to be back in the US on Thursday. So I don't know if you say I got delegated to or flew to my lap. I'm not sure how you want to do it. So I will do my best impression of DOS, which is not possible. Nobody can replace DOS. But I am very familiar with what we're working on, because my teams are actually doing a lot of the work here for this particular chunk. OK, now we're not getting slight advance. So I think we started off life with I need numbers and I need a way to arrange them. So the way I defined my infrastructure was I used my fingers and moved it around. Started life pretty simple. We then found ways to automate it, but it's still not defined by software. It's still defined by, OK, I need to go move some cables around. And so I hire people, it looks like ladies, to go move cables around. I don't know. In the US, it used to, it was more like a person-defined infrastructure, right? So you pull the cable out of one and put the cable in the next one. And we now find ourselves in the era where we know that we have to do more. We have to do it a lot faster. We each carry how many devices, how many, what's the average number of IP addresses per person now? And that increasing number of devices drives demand in the data center, which then drives more services, which enables more devices. Intel thinks of it as this virtuous cycle, right? Every time we add new capabilities, it drives new uses, which drives new capabilities. Wonderful situation to be in. But that same driving means that that patch panel room where the lady goes and moves the cables, she's not going to be able to keep up. And when we look at the volume of change, the volume of activity we see coming, we see the drive to reduce the cost of compute. And we still, at least as Intel, we still see the elasticity in the market. So every time we reduce the cost of compute, the demand for compute goes up. And part of it is cost. Part of it is how easy it is to access that compute. Amazon showed the world how to do it with an API. And we now are at an OpenStack conference that has established a means to go drive that cost down even further. And we see, of course, the data volume, the number of applications. I keep wondering if this application number in the era of containers and dockers is going to get higher, right? It's not. This is kind of like the starting point for where the growth is. But all of these growth factors drive demand for us to deal with it in the infrastructure. So how in the world can you take that kind of growth, that kind of demand, without being able to use software to then go define what you want the infrastructure to do exactly? So the first wave, and we call it software-defined infrastructure, at least at Intel as we work with our partners. But what happens next is, OK, if I now have the ability through software, through programmatic means to call an API, get infrastructure allocated to me, configured the way I want, like, great, I'm done, right? What happens next is you're going to want to know, am I using it as I intended securely? Am I efficiently using it? Are there underutilized or overcommitted resources in my environment? So that happens next, right? And first, you've got to be software-defined. And then when you've got the software I'm doing it, how do you know you're doing it well and correctly? And so we tend to call it the watcher-decider-actor loop. You need to be able to watch the infrastructure. You need to do analytics to make decisions. And then you need knobs to turn to be able to make sure that you can make change in the environment, right? So collect a lot of data, do the analytics on it, and then watch the services that are being measured, pull levers to adjust them. So that's another form of a virtuous loop, if you will. So it's great that we've got that spiral of every increasing demand, but we've also got to find ways to maximize utilization. And the great news is, with OpenStack today, you can get that API. You can do some of the data collection, but there's a long way to go to really give us the richness of infrastructure that we need. And from our worldview, we get a little data today, the top peak of the iceberg, but we're actually missing huge chunks of the underlying data. And we've seen a number of examples in one of the products I do that plugs into OpenStack, where we see operational data on VMs from the infrastructure side, let's say, looking up, that if you correlate it with app data, let's say looking down, that you can see some very interesting patterns and trends. So we think this is one of those areas where there'll be dragons here, because it's kind of complicated and there are some strange things that happen. You see, for example, sample points of data. The data coming off sensors is noisy. Missed things, the timing isn't quite right. And so there'll be dragons there, but there'll also be gold. And so once we establish the base model of being able to gather data from lots of sources, then that gives us some options in terms of use cases. We can go do much better capacity planning. We can look at ways to rebalance the workloads within a data center. Perhaps rebalancing is for power. Perhaps rebalancing is for performance or quality of service. Perhaps it's to rebalance and take equipment out of service. There are a number of reasons there. Of course, we love to talk about initial workload placement, and that's important. We have to do a much better job of initial placement, but that's just the first step. You gotta live it through its lifecycle. If you're doing short-lived jobs, maybe as Google does in some cases, that's great. Initial placement is everything, but most of the VMs that we run are gonna be long-running VMs or containers, and so you have to manage them through their lifecycle. And of course, every IT is gonna wanna see the data be used, and one of the examples that can be used is for billing and showback. Whether you're actually charging the customer farter or simply showing them how it gets used. Now, the other reality that I think we have to understand is that there's not exactly one cloud here. It's gonna be multiple sets of them, and we have to be able to aggregate the data, aggregate the control across multiple facilities, multiple installations. I think as sooner we get to that, and I live in a world within Intel where we love cattle, but pets are what the world's got, and that's what I gotta go run in my environment. So if I just stepped on the alligator in the room, then there it is, right? I think we have to, we believe we have to go support both, and the ability to look across multiple environments and manage services across a set of environments is a necessary part of the puzzle. So Intel IT went through this transition, and they had to learn how to use a flexible environment they had to retrain people, they had to learn to take storage policies and embed them in the environment, but the end result being that today from a business unit initiating a request, they can have that activity in production within a day, and this is the official number, but I know from my own business unit working with Intel IT, they often go within an hour, it's often quite quick. They have really adopted the business unit in charge, they just go driving to make it happen, and they've driven to better than four nines worth of availability. Some of that through self-remediation, some of it through coaching us how to design our app so that we get the availability we need, but it's been a very powerful transformation within Intel IT. They run about 70,000 servers within an engineering compute environment, and they run something pushing 10,000 servers in the rest of the environment based upon these, not all of it based upon SDI, but a good chunk of it. So, good story there. Now, from my side, looking at SDI and why should an end user and end customer care, you know, this is all great technology. Boy, it's a lot of fun to talk about it, but we already know from our homework that there's a huge cost savings to be had by the end user customer by adopting an SDI technique. The left column is a, let's call it a modern IT, virtualized, but still mostly focused on pets and fairly heavily crafted VMs and network and storage environments. The middle column is, let's say, largely modeled on Intel IT, so they're pretty sophisticated. They've got a good size team working on OpenStack. They understand internals of SEF. They know how to optimize storage, but they still left a lot of, they still were forced to use a lot of legacy models for the way they operate just simply because they couldn't transition everything to a new model immediately. And then the right column is if you were to say, you know, I'm gonna remove old, you know, suspend disbelief, I'm just gonna simply land where I wanna be, that's where the right column would be. And we know that that is anywhere from a 46 to 66% cost savings from the left column. And in cases where I've gone through this within customers, they often say, wow, I'm gonna go do that. And you say, wait, this is painful. I mean, you gotta change the way your IT operates. You gotta look at things completely differently and they're like, yeah, yeah, yeah. Show me how to get there. So I think this conference is an example of how we hit one of the biggest areas there around the people costs by implementing orchestration and bringing standard policies into the mix. So Moore's Law gives me the advantage of being able to drive more capability, but also there is a transformation in the way we do compute network storage and security that will drive these changes. So with that, I'm gonna transition to Shaw and he's gonna walk you through their story. All right, thank you, Bill. Can you hear me? Okay, great. All right, thanks, Bill. Great talk. So I'm going to, first of all, introduce myself a little bit. My name is Shuoyang. I work for Huawei. I've been working with Huawei for four years. Before working for Huawei, I've been working with Google for another four years. In this talk, I'm going to talk about a project we built called Compass. Compass is basically trying to resolve the issue described by Bill that in a software-defined infrastructure, how can you make sure your infrastructure a jail? How can you provide agility to your software-defined infrastructure administrator? So we started this project all from reading a book. This book named The Data Center as a Computer is written by several Googlers. They basically advocate that right now in the wire-scale computing facility, the new data center is becoming the new computer. So we started reflecting from that book a lot. So let's take a look what happened in the 90s, right? In the 90s, you get a box and within this box, you have a NIC, you have a CPU, you have disk. Essentially, these are the networking compute and storage resources in this box. And with Linux, it's essentially assemble these resources together, expose the user so that you can consume these resources as a computer. However, without, you know, Linux wasn't so popular adopted. This box is a Linux box right now. And everybody I think right now is so widely adopted. The reason being, right now we have this provision, the deployment tool, something called Live CD. You insert the CD or USB, then essentially automatically install the Linux box. So another thing people have been building is things like a system monitor. It essentially tells a user that how is my computer doing, right? Is it functioning well? So with that in mind, let's take a look at, you know, what's in the world for OpenStack. So fortunately, right now we have OpenStack. As a community, we are building something huge together. And in the data center, we can see the switch, the server, the storage server. They are essentially the new joint computers, networking resources, compute resources, and the storage resources. And the OpenStack individual, you know, project is managing them together as a whole resource provision to the user. So, but probably people also is aware of that compass, if you want to, you know, deploy a OpenStack cluster, you want to build this a joint computer, it's not an easy task. So there's lots of tool around trying to solve this problem. And the compass deploy is one of that. We want to basically say, hey, can we build something in the long run over time? It will become something like live CD for a giant data center computer. And after solving that problem, can we give the user of this giant data center computer some assurance that, oh, my computer is running well? That's something we're built for, you know, this called the compass view. And if we have that, and if we have that in a, you know, restful API way, we can basically provide the user a lot of flexibility, a lot of agility to change your software defined infrastructure. And basically compass model, our, you know, modern data center as software defined infrastructure. I'll show you why. So this is like a very typical data center equipment, three racks. Let's open a box. In this box, what you want to do is you want to deploy a lot of, you know, compute related demons, right? You want them to, you know, serve your compute needs. Secondly, you also want to deploy some, you know, storage related demons. So you want to them serve you as a storage resources. And lastly, you want to deploy the networking related demons, right? These are the ingredient of your software defined infrastructure. If you build them, you can, you know, make your final product. I like the, you know, keynote speaker yesterday, Jonathan's metaphor. He's saying in this software defined word, you know, everything is looks, start looking like a Lego game, right? You have these pieces. If you assemble them into the right, you know, connection connectivity, you build out your final product. Your final product is the shape of your Lego, right? So not only that, you with the advancement of software defined networking, right? The networking fabric builders is trying to basically deploy their SDN software stack onto this network fabric. And they basically have this totally software defined word. So this is the, we call the compassing action in this software defined infrastructure. Think of this. You have a bunch of a bare metal devices, the networking gears, the, you know, servers, right? The first thing you want to provision your SDN stack onto those networking gears and make sure if they are like, you know, SDN, the open flow based protocol, make sure having them talk to the right controller, right? If you have this, you know, learning switch based thing, then the second step is not necessary. But anyhow, you need to have this provisioning process. And this is the same mechanism. You can, once your networking fabric is built, you want to do this server software deployment, right? And this is a lot of, you know, product in the market right now is doing, right? But we believe this compass, we can use the same model to do, you know, make a universal thing so to serve this purpose. And this compass is basically a programmable framework. It's provided a restful API. The reason we are happy about this, our model, is that we believe we have abstract the, you know, common, you know, concept of your deployment process into this restful resources. For example, this is a documentation from our website, syscomp.org. Basically it describes your, you know, basic restful resources from your deployment process. Also keep the detail, but you can find it from our website. But once you have this basic restful API abstraction, you essentially do a way with a lot of boilerplate code, right? Imagine, without, you know, this kind of tool, what do you want to do when you, you know, deploy a software-defined infrastructure? You want to write a lot of script. Right now, hopefully, you know, tools like compass help you do a way with that. You focus what you really need to answer when you build a software-defined infrastructure. Forget about this, you know, automation. Even without automation, these are the questions you need to answer regardless. Even you manually install something, you need to answer these questions. Right now, we are giving you an API, you answer them step by step. And once you answer that, you know, your software-defined infrastructure is up. So this is a very quick architectural diagram. Essentially, as I said, we build this restful API layer. We allow the savvy user to program directly against this API. We also build a, you know, nice UI. You can try it out to have this, you know, wizard-based UI installation process. These are the, you know, heavy lifting parts of the actual work. Remember I said, you know, you want to do a way with the script. Your user want to, you know, repeat every time. They want to do something. This is something we help you to abstract away. And we do not want to reinvent the wheel. We want to utilize the best wheel in the world out there. So we use Chef as our package deployment tool. We use Cobbler as our, you know, OS provisioning tool. We have a lot of, you know, software management functionality. Those are the kind of heavy lifting stuff. But actually, the plugin itself is a pretty thing. It's like 200 or 300 lines of code. We do not want to lock you into Chef. Because sometimes you may choose to use Ansible. Sometimes you may choose to use, you know, Puppet or Juju, whatever. So we want to say, hey, all this thing, if you want to do that, you know, write a plugin, you can do it. You can consume your manifest. You can consume your, you know, playbook. So that's the basic, you know, extensible way we model that. And as I said, we really want to make this product extensible. Extensible in two ways. In this slide, I show you, we want to make it extensible from the end user's perspective. First of all, this is the very first version of our release. We were able to automatically deploy OpenStack. And this very, you know, after the basic release is stable, we add a little more effort. We were able to do Chef deployment. In this, you know, Intel, Huawei, and the canonical joint POC project, we deployed a Chef plus OpenStack cluster in Intel's Oregon SDI pod. And also, we want to allow the user to use different, you know, host OS. We allow user to use different, you know, hardware. If you, for example, if you are an OCP vendor, right, write some plug-in, it should be simple. So we do not want to lock our people, you know, our user into a particular, you know, package management system. You like Ansible, you do it. So you like Puppet, you do it. You know, there are lots, this is not a complete list. You like Jujo, you do it. So this is, you know, we are using Cobbler. But if you like Razer, let's, you know, work together. We can write plug-in for that. So this is a quick overview of Compass Timeline. We started, this is when we started reading the book, right? And then it takes quite a while for us to even, you know, be able to automate the deployment of OpenStack. But fortunately, the next round of target system we build, the time is much, you know, much shorter, much, much shorter. So if you have some idea, you want to use this tool to build something, you know, we can talk. By the way, there is an interactive design session in the afternoon. If you are interested, come over to join us. And I really like build and that's this, you know, this slide, this very visionary. And the reason I enjoy this talk is because it's somewhat, you know, is very similar to what we believe in the data center, you know, in the long run. We build something called Compass View. We solve this problem and we are working with university professors trying to build even, you know, the feedback loop. We won't think of this. Standing up infrastructure is not the beginning. It is not the final step. It's just the beginning of your journey, right? How can you make this cycle, you know, adjustable? That's the key thing. Otherwise, your infrastructure become orphaned, right? So this is a Compass View. Basically, we are saying, hey, let's forget about each individual component. We want to build things pluggable, right? But we say, hey, what should happen, right? If you want to build, if something, a new plugin come in, you want to, you know, swap in that easily, what that should look like. So basically have several layers. We have agents. We have, you know, proxy or distiller layer. We have this, you know, search functionality, you know, temporal data functionality. By the way, I'd like to share with you about one of our, you know, 200 nodes deployment. With just 50% of actual space, you can get, you know, the full text search functionality of your log. By doing that, you provide your user a lot of root cause analysis capability. So that's something I like to share. But with that, I want to show a very quick demo to after one minute, after which I will, after which I will give the stage to compel. Yeah, one minute. I'll skip all this installation part. I'll show the new functionality of our monitoring. So that's about my talk. And the next step is I will give the stage to compel, showing the last mile. Can I get a mic? Oh, but it works. Excellent. So my name is Kapil Bungavelu. I'm a technical architect at Canonical. I focus on public private cloud deployments and workload orchestration. So we've gone through the software defined infrastructure journey, and now we have a cloud. But what do we do with it? The business value of infrastructure is in the workloads and solutions that it's in service to. And now that we've got a cloud, we want to start deploying those workloads. Well, what do we have to do? Well, we have to learn the operations around given workloads. We have to learn how to get scale them out, how to get them portable across our public clouds and our private clouds and onto our bare metal. And all that takes time and agility. What if instead we could deploy any workload solution with just a single command? This is actually deploying a SQL data warehouse analytics solution with a single command line. Or if we could get the latest and greatest of, that's not supposed to be there. There's a Docker underneath there, Kubernetes underneath there. Or we can do it inside of a GUI tool and get the latest and greatest technologies. This is actually a Kubernetes deployment with deployed directly from a GUI user interface tooling. Or get the enterprise solution for deploying a pass for simplifying operations and increasing developer productivity. And it's also fairly complicated as far as the internals. So we can, we can do all that stuff today. And Juju is a cloud native orchestration tool that gives the, that is built around the notion of managing services instead of managing machines of adding capacity and managing the throughput of services. It's written such that you can write a service definition in any language, taking advantage of your existing tooling, taking advantage of your existing expertise around a given set of tools. You can use everything from puppet to Docker images inside of a service definition. It incorporates provisioning so that when we're going to scale out a service we don't add a bunch of machines and then map that to our configuration management. We just say add more capacity to that service. It's event-based, well, it's state observation-based and such that it's always reacting to what's happening in the environment. If services are, if particular machines are going down, then the system becomes aware of that and is able to reconfigure, say, my load balancer to drop those nodes. And it's designed for easy composition and reuse. So these service definitions, they model the relationships between software components as a native first-class entity. And as such, they're able to get, derive real reuse from a catalog of components. And as a result, we have a library of hundreds of workloads. This is just a small sampling from big data to NoSQL, to OpenStack, to Pass. There's solutions around all those things that can be readily used. So application components and full solutions. And not only that these components exist, but we're able to reuse entire topologies of these components and distribute those and reuse the topologies so that we're able to not just deliver solutions, sorry, components but in applications, but actually solutions that organizations can reuse internally to find their own and ship around. So the core business value of being able to have a end-to-end workload management tool is being able to simplify and automate the deployment of services, having that library of services, being able to leverage your existing tooling and just getting increasing speed and agility. And the internal architecture of Juju underneath the hood is a set of state servers, managing a database with an API. That API is, it's a web socket, what we have both command, rich set of command line tooling for, a rich GUI for, as well as, and then on the actual machines that they provision from an IAS provider, it will drop an agent on there that will set up workloads either in containers, LXC or KVM containers directly on the bare metal. It's fairly flexible as far as how that all can be mapped out. You can associate provisioning constraints to the services so that as you're allocating new resources that they're automatically going to bounce across zones, they have a particular amount of storage or CPU or memory. And now I wanted to try to put all of this together as far as the infrastructure that was deployed on an Intel SDI pod as, and for networking and demo gods are with me. So this is a open stack that was deployed in the Intel SDI lab on top of Intel hardware. We're using the compass tooling to lay out and lay down an open stack installation. And now we've got Juju on the system that has a deployed workload. This happened just to be a simple one that I could get going and this morning. This is MySQL with some media wiki instances front end by a front end HAProxy. We can see though that we can simply search around for additional workloads that we might want to use. If we want to use that, we could. We could do say an ELK stack, but I'm going to go ahead and take a step out from that and show just for example, that this is the HAProxy unit. So I'm just going to go ahead to it as far as being able to look at it and that will load up. And here's the workload deployed. And really that was quite as simple as looking through this catalog of workloads, finding the workload that I was interested in and trying it out and dropping it on the canvas and trying it out. And from this, let's say I want to go ahead and scale up the media wiki. It's a simple matter of saying, hey, give me 20 of them and commit the change. And off we go. I don't actually have that capacity in this cloud, so it'll probably air out, but fair enough as a demonstration. All right, I think that it covers a demo. So in summary, we've got a hyper evolution of continuing increasing data set sizes, increasing data center sizes, the ability to be agile about that, our infrastructure, compose it on demand is key, and that is the origin of our software defined networking. And this is actually some of DASA's slides, unfortunately it's not here. And let's take part in automated application deployment in data centers. Give deliver that end-to-end experience for users from the point that laying down the infrastructure to laying down the applications to delivering the business value. Any questions? Thank you everybody for attending.