 Good morning. Thank you all for coming. It's great to see you guys and certainly welcome to Austin. We are presenting on architecting application workloads for OpenStack Clouds. So we're going to do a quick round of introductions. My name is Megan Rosetti, and I am with the Cloud Operations team at Walmart. I'm Samir Adhikari, and I'm at the Cloud Engine team at Walmart, too. And I'm Craig Sterrett, and I'm with Intel, and I'm part of the open-source technology center. So what we want to talk about a little bit today is a quick overview and introduction, and then two workloads that we have been working on continue to kind of go through the e-commerce workload and the web services workload. Then we want to go through a summary and make sure to leave enough time for Q&A at the end. All right, I'm going to jump right into the introduction. So the workload reference architectures are a series of documents that have been created by the Enterprise Working Group. We have received a lot of feedback through the community that the reference architectures was something enterprises have been looking for. So we started with the web services and the e-commerce applications. Something that you will see in the diagrams, we strive to make sure that these are vendor-neutral, agnostic, very focused on open stack and components to support the workloads that we have shown. And then where applicable we have to heat templates. These are the diagrams are things that we are building out. It's a living document. It's currently in process. We're certainly revamping how we're looking at it what we're moving on to the next couple of stages. So heat templates will go through briefly. As I mentioned, these are living documents. We're very open to feedback, kind of your thoughts, your ideas on what we've presented today and how to make it better. And we'll also talk a little bit about our next steps as well. And priority. We've started with a couple of workload diagrams. We'll get into what we're looking at next step. And then we'd really like your feedback on where things are, kind of where we're headed, and as a community what you would like to see as well. And then we have a link included for where the current sample configurations are. And I'm gonna turn it over to Samir. Thank you Megan. So before I actually start talking about the reference architecture, one disclaimer. Like any good movie, this is inspired by a true story, Walmart's e-commerce. But like any good story, there is some creative licensing here. So do not assume that this is exactly Walmart's e-commerce architecture. So the all of you probably have shopped online in this day and age and but to just reiterate, fundamentally when you go to a website for any retailer, you look for products, you find something you like at the price you like and then you buy it and then you obsessively check how soon it is getting to you. So the architecture that I'm going to talk about fundamentally addresses those things and it probably has ideas for other architectures too. Do not constrain yourself to just think that it is for e-commerce. So the goal of this presentation is to highlight the different OpenStack components that can be put in place for such an architecture. And we use some of these components at Walmart. We don't use some of these components at Walmart, even though you'll see them at the picture. But the way we have tried to organize it is to give a logical view that if you are assembling these pieces together, how do they interact with one another and how do they sit with one another? So without further ado, I will actually go to the big picture. I will leave the slides for your consumption later and my apologies up front for people in the back because of trying to fit so many things on the slide. You'll probably not make out the font very well out there. But I'm going to start from the left and work my way to the right. So when a customer sends in a request, there is a global load balancer which routes those requests. The request could technically be routed to multiple data centers. This picture, you can think of it as being partially replicated across multiple data centers. Some of these components might actually just sit on one. But almost all these components have to exist in some form or the other. So once the request gets routed to a web server, we have a whole suite of web servers which are just essentially generating the web page that you actually see. And by web page, I also mean the mobile app out here or the desktop web page that you see. But in today's world, pretty much, it's rare to have a purely static web page which comes to you. It is generated off of a template. So there's a whole suite of servers doing that part. But behind the scenes, the next layer, which we call, which I've titled Services, is fundamentally a huge bunch of microservices. So each of them effectively does one function. So when you're browsing for a product, you know, the browser services, which is essentially responding for all the data. So all these services have to talk to each other to enable the actual workflow. And a large chunk of that communication is through a messaging layer, which you can use different potential solutions and OpenStack provides you one, so you could use that. But the key to take away from this layer is that each microservice is isolated for the very reason that they can be developed independently as long as they're compatible, backward-compatible with the previous features that they provide. And it helps accelerate the changes you can do instead of having one monolithic monolithic layer at that point. So behind all this, apart from them talking through messaging, you actually need the data to serve all the requests. When you're viewing a product, you need to pull in its price information. You need to pull in its quantity that you have available. That's how you show that of limited items left. You potentially have to show other relevant data about customer reviews on that item. So all that, a huge chunk of that comes from databases. You could use Trove to run those databases. And at the lowest layer, almost all of the things I've talked to so far are running as VMs, which NOVA provides. But databases individually to store the data, you need block storage for those. And in e-commerce or potentially in other cases, one of the biggest data needs is for images. All the images that you see on different web pages, these individual microservices are going to just contain references to the images. But when the page is being rendered, all those images have to be loaded into the web page. So there is a huge object store supporting all that. And you can use Swift for that. So this is the workflow, transactional workflow that I talked about. And then there is an offline workflow, because how do you show recommendations to the user? How do you show related products? How do you show what matches? So all that analytics has to go on in the background. And you have to actually run another big data store and a lot of processing on that. And so for that, Sahara provides you some options. And what I have left out in the interest of time and in the interest of keeping it high level is that you don't have to have all these components together. You can plug and play. As I said, for the disclaimer up front, for example, Walmart does not use every single component in here based on your needs. But this will give you a frame of reference to figure out what of these pieces and even for a non-e-commerce architecture that you can use. The legend below is fundamentally just talking about the different components that we are using, NOVA or the different security groups which are being applied. And for a prevalent in all this, Glance is there because that's how the VMs get their images. Keystone is there because without Keystone or Neutron, even Neutron for that fact, you cannot have a running OpenStack environment. So how this architecture maps to those is a discussion in itself which we leave out for now. And what issues come about, we'd be happy to answer questions related to that. At this point, I will hand it over to Craig for talking about web services. Okay. So I'm covering the web services architecture. And nowadays, there's almost an unlimited number of choices for your web services stack. And so for this example, we decided to go with the LAMP stack which has continuously come up as the number one stack on OpenStack according to the OpenStack user survey. And so in general, just a quick overview of the, you know, we're looking at standard three-tier architecture. You have a web presentation tier. So you have a cluster of web services to service the incoming requests. You have application tier which is doing the main processing and a database tier which in this case is a MySQL database in a master slave situation. Then you have some kind of firewall or security rules that are doing some filtering at each of the network layers. And then a load balancer both for the incoming requests and for the application servers. And then you want some kind of on-demand scaling and persistent storage for the database. And so this is how this all sits on top of your OpenStack cluster. So your OpenStack cloud has to serve certain background processes. And that's your keystone, your glance, your horizon, your heat, your zealometer in this case. And then your three-tier architecture is mostly using your Neutron Nova and Cinder. And so we wanted to create a architecture that was horizontally scaling for both the web and the application tiers. And so your user requests come in and the first thing they do is they go into the load balancing as a service feature of Neutron. And for this example, we're actually starting with Elbas version 1. Although it was deprecated in the Liberty release, Elbas version 2 wasn't supported in heat in Telmetaka. So most of our testing has all been with Elbas version 1. And so user requests come in and Elbas hands it off to one of the active web server nodes. And then the web server hands it off through another Elbas again to one of the active application nodes. And then the database is MySQL database with a master and slave setup. So we also created heat templates for all of this. And so the heat is we have autoscaling groups set up for the web tier and the application tier. And Celiometer on the OpenStack cloud is monitoring the overall activity on each of the nodes. And looking for, in our case, CPU utilization, although the Celiometer meters support disk IO, they support memory utilization, they support network IO so you can trigger on all sorts of things. And when those Celiometer meters or alarms trigger, that then triggers heat to autoscale and create an additional instance to meet the user demand. For example, we also created downscaling groups. So if your activity drops and you have more resources than necessary, you can use heat to downscale your nodes, heat and Celiometer. And that will reduce your overall expenses and reduce resource consumption. So for these, like I said, we've created heat templates. And so for the example, we're writing them in hot format. Heat does also support CFN format, but in our case, we're writing them in hot format. And the heat template basically sets up three private networks. It'll set up the LBAS and your security groups to do port-based filtering for each of the private networks. And then create your autoscaling groups and set up all your Celiometer alarms. And then heat will automatically spawn and configure your web apps and database server. And so for this example, we decided to use kind of on the fly build. You can either do store a single server image in glance and then retrieve that and do an on-the-fly build where you install your individual applications and processes. Or if you want, you can build customized images for your application, your web server and your database server and store each of those separately. And they both have some advantages and disadvantages. Pre-configured images spin up faster, but you need to make sure that you take care of any licensing issues as the instance boots up. So in this example, it's just a sample of the heat template. You can see that in user data, we're passing in the cloud in it all the commands in order to do the updates and installation of the Apache web server. And then this is just a quick sample of autoscaling. And so it'll set up a web scaling group and it'll set up web scale up policies and CPU alarms. And so Celiometer will automatically alarm if CPU in this case exceeds 50% and then trigger an additional instance to spawn off. And when we actually release the heat templates, they'll be fully documented to the point where anybody even without heat experience will be able to easily fall along and understand what's going on. Thank you. Certainly understand probably one of the biggest questions will be where can we find this documentation? We are in the process actually of building out where these will land. Right now we're looking at adding them off of the sample configuration page. But that is something that we're asking for your input and your feedback on how would you like to see and be able to process this information. And also along the lines of does this information what we've presented today meet needs for adoption, questions about I need some type of architecture to get started and these diagrams fulfill those needs. Mainly are we headed in the right direction. So how you can help because we definitely are looking for your help. Encourage any and all feedback. In addition to these two we have others that we're working on. The next one that is close to completion is big data. And then we do have others in the pipeline as well. That is a question I'd certainly like to hear back from the audience on. What other reference architectures would you like to see in addition to the ones we've put together and the ones that we're working on. What are areas of interest? We need to certainly prioritize out which ones people feel would be most useful, most relevant to the work at hand. And naturally feedback can all go through the enterprise working group. The enterprise working group was founded to help with enterprise adoption. And when we first started we identified different barriers to adoption. So through that we've been working across many different companies and many different layers and partnering with several other working groups to move enterprise adoption forward. We meet weekly. We have a wiki page and then also a mailing list as well. So please feel free to join our meetings, send us an email. However you would like to reach out and contact us. What we would like to do is turn it over for Q&A and also your feedback. Really want a good understanding of how does this meet your needs? Is it what you're looking for? And what are some other areas that you would be interested in? So I do know we have a mic stand right here for anybody who has questions. I have a question for you, Samir. You chose Object Store for your images. Can you elaborate why Object Store for images versus file storage and what did you use for your analytics? The second? Object Store for images and then? Analytics. So Object Store for images, fundamentally when the product images don't change much. Once in a while they might get updated but they're never edited on the fly certainly. And because if you think of Amazon which potentially has a few tens of millions of skews of products, sorry, Walmart, and if you think of Amazon which has about 200 million skews of products, if you look at those environments the image for a product pretty much will stay consistent for generations of the product. What you need are good tools to be able to change them when the product changes. So Object Store fits that use case very well. You design the system to pick the object out but what you don't know is the access pattern for that object. You don't know that the object has to be loaded on the page but broadly but you don't know that which objects at any given point of time customers are interested in viewing. So in a file kind of a scenario you can design out scale out solutions with Object Store much better than you can design with file solutions. So not changing them much being able to scale out very quickly with very low latency load the page, essentially what drives the need for the Object Store there. And for the analytics cluster so there is a certain amount you need live to maybe potentially detect. What you need live is essentially to get an idea of the consumer's interest or intent to be more precise. What are they looking for and that data is only built up over time because most of the time when these commercial sites show you related products that is based on essentially some form of a summarization over aggregate statistics collected over time of lots of consumers. And similarly you want the critical need there is that you don't necessarily need the 100% accuracy on what product you're going to show the user. What you need is that quickly show him something or her something which they can pick out. So a lot of that processing because it involves a lot of different sources you might be ingesting social media data, you might be your traffic patterns of your website, you know other data sources that you can find computer prices, so many things. So that processing take first of all that data ingestion takes time and then the processing take time which essentially at the end of the day what you have to come up with is a matrix which you can quickly look up based on a user from a certain location, maybe a certain time of day or the product they are currently viewing. You have to pick a row in a table somewhere which will relate other potential products that the user can buy. So all that number crunching essentially takes time. It's very hard to do it live. Live stuff you can only do incremental. You can never do the whole data store analysis which is necessary for showing those correlations. So but architectually the way you know the implementation of the analytics cluster. So that is why it's very intense. You tend to use there's a lot of IO demand for moving the data around in that. So there the solution tends to usually be bare metal. And even if you run a NOVA VM on top physically to do the processing part, to handle the IO needs you effectively expose the bare metal blocks to the NOVA machines. Does that answer your question? Any other question? Could you repeat that? Sorry. Oh, what do we know about the next reference architecture on big data that is coming? I haven't looked at it yet. Do you know who's working on it by any chance at the top of your head? Sorry, go ahead. The big data is still a work in progress. What we are looking at is not only the reference architecture but then building out some documentation. And they're right in the midst of going through that I don't have an exact date of when that will be released. But the big data is the next one to be released. Do you think that we're on track with what we're looking for because we'll right now we're having big data follow that model? Is that something that would be of interest? Do you think that we're headed in the right direction at least with that? But on the two that we have here, so with the two with the e-commerce and the web tier, do you feel that those are on track with what you would be interested in or what you would be looking for? Okay. Okay. And that's great feedback. Thank you very much. Yes. One question. Are there any plans to implement the reference architecture for CI CD pipelines? For example, like one of the largest open stack cloud consumers is open stack itself. And I would be happy to see the reference implementation of open stack infra like Garrett, Zool, all the built pipelines of open stack as in reference architecture for enterprises built in the SACD. Perfect question. I was thinking if there were no more, I would raise that as a comment. So what I didn't talk about right here is how you deploy all these pieces, right? It's a whole complex system and one of the challenges even if you have open stack running is deploying all these other things. So for that, I think the good news or the bad news depending on how you want to take it. Walmart uses its own application lifecycle manager, which we have open sourced and you should check it out. It's called one ops. And that what, you know, what it does really is hides the fact that open stack is what the developers are writing against. So we do to deploy open stack itself. We use open stack Ansible and we don't use the open stack infrastructure pipeline to deploy open stack because we tend to deploy a stable review. We use open stack services. And if anybody in Walmart wants to contribute, it is done outside of what is deployed in our environment. But for deploying all these big pieces, the interface that one ops provides, so there could be different solutions. I'm talking about ours is that the developer goes to the application lifecycle manager, logs in based on their internal credentials, Walmart credentials. And then they tell the one ops tool that, hey, I am going to deploy a three tier app because each of these components internally can be multiple tiers too. This is a very, very high level view of the reference architecture. And they say, okay, I need a patchy Tomcat component. I need a Java component. I need a certain database component. And then the orchestration tool actually kicks off VMs does the install. So we still follow the model of doing installs on the VMs. So we don't do, it's not prebuilt and baked into images. So the VMs come up, the tool does the installs. And when you're in the tool also allows you to pick choices like I want a load balancer. I want block storage. I want a failover capabilities. All those things are interfaces in the tool. And once you pick and choose, then the orchestration tool talks to OpenStack, brings up the VMs and does all that. Sounds really similar to what Murano does in OpenStack. Pardon? That's what you're describing sounds really similar to what Murano does in OpenStack. It does. And so any other questions? First of all, does that address all you asked for? Okay. Any other questions? So I have talked about the reference architecture which maps to walmart.com. First to answer part of your question about, he asked Voodoo to repeat it for everybody else. So Voodoo does video and which involves depending on the device. You know, you have to serve out the same content in different manners. So there's associated metadata and transformation of the videos when people do fast forward or rewind. All those actions require some background processing. So I personally don't have insight into how Voodoo does it because this is the area I focus on. But there are, invariably, they would involve a lot of processing and a lot of storage, even more than the images potentially depending on the catalog of movies. So yet a non-answer to your question is I don't know. Sorry to answer your question. So I'm part of the working group as well. So we do actually look at the media transcoding and distributions workload in the pipeline. So just follow up with the summer configurations. We on in the next couple of months, we will be working on the big data starting from a simple configurations. And we probably also look at the media transcoding and see how we can use media, the OpenStack components to support the media transcoding and distribution workload. So I guess that gives you the motivation to go look at the reference architectures. Please. I have sort of, I guess would be a non-standard question for you. I'm curious, given that we have an enterprise working group and we're doing reference architectures, reference architectures and enterprise are used for lots of different reasons. For architects, they're used as a way to kind of jump start something and move it forward. But they're also used in the executive level and the management level to kind of paint a picture of what you can do with the solution. And they also are used in economic analysis to try to figure out what is the capital cost, what is the scaling cost or something. Which of those things are you focusing on as a group? I think this one, I will let either Leong or Megan answer. Initially what we're looking at is more of a jump start. Because we've had some sample configurations but we've been asked for much more detail. And then the next step is to look at particular use cases. So it's a jumping start. It's a jump off kind of a start for these. Where do we need to go next? And then do we look at building out different use cases than around the reference architecture? Because you're absolutely right. You bring up three really strong scenarios that people can use those for or they can use them in combination with other ideas or models as well. So it's a starting point but I agree we do need further development and then your feedback is exactly what we need to make sure that we're online with what's really going to be the best use for putting a lot of work into the reference architectures and making sure that they're easily consumable for businesses for enterprise needs. Which can vary greatly depending on what they're looking for, what they're building out, scale and so on and so forth. But more of a starting point right now. And then hopefully in six months to a year we'll have a much different answer. Thank you. Any other questions? Okay great. Don't think I see any other hands up to you guys. I just want to make sure. Thank you guys very much. I appreciate your taking your time to come and listen to us this morning. Thank you.