 Hello, everyone. My name is Michael Kedera and welcome to Orchestrating and OpenStack based IoT Smart Home. Now, I am, I consider myself lucky, I really get to help people deploy cloud and data center applications. And as part of that, what I do is I work in Intel's open source technology center. And when I work with customers, I leverage a lot of what I've done in the past in working with Intel's IT department. And so, for example, I used to deploy a lot of their cloud environments as well as platform as a service and applications as a service. And I rely on that history as well as working with great people like Leong and John who are actually working within the community as well as on our platforms to make OpenStack and cloud services a lot easier to use. And that's a lot about what we'll be talking about today. My name is John Geyer. I've been working in data centers and labs and different spaces like that for 20 some odd years. Most of those have been at Intel. So my history is basically in the trenches. I didn't do a whole lot of customer interaction, although here and there, a few years here, a few years there, I did do some. But in that time, I've had a lot of experience with Intel hardware from the ground up and Alpha and Beta stages and everything from mobiles all the way through servers. So as I'm in this role, and I actually report to Mike, moving more into the community and looking forward to doing a lot more of this. Hi, I'm Leong. I'm a senior software cloud architect for Intel as well. So we are in the same open source technology center. I actually joined Intel last year, so I'm pretty young to Intel compared to them more years than me in Intel. But before that, I have been working in the industry for more than 15 years. I have got a chance to work on different kind of applications, including peer-to-peer applications, peer-to-peer applications, media streaming applications, and also in the academic research as well. So I completed my PhD in 2013. My PhD is about multi-cloud orchestrations. And I'm so actively involved in the Enterprise Working Group. So in the past two years, we delivered two e-books that talk about what is OpenStack and how to implement OpenStack in an enterprise context. And in the upcoming Friday, we are going to do a third e-book. And I see my fellow contributors here as well. And the third e-book basically talks about how to build applications that run on top of OpenStack. Moving forward. So today, what we're going to do is we want to start by demonstrating an IoT use case that we created that's actually an IoT smart home. And in this demonstration, we'll show a number of different devices, all connecting and working together, not only locally and managing that with the local analytics, as well as the RESTful interface. But how we're going to be able to communicate with that and manage those solutions in the cloud. Now, as we go through this, what we'll do is I'll walk through the demo environment and how we set it up and why, as well as John and Leon will finish things off by looking at specifically what OpenStack can do for your cloud environment and how it can actually enable IoT applications. And then they'll wrap things up with some data analytics. Now, our customers are really going to demand a lot of our cloud when you really start thinking about what's coming with the number of IoT devices. $50 billion is what I've seen projected by 2020 and how this data will really impact our entire environments. Now, in 2008, there was an event that went not necessarily unrecognized, but I think unnoticed by a number of people. And really, what happened at that point is that was the point that we had more things, more phones, more connected sensors and devices on the network than we had people using them. That was a report that was given by Cisco back in 2008, I believe it was called Internet of Things Simply Enough. Now, start looking at where we're going. Now, you look at the example here we have with people. And now, we're all techies. I don't think a gig and a half of internet traffic a day necessarily sounds so big for us. But I've got to tell you, if I start applying that thought to what my mom might be doing on the internet, that's quite a lot. You start thinking about how much data is coming from people, on the average, that is a lot. Now, that's up from 650 meg of last year, so we're probably just a little bit over that. Now, definitely a decent amount of information, but now let's start looking towards machines and what we're doing there. Smart hospitals, they'll definitely be generating a lot of information with 3,000 gig a day. But self-driving cars, you think about it's basically a data center on wheels with some of the stuff that they're doing. You have to have really detailed analytics to know that there are hazards in front of you or all the cameras and processing that has to happen. Quite a lot that has to happen within that and also processing at the edge versus in the cloud. Connected planes, of course, numerous sensors. I'm not a pilot, so I can't go into all the details there, but a lot definitely going on there. And then factories. This is just amazing, 1 million gigabits a day. Now really, you start looking at all of that and all the machine communication. And part of the projection here is that machine to machine traffic is projected to be about 40% of the internet traffic by 2020. So truly, I guess that's the start of the rise of the machines. Now, we do have a demo here. I think I'll actually go through. I'll unveil our surprise for you. This is a example environment we set up. Yeah, so you have to imagine this is where we actually keep our most promising developers. It's a fine-meat-in house. Yes, that's right. We have a number of different sensors all included in here. Everything from motion sensors that's over here, I think it's actually detecting motion in the front, but it triggers our LED lights from red to blue. We also have temperature sensors, a number of things in here, and are actually controlling this using what's called Iotivity. And Iotivity is basically what it does is it provides device discovery of a mix of different environments. So if you can imagine all of these temperature sensors, we have a CO2 sensor, are all manufactured by a range of device manufacturers. Now, what I'm able to do is through Iotivity and through open standards, we're actually able to bring these all together. So if you can imagine some of the challenges that many of our developers have with Iot devices and solutions, you have a great smart TV, then I've got a wonderful smart thermostat. Well, I can't control my thermostat through my TV. I'd like to have that all working together, but now I have to work and make some bridging information. But I have a simple application that we created on our Android device. It allows me to turn on. I've got a buzzer here that you can actually hear going off. And that's basically also it's connected to a CO2 alarm that we'll also turn off. And that actually triggers some simple analytics for the fan venting the environment. Now, of course, this isn't to scale, but all of this is a set design to help give people an idea. And the reason why we built this was really to start to show how data starts to flow from these individual devices, how you can control them on the edge, and also how you're going to manage data. Not only here and locally, but in the cloud. Now, let me show you a little bit more about how the architecture looks. So this is the overall architecture. I mentioned Iotivity is really the center point of how we're communicating. Now, within the environment, we have Ostrow. Ostrow is a Yachto-based Linux distro that was created, and that's what's running on our gateway. That's a Minnowboard Max. Maybe actually seeing our solar panels tilting based on the time of day, and it just moved. So we also have Zephyr, which is a real-time operating system. So that's what's powering our smart meter. And then on the cloud, this is where we have OpenStack, and that is orchestrating our cloud foundry environment and also big data analytics. Now, what I'd like to show you is switching to our cloud. Now, a bit of a warning here because of connectivity issues we had with just being able to get our gateway to connect to an external IP and communicating with our cloud. What we're doing is we're sending example data to our cloud, so this gateway is not directly connected. So we set up a container that is simulating the data here that you're able to see. So before you laugh at some of the things in there, you'll see maybe a little bit of an odd temperature in that house, but it's all here just to show you the examples and how it all comes together. Now, I mentioned that we are using cloud foundry. And as an example here, what we did is we created this show pretty well, but you can see it's a little blurry up there. But we have right now five instances deployed all behind that URL that you see here. And that's very easy for us to rapidly scale this. We just tell it the number of instances we want to deploy. It goes out in provisions of those solutions and load balances all of that behind our environment. So that is one of the things by developing this application to use cloud foundry, we're able to take advantage of quick deployment, easily scaling that solution. Now, we've also created this so it can work in containers as well. So we can be pretty versatile in allowing others if they don't have a cloud foundry setup to also test it and play around with it in a container setup. Now, part of the data that we're bringing together, you can see some simple warnings that we have based on motion and a CO2 alert. So we're able to track that. But we're also bringing in additional information with, I can scroll down, unfortunately, I think I'll have to just, there we go. So you can see a little bit more of the information we're tracking. We're doing some simple analytics here that's based on the weather locally. And if you adjust the thermostat based on power consumption, of course, it is a little warm in that house. So I wonder what the temperature is. It's about 3,000 Fahrenheit. 3,000 Fahrenheit. So I had to convert that for me, thank you. So you can see that we're tracking some of the information there, but now, switching to some of the historical information, this is where things start to get a little bit more interesting. So we have historical information on the past month and the last year. And this is what we really wanted to show is what we're storing here in the cloud. It's that historical information and data retention, so you can do that more detailed analytics. So when I start looking at it from one single house, and now if we start to scale this out to, if we wanted to look at it from, maybe I'm managing a number of homes from a neighborhood or a grid, for example. Now we have a couple of these environments set up, one that's running right now in Asia. But let me bring up ours in Barcelona. And you can see our environment here. And what we have is this is representing the gateway that we have right here and some of the historical information. And now what we've done here is we've provided example data that's based on the analytics that we're collecting that is showing some of the warnings that you have. So you can actually go in and manage other homes and actually start to have projections on data consumption. If you're looking, if you're managing grid, you can actually start to understand how your solar and grid energy can impact your solution. So just a kind of overview of the whole interface and the things that we can get from Cloud Foundry and easily deploying our environment. Now I showed you the application on our Android. Of course, that's a little bit hard to see from there. But we have simple controls locally. But let's start looking at the application profile. And I think you'll see initially there are some things that are very common to Cloud applications. But looking at a few other things on what might be a little bit different with IoT applications. So behavior of steady and predictable, I think we've all seen applications like that would be great if every application operated that way. But not all IoT does. Now we have to look at, of course, the growth and projection of that. IoT devices are so sometimes so simple to deploy that really that high growth is a really big potential within your environment. You really need to plan for that. Now, also one of the characteristics of IoT solutions is that they're not always connected. Sometimes they're on, sometimes they're off. If you have, for example, a shipping application and they're going through the mountains, you need to know what you need to track and re-sync that data or even back it up and bring everything back online. How does that all work within your environment? You need to plan for that and potentially do analytics at different areas to make sure that things stay within plan. And then overall bursting, a periodic bursting, that's great. You can easily plan for it. But random bursting is really difficult to plan for. And so you need to, of course, work within your resources to make sure you can scale. Maybe you're going to implement a hybrid cloud solution to help with that bursting external. Now, I think a lot of people are familiar with microservices, especially from a cloud perspective. But I think looking at it from an IoT and inclusive of IoT devices changes things just a little bit because a lot of times, I think developers aren't thinking it from the full end to end. So application services completely apply to that and how you bring everything together. But collection orchestration. Of course, within the cloud, you're going to have things like Chef, Puppet, Ansible, solutions like that, that heat templates that can help you with collection orchestration. But you also need to consider how you're going to manage your individual services at the gateway level. Our gateway, we actually are doing some simple analytics there. But it's easy. This actually has plenty of horsepower that we can run containers on there and easily switch out services and upgrade them easily. But that would be a big part of continuous integration and development. Growth and scalability is also a big part of it in just making sure you're ready. So if you're scaling up, one of the things, of course, you always have to think about, as always, it starts with a database and making sure everything is ready. Potentially, you may need that N plus 1 environment to make sure that when you scale out that application and that database, that it's ready. The data is ready for other applications or web services and caching services to connect to it. The lifecycle, of course, I think a lot of people are familiar with all the work that the OpenStack community has been doing with upgrades and managing APIs. But you do know of the challenges that goes with that. You need to make sure of that across the board of all your devices. How are you going to perform these upgrades? You don't want to have to go in and rip out a device from a wall or the floor, or potentially have to swap out something in your car because you want to patch that environment. You need to be able to consider that within your plans as well. And we talked a little bit about continuous integration and deployment. You need to, of course, include those devices in your plan. Oh, there's so much going on right now with that internet storm. Has anybody have a web camera that's been hacked? There's been so much going on. It's really kind of a nice, the better half of it. It really applies well to my slide right here. Not only do you have to think about user authentication, but who is actually accessing those devices? What devices are attaching to your environment? So of course, the lesson learned. Make sure you change default passwords, do intrusion detection. Things like that all need to be considered. But of course, network encryption and data encryption. When I left my house, one of the things that's really nice, it's kind of a surprise earlier today. I actually have a few IoT devices. But one of the reasons why I set it up was that my kids kept on leaving my garage door open. So I don't want all those tools that I have in my house walking off. So I get alerts when my garage door has been open. So I'm able to actually close my garage door from the other side of the world. But I don't want to give that key to my neighbors. So making sure that that's good and secure, or providing those updates that I'm the one that announces to who that I'm out of town and that whether or not my house is unlocked or not. So patching, of course, just as simple of other things, you have to look at that from a security perspective. How are you going to patch and keep those all those devices current? Now, I'd like to change really the look now to how we manage data and where you're gathering it. Because really IoT brings more options for you with data gathering, but really looking at it from, we all know what we can do with big data analytics. Really in the cloud, you get that deep compute, deep analytics, a lot of historical data. But you also have benefits within IoT devices, and they're getting strong, and you can do some analytics there. You need to take advantage of that. And speaking of that, this is a great slide that Tom Bradich, he's with HP, he provided a presentation on this earlier in the year and allowed us to use this slide. But it's a good example of really showing that spectrum of insight when you're looking at where data analytics can happen. And as it says on the title, location can be everything. Because if I have a car that's noticing that there's a truck driving in front of it, I don't wanna have to communicate that it just recognized the truck is in there. I better send that off to the cloud for analysis. And meanwhile, my car just continues to barrel into it. You need to have that local analytics and the power and horsepower behind that to make sure it can make those quick response, quick action decisions at that time. But looking at what we can do, start from the cloud, and of course you have the ability to store lots amount of data and bring in a lot more integrated data into that solution and do that combined analytics. Now there are trade offs there because not everybody's gonna have the power that a lot of those autonomous cars have in the environment. You're gonna have to do trade offs and what you can and can't do at the edge, but there are some strong devices. I mean, for example, the thermostat on this is not changing, it's probably around 20 degrees or so. And I don't need to send three updates a second to the cloud, that's just noise. Or if I'm driving my car and I'm collecting tire pressure, I don't want to have to store a data array in my garage just to keep track of all that tire pressure. I just want some updates. But I do wanna actually know that the performance of my car has shifted out of normal. I wanna be able to track that information locally or potentially even have more of a range to potentially warn me of, excuse me, but potentially warn me of maybe a failure in one of the devices that could potentially be happening. So there is that trade off of compute and you'll see that you can actually leverage more and more of the more you shift to the left and start looking at some of the analytics and leverage both ends of the spectrum for the capabilities that they provide. So let's take a look at this from the top down. We've talked a lot about what can be happening and how all this works and let's look by layer how this all stacks together and becomes something that's actually usable. With IOT devices and applications, you can be talking about a lot of different things. I mean, you could have someone in a vineyard somewhere who wants to measure water in the soil, nutrients in the soil, the thickness of a stem so that he can produce a better grape for wine by controlling how much sugar is there. From an environmental perspective, maybe we measure chemicals and run off into rivers and streams so that we can track where maybe we've got some issues and we can maybe save a few fish. Athletes can be tracked as far as stats. You can then maintain a healthier athlete, better team. And then in this case, we're tracking a lot of data and what are we gonna do with this data? How do we get it to the cloud and what do we do with it once we get it there? So as this data is collected, we need to move it somewhere and these applications have to run somewhere but the application designer doesn't wanna have to deal with infrastructure, doesn't wanna have to deal with, well, I mean, how much hard drive space am I gonna need, what do we do here? So we give a layer where things like platformers as a service, big data containers, VMs, all that can live where the application designer and owner only has to work in that space and they don't have to work beyond that space. They have someone else to take care of it because it's all deployed somewhere off their premises in the cloud. So there's a lot of things that can fit in here, cloud foundry, Hadoop, Docker. You can have multiple different kinds of VMs whether they're running straight up on a cluster, one little machine that has several VMs, lots of different things that can happen but what do we do with all those? Because all of those work great but they really don't work great on their own. So in order to support those applications, we need something that's more of an infrastructure as a service and that's where OpenStack really starts to come in and play because here we've got something that's scalable, it's dependable, it's consistent and it's able to handle a lot of different kinds of things. We can have all these different platformers of service items running on it. We can have all these VMs running in different private spaces. We can have bare metals sitting off to the side where someone needs raw hardware power and doesn't wanna share with their neighbors. So as we put all this together, there's a lot of different ways to deploy OpenStack as well and that can get into a pretty long talk but there's different things where you can use, you can use Brandis Fuel, Red Hats, RDO. You can go a little more hardcore and try it with DevStack and have fun with that but if you really wanna punish yourself, try to do it from the command line, line by line and that's a project by itself. But all these things, oops, all these things really stack up to a, I just lost my phone the other way, all these things all stack up to a lot of software but what do you do for the hardware and that's where a big question comes in and most people from the application and are gonna just pay someone to take care of that but what if you're the guy that has to sit there and actually build this hardware and this conversation can go a long time and there's a lot of different factors in how much data are we gonna use, how much processing power is this gonna take, how much power are we gonna burn, what do we do to cool it? We did a session in Austin and we have a backup slide here at the end with some information where you can link right to this but we actually did a whole session on building a flexible OpenStack cloud from the ground up where we talk about what it takes to design the space, what it takes to figure out what kind of hardware we're gonna need, do you need to be scalable or are you gonna be just a little unit and all this stuff and you can go through all of these things but what it comes down to is we selected OpenStack because it worked so well for everything that we needed to do. We want the flexibility to have cloud foundry. We want the flexibility to add something from another vendor. We want the ability to run some containers. We want some bare metal hardware on the side. We want a bunch of VMs and all this stuff can either interact or be in its own space and OpenStack will provide all of that in one big platform. So as we look at that, so some of the platform as a service options, cloud foundry, this is an industry standard and it's kind of a one-off all on its own because it's hard to understand how it actually works but it really does provide a buffer from both sides from the application engineer and from the hardware and it interfaces with all of these things kind of blindly and is capable of handling all these other industry standard connections but once you set up in cloud foundry if you have to move to another data center you pick up and you move and you redeploy cloud foundry and all your stuff just kind of fits right back in there. OpenShift is another platform as a service that's an interesting one. Cloud Foundry and OpenShift are both end-to-end so they handle everything from development to deployment all the debug in the middle and all that kind of stuff. OpenShift is a red hat, oops, red hat component and it's built on top of Docker and Kubernetes. Juju, I thought about not having it in this list because it really isn't a platform as a service but with the introduction of the charms that go with it this is a canonical product that is also based on Kubernetes with the charms that you do have some orchestration so we left it in this list as platform as a service. So there's a bunch of these and you can kind of go and pick and choose and decide what they all are and they'll all work on OpenStack. Containers is another one and there's been a lot of talking containers over the last few years. These summits are full of them. This is an interesting option that allows you to package applications and all the dependencies in a neat little package and then just start deploying them and scaling them and moving them around and they just kind of plug in wherever you need them. So that's all great but these items don't run so well without something else underneath them. Infrastructure as a service solves a lot of that. With VMs, we do traditional VMs you've got we can use several different tenants on one piece of hardware and then everybody splits it up and gets a piece of it and then it'll use your hardware better but your performance may not be as good. So in some cases people are gonna want bare metal where you put one tenant they have all access to all the hardware but the efficiency isn't there. So with all that said, let me restate OpenStack was the direction we went with this project because of the fact that we can now build this application in many different forms or in one form that'll fit into several different holes. And as we go along, you'll see and many of you already know that OpenStack provides something that it's hard to quantify in how much does it cover because so far we can kind of throw everything we want at it and it'll run all this stuff and it'll orchestrate all this stuff together so that your user base doesn't have to see all the mess behind the lines and from the hardware end building your cluster, you're able to scale and flex and add pieces, subtract pieces and bring in pieces that are brand new and new to the whole entire scheme. So with that I wanna turn this over to Leong. So I think just now John mentioned, Mike as well mentioned about different options that we can use to build the backend cloud to do all the processing stuff. So we use this example here to show you guys is basically using a Cloud Foundry deployment or OpenStack. So if you want to deploy Cloud Foundry on OpenStack, so these are the few things that you need to be concerned with when you try to build Cloud Foundry on OpenStack. So you really have to measure that all your OpenStack instances from an OpenStack API perspective, you have to verify that your OpenStack API credentials and can make API calls to your OpenStack instances and have to ensure that all your VMs can access to all the metadata services and also need to measure all the VM in between need to be able to ping from each other and also the volumes, the center volume, you need to verify that you can really mount a volume from the VM and have to ensure you also need to be sure that you can upload or deploy. For this case, you're using Ubuntu folding over for Cloud Images. And because of all those security for the VM, able to communicate with each other, you have to make sure you set up your security groups correctly by configuring all the ingress and egress filter. And also you have to make sure that the DNS is working for in this environment as well. So these are all the things that you need to be aware of when you come to deploy Cloud Foundry on OpenStack. So this, on the other hand, here is basically the Cloud Foundry architecture. And anyone use Cloud Foundry before? Yeah, I don't usually see it. Yeah, so this are the generic architectures that you can use. So Cloud Foundry really works very well on the OpenStack Cloud environment. And I think in this demo, we actually as Mike has mentioned, this was deployed using Cloud Foundry on top of OpenStack. So next thing I want to talk about is the data processing model, which is more on the IoT analytics side. So if all the IoT data is coming in, there are basically three ways of data processing model that we can look at. So first thing is what we call a batch processing model. So in a batch processing model, this kind of analytics basically based on the historical data, can be your operational data, or you can also draw or link information or data from your enterprise system or even the social media. And then you can use this model to process the IoT data which is coming in and you store those IoT data in some beta stores. And then later on you can do the batch processing and integrating them with your existing enterprise data or social media data and generate a kind of like static view of the report so they can do some analysis. So this is the very first batch processing model that you can do in IoT analytics works. The second thing is about what we call a real-time streaming. So in the real-time streaming as the name shows, basically allow us to do real-time analysis. So this is what we call a stream processing model. The data is continuously streamed and directly analyzed at real-time. So in this model your actions can be treated in case of some occurrence of specific events. So in the IoT examples, if you're using a smart home, as Mike just now mentioned, if somebody opened up your garage, you can immediately trigger those alert and send those alert to the cloud and then notify you through any of the SMS or mobile notification systems. So that's one example. And other use cases of IoT apart from smart home, other IoT use cases example can be like credit card analysis or fault detection. So those can be used as well. So the streaming processing, real-time streaming processing model, it allows you to do real-time event-based analytics. And the other model that we wanna talk about is what we call predictive processing. So predictive processing basically, as the name says, it predicts outcome based on the most recent and historical data. So this model basically is used to predict future outcome, the possible outcome or behavior or any other actions that you want to perform in the future. And in this analytic model, it generally requires to program some predictive algorithms in your application. And so one example that we can think of is, for example, doing some predictive maintenance. For example, let's say you do IoT devices in the manufacturing use cases. Every manufacturing machine can have IoT data being sent over to your cloud and do predictive analysis. So those data can come in from the machines or from the engines for all the sensors. You collect it and analyze them so that it can perform some predictive actions. For example, you can recommend when is the next, when is the best time to do the next maintenance cycle before a potential failure can happen in your factory. So these are basically a tree processing, data processing model that we can use when coming to IoT analytics use cases. So having understand the processing model, I still have time, right? Yeah. So having understand the data processing model, how are we going to process those data? So a common use case, a common example of application that we can use here is Hadoop. I believe most of you understand Hadoop. Anyone doesn't understand, does not know about Hadoop architectures? No, everyone knows? Okay, only a few of them. So basically this is a very generic architecture from Hadoop perspective. So in the Hadoop environment, you basically have this name note which is in the master clusters. The name note basically store the metadata about the data and you provide some lookup functionality or tracking where the data and files are in the Hadoop clusters. The name note itself doesn't store the actual data. It generally, the name note basically generally requires high memory or RAM allocation and most of the processing tasks being done at the data node. So the data node at the bottom belongs to the so-called slave architectures. It is a worker node that do all the responsible for storing and compute all the data and it respond back to the name note for the file system operations. And the name note, the data node generally requires high amount of storage space or the CPU processing. So there is a standard Hadoop architectures and I believe you, if you still remember the keynote we mentioned about the workbook reference architectures. So we work, I work very closely with the enterprise working group. So this is one of the example Hadoop architecture in a simple clusters that you can do with OpenStack. So as John mentioned, you want to make these infrastructures flexible. You want to run Cloud Foundry. You want to run Hadoop. You probably want to do some application running on container or VM. And OpenStack really provide a very powerful platforms allow you to run multiple options to support your developer's needs. So for the Hadoop case or the, if you're interested you can find more information from here about how we can deploy Hadoop using OpenStack. So with that, I just pass over to... Sure, Mike. Okay. We're kind of bringing things together really. Looking at what is different from IoT applications. Now it does have a lot of the cloud characteristics that we're familiar with, but there are some things that really need to be considered as we all have shown, there's a lot of different approaches towards data analytics, but really where is that data collected and where do you need to make decisions on that data? Is it's quick decisions, things like that, you may want to push that more towards the edge or have the compute power with you at the edge versus the deep and detailed analytics that you would have in the cloud. Of course you're going to balance that control depending on where that location is and all the other pieces that come into that with scaling of these applications. That's why the benefits of a platform as a service solution like Cloud Foundry come into play and provide that scalability for you. In the end, you really need to know your requirements of your applications. I don't think that's anything new with anything we've all been doing and deploying applications from many, many years, but just they are a little bit different at knowing how connectivity works is a big part of it. And so loss of connectivity is not an issue. It's just sometimes part of the way those applications operate. We need to plan for it. Design for failure. Exactly, yeah. So in bringing things together, we could only skim the surface on everything that is involved with this demo all the way up to the cloud. And so what we've done is we have a white paper that is available on 01.org. It's Intel's open source site, but there's a QR code. You can go ahead and look at it. It actually links to the Git repo where you can actually download. This is all open source. Every bit of it, even the code we deployed. So you can actually deploy it and play with it and have fun in the IoT environment. And so, Iotivity, there's also additional links there that is a Linux Foundation project. So that's completely available. Over 200 companies all working together to create these standards within the environment. So with that, thank you for joining us and we'll open it up to questions. You only have 30 seconds left. Wow, how's that for timing? Yes, am I using? IPv6. IPv6, no, not within that environment. The question is if we're using IPv6 in our environment, well, we're not. And we haven't tested to that level, but let me answer the other question of why Cloud Foundry was necessary for IoT. Really, it was for the platform as a service. It wasn't specifically that we had to use Cloud Foundry. We could have used Kubernetes or Docker Swarm or other rocket or kinds of solutions that help us with containers. But what Cloud Foundry and other PaaS solutions do is they provide, similar to what we're very used to with infrastructure as a service, is exposing that infrastructure as an API and how IoTivity exposes all these devices and does a device discovery and manages it through an arrestful API. Same thing with pushing your code. All I have to do is just push my code using that API and I can rapidly scale it and manage that environment so much easier. You know, I don't have to worry about the individual. I need to employ a LAMP stack. I need to do all those security and pen testing. A lot of times you can go through and validate that within your PaaS stack. So the example that we show here, basically, we just use Cloud Foundry as one of the solutions. You are free to use any other options, including, very safe, any other. Well, it all depends on how you develop your applications, right? You need to develop applications to support IoT use cases. You can develop your application in the microservices way on Container and which also can be run on OpenStack. You so can be deployed those microservices in VM without using any other PaaS layers. That's all up to your developer choices. So what we're trying to say here in the presentation is having OpenStack as an underlying platform, you can support different type of options that best suit your developers so that you can deliver the software faster and easier to support IoT use cases. No, no, it's really, it's a matter of choice. Yeah. Yes. Sahara. Sahara. I think at the time when this project was started, the Sahara is not mature enough for us to test it. That's why we run our own Hadoop clusters on it. So definitely, yes, Sahara is another project that you can look at when coming to big data processing. So in that case, if you have a development with Sahara on it, then you would need to have platforms or OpenShift or Shushu. That's correct. Okay, thank you. So use OpenStack. Yes. All right, well thanks everyone. And if you have any other questions, just feel free to come up and join us. Thank you.