 Think it's about time right? Yeah Okay, let's get started Hi there Thanks for coming out. It's a last day of the convention. It's an hour before lunch. So that takes real Confidence to come here at that time I'm Martin. I'm the lead DevOps Operations and support for for data and Today I will be telling a little bit about what for data exactly is who we are and How we work together with NetApp in a story. That's about hybrid and private deployments so Data is actually an internet of experience platform. It's an extension of Internet of Things. It's the new level that we have and You heard during the keynote Jonathan was talking about it's all about collecting and moving Data around so actually what an internet of things or an internet of experience platform does is It not only makes sure that we can collect that data it allows you to analyze data on top of it and Build decisions and make decisions out of it so It's a connected world. We're getting into a connected world. You can connect basically anything to anything Which allows for the creation of new and user experiences? on top of these platforms, we're starting to be able to Correlate data between a huge amount of different data sources Which opens up an entire new range of possibilities? the Verdita cloud Verdita is a cloud platform a cloud service a managed service and Like I said before it's a platform on which our customers can come in and build new type of applications and create new value-added services for their customers We have a set of client libraries that allow devices That are basically any type of device can be an iOS device can be an Android device can be a Linux Embedded device system on chip if you get into the blueprint We have libraries that you can put into those and you can start ingesting data from all those devices We use a published subscribe mechanism for our platform, which means that the communication is bidirectional You can ingest data from these devices But you can also single out a device or a group of devices and send information back to those We use MQTT for that and then we also have a set of a rich Public API's that allow you to retrieve the data and work with the data as you were In origin we serve a number of vertical industries So one of the industries that we can cater to is automotive where we can do stuff like Reading real-time information coming out of cars tracks We can store all that raw data in a historical fashion So for example, you could have dashboard information that you can retrieve in real-time Or you can have an historical overview on the performance of the car We do stuff around health as well where you can have a real-time overview of complex sensor data correlate that and Make decisions based on that With while also having the historical overview of a customer patients Health as to it is available Then another industry that is very active is The retail industry where indoor location based tracking is becoming more and more apparent and more and more to the forefront Where you can have services like or when you enter a shop, they know who we are They know what your interests are and they will serve you with Individual targeted advertising when you want it and when you need it So I'm going to show you a couple of use case which will show off the capabilities of a data platform I've got it sitting as a movie here, but I think we can take the chance and try to do a live demo Which actually gives you a better view on its capabilities So let me switch here so what you're currently looking at is a live multi-clouds platform Public cloud platform where you can see that we have a number of connected devices. We have about 180,000 connected devices on this system and you can see the historical overview Hour by hour, so it keeps updating real-time and you can also see the the view over time itself I'm not going to go through all of the demos that we've built on top of the platform to show you I'm just going to give you a small sampler if you're interested You can always contact us afterwards and you can get a better view on the platform capabilities But I did want to show you a couple of use cases that Are I think pretty important To give you a sense of what the capabilities of the platform actually is so for example, this is a automotive Fleet management demo that we have where we have a set of cars that is driving around We use an OBD dongle that is in the diagnostic port of the car. Every car has such a port It's got a Bluetooth interface the Bluetooth interface is reporting to the smartphone and the smartphone is sending that data back into the platform So what we can do here is we can actually have a view on one of these cars We can single out the car so you can actually see them driving we can Go to its dashboard and you will see the real-time Dashboard indicators of the car while you also have a visual representation of where the car is actually located due to a representation in Google Street view and Another thing that's quite nice to notice is that we're also recording The driver's heart signal. This is due to a partnership with a company called the sensors That retrieves the heart signal from the driver It's a hard signal not a beats per minute which says which means it's granular enough to identify The person driving the car and which allows for advanced analytics such as detecting Very well in advance 10 15 minutes in advance if a driver is going to fall asleep. Yes, you know So this is the real-time capability you can also go back in the historical view Where you can check out the trips that this vehicle has made in the past and Once you have that information available It will allow you to do things like cost calculations in order to See where the car has been and how much it actually consumed And I think here the gods of demos are interfering a little bit And so let me refresh that so I think I just lost my connection Yeah, we did so here we're back So let's try the historical view again so here you have the historical view and You can do a cost calculation based on the fuel consumption of the car per trip The duration you can go back and check its route one of the more interesting things you can also go and check the performance Of the trip meaning that all the statistics coming out of the engine block are Reported as raw data and stored into the platform for indefinite Duration one of the really good benefits is you can go to your car dealership and say I have a problem They will tell you we're not noticing anything and you say oh look at the details It happened during that time and this is the problem that I was actually seeing Okay So it's a nice demo with we found out that yeah There's actually a lot of people that can do something similar to this So how do we differentiate us? One of the things that for data in its origin always had was scalability We did not take the approach where we have a solution that we're trying to scale out Possibly to millions of devices now we started off with an architecture that allowed us to start from millions of devices and skill potentially to billion of devices so therefore we did another implementation of this and Currently what you will see is we have about 300,000 cars driving around that we can actually track individually Where is this data coming from people are kind enough to publish their trip data to open street map? We've taken in the data and we're replaying it to a set of simulators Which gives us the possibility? To give you this very nice feature again. It's not our intention to build these applications These are applications that are intended to be built by our customers that create added value on top of the platform Specifically for them. So this is another view that you have where we use octagonals So the previous one of you that you had was a heat map according to the density of the cars that we're driving Here you will have a view that combines the actual speed Of the cars in combination with the density of the cars that you have driving around So cars in this area 107 average speed 0.38 kilometers meaning most of them are actually standing still it's nighttime. So yeah, that kind of makes sense Let's go a little bit. Let's dig a little bit deeper and at a certain level. You actually start seeing The individual cars and you can see the cars driving around Our design is modular meaning that we're using a different database as the one that I was using for the previous Demo Which also shows the flexibility of the platform in order to create This overview. So here again, you can track all the statistics of a single car And you can see it's actual if if I'm fast enough to grab one of them No, I'm not. So let's try it again Okay Anyway, I think you get the picture The the statistics that we've shown before is exactly. Okay. Here we go So you can see the route and you can see all the engine statistics In time series based overview for each of these devices Okay, so that's a little bit of you on the capabilities of the platform We have others we have health demos. We have anti-churn demos Built on top of it, but I think you get the drift I think you get the idea of the capabilities of the system so far Now, how do we manage this? That's That's a big question to hold on and that's this thing Okay So we when we set forth to actually manage this platform We we said we gave ourselves Base of requirements that we had to adhere to first of all, we had to be cloud agnostic We had to be capable of deploying in any type of cloud To this day we support IBM software we support Amazon we support open stack And we're fairly confident that we can build drivers for any other cloud provider out there in a couple of weeks Flexibility and elasticity so We did not want to be bound by a single type of solution We wanted to be able to mix and match The components that we needed because every customer is different every customer has its own Use case so we've got the pay-as-you-go model with a multi-tenant platform here But we also wanted the capability that if a customer wants on-prem local We can provide as well and we also wanted to view that if you need a performance system You want to choose between performance and scaling why it's being elastic you should have that as well So bad metal versus VM On-prem versus public cloud all this needed to be supported in a single system Managed versus firefights very important. We took a DevOps approach to the implementation meaning that we wanted to serve our Development community as well as our support community as well as a customer community for one single point of view and to end I Still think in the industry we need to have much more focus on The end goal being we're not looking only at infrastructure we're looking at solutions for customers and You can look at a solution from a customer point of view and you can look at a solution Delivery point of view and it needs to be seamless throughout the chain So that's what we set out to do only in that way. You can actually manage a huge platform With a very small set of resources. That's the only way you can do that and The good thing is I heard a lot of talks about microservices containerization So it seems that like indeed we're moving in that direction, but I still think it's the entire flow. That's important That's what that's where we should set our goal okay The special view on Our deployed solution so on the left-hand side you see where the devices are actually reporting the data and when the devices Where we're sending the data to Once you get past the protocol adapter Which terminates your incoming message? It sends it on to the message queue from that moment in time. We will not lose your data It is stored We stored data hot for a month and we also store it in an archive and We can store it a couple of years. We can store it indefinitely That's up to the customer to decide but as long as you want you have all your data So if you want to do analytics on top of that and at one point in time you get to a conclusion Okay, I was looking for the wrong thing. I want to revisit all my data You can do that. You can just from your object store Gather that information back into the system and do an analytics on top of it with service such as spark Which is also something that we offer No, I think Covered most of this so the ver data deployment models as I said we have a full Public offering a multi-tenant offering where you pay as you grow Which is cloud agnostic can be deployed on any type of public cloud and you can have Storage solutions behind it that fit your need We then have the hybrid the puddle deployment model where we actually have two options You have the option that I think is most relevant to most users where you have a an enterprise or or Large corporation that actually already has its own infrastructure and is ingesting data from its own devices and Wants to extend that capability or needs to extend that capability with device data coming from the internet so then you go to the hybrid model where your Messaging infrastructure is located publicly ingesting all the public data coming into the system and Your data back-end is on-prem locally taking care of all your data making sure that all your data is in adherence with your privacy and data concerns governed and In the meantime you can also ingest data from within your intra net Possibility so it's covering all the angles why you still have full control over your data locality and your data governance Then you have the other option where you decide I want the flexibility to really scale out Using VMs. I'm using the compute From the public cloud, but I still want my data local so you can have your storage solution locally While you use the compute the compute from the cloud And then you have the private Managed service so you have the convenience of a full Private system while you still maintain all the benefits benefits from a managed system Like the elastic healing the self-healing the elastic scaling From the managed services So how is it done? we basically Took open stack and Expanded on top of it making it into a multi-cloud management platform Where we directly go to the APIs of the different cloud providers and start provisioning the system on each of these clouds products Very important distinction is that we don't look at it point of infrastructure. We look at a point of Service delivered service. So we take care of the infrastructure. We take care of the application We take care of the the the health of the applications on top of that all this on top built on top of an open stack solution so the hybrid solution model a little bit the same Same feet where we directly go to the cloud APIs so the the the The cloud provider has its own API Open stack has its own API and manager on top of that and so you can see the high-provider Details through it, but the actual provisioning the actual governance of the infrastructure is done through the multi-cloud manager same thing for the private solution deployment where where we go to a flex spot and provision The services on top of that So why open stack and why net up? We started on this road About a year ago a little bit over a year ago where we decided that open stack already had a lot of The capabilities of what we needed It stopped at the infrastructure layer, but at least up until that part. It all made sense What we were seeing So with and it was also a very nicely compartmentalized in different services which made it very easy for us to integrate whatever we needed on top of that And yeah, the usage of the APIs you can deploy open stack pretty much anywhere. So that made it a very good generic solution Then the value of the the net up augmented open stack services So what is very important to realize is that the dynamics of a public cloud environment are Very different from the dynamics of a private cloud deployment in what sense? for example, you don't care or You want to care, but you cannot about as a lace in a public cloud They're not giving you any as a lace in public cloud So but you you can skill in pretty much infinitely So if something goes wrong, you can have other resources you can go to other data centers things like that When you look at the private clouds deployment, that's typically not there People only want the infrastructure that they need in a private cloud deployment meaning that it becomes much more vulnerable to defects things like that So it absolutely makes sense to go for an enterprise solution as opposed to just having a bunch of commodity hardware and scaling out That's the two approaches you can take you can say, okay, I really don't want to care But this enterprise great thing, but then you will have to make sure that you have enough commodity hardware to cover all your bases Frankly that creates an overhead in terms of your operations and you will have to do a lot more stuff So it kind of makes sense to go for an enterprise solution if you go for a private cloud there Compliance and security data locality data governance Very important You want to be able to state that you exactly know where your data is what it does And that it's it's actually under data governance control Consistent performance and as a lace Usually when you go to towards an enterprise system, it's validated. It's tested You know exactly what you get You know what performance you should expect from it as opposed again to commodity hardware Where you're usually mixing different systems together with each other. You will end up with a lot of issues if you scale Beyond the certain point And full control over your private and hybrid clouds One of the things you want to be able to do in a private solution for sure is you want to be able to direct where your resources go Which is something That thanks to the augmented surface again on and that place is pretty easy to do Okay, I will give you a short demo of the Vredetta multi-cloud manager I will then show you a deployment of a hybrid model using VPC and Deployment on how the deployment on an X on a flex pot actually happens A few things I need to tell you is that what we've built on top of open stack Raises the view on the system. So we've actually started from release creation So within the platform, I'm going to decide to create a new software release When I click on the button, what actually happens behind the scenes is that I branch all my source code I take all my attributes. I take my binaries I take my orchestration cookbooks and recipes and I bundle it together in a release Once that's done, I can provision my cloud providers that I'm going to be using in the system I can have different. I can use different credentials if you want to do if you want to switch between accounts I create master images from the system in order to have a base that's under my control from where I want to start off from You create the flavors that are Correspondent to the different cloud providers You create facets. What is a facet is facet is basically a VM that has a very specific role So what you do is you create a facet and you start building your recipes You put your recipes in your facet and it's Then becomes a certain VM role On top of that what we needed was the view from a perspective of clusters For example, if you use a GFS, you've got the naming nodes. You've got the data node. They all serve a Single application. So you want to group those facets into clusters. So the concept of clusters was introduced And above that you have the concept of an environment an environment is a controlled Service deployment What does it mean? It's a set of capabilities a set of clusters that come together That are Orchestrated in such a way that they combine a single service identity So for data as a platform is one of those services We have another service park as a service that we do similarly to that And I will now show you a little bit of what this multi cloud manager actually looks like so as you can see it's it's very similar to a vanilla Open stack deployment with the The main difference that there's an extra tap in there. That says for data And under for data you can find the concepts back that I just mentioned. So under configuration You will get to decide which cloud providers which master images which facets which flavors You are going to have you can set attributes for your machines But above that you have the release creation where you create your software releases in You can do event management. You can track who did what in the service and here You have an actual overview of the environments that we currently have environments So for example, one of the environments in here is our actual development environment So we've also added life cycle management to the capabilities in the sense that we Launch our development environment every day from scratch At six in the morning and we break it down at eight in the evening when the developers are gonna have gone home like that we save a lot of money and we Pressure the developers into making sure that stuff works because the next morning the platform needs to come up again and work for everybody So from an operational point of view very good benefit So I can just add an environment here It's both Lee shows you a very nice name because I've been told that I'm terrible at picking names So we included something that gives you a name for you So summer snow There you go, and I can start adding instances I can Select my cloud provider. I can select the data center that you want to go to I Can select the facet the role that the VM is actually going to play I Can select the amount of VMs that I want to launch And I can also select the release that I wanted to play So I can just add one of these so that's an EC2 instance I can add another instance Let's make this one a software instance Same software lease and Then I will add one more which is then how we deploy on a flag spot by using an open stack implementation And here you go. So I've now I have three instances in my environment So all the security groups are set for this environment to behave as one coherent part So I can now start the environment Some of the things I can do so you will know The system will now go out to the different cloud providers request the VMs Once that done when you once you get confirmation It will try to SSH into the machines once it's done that it will start your orchestration in order to build the applications on top That you need to to get into that environment Some other things that you can do is if you decide to change the firewalls you can you can Re-provision the firewalls directly through it. You can kick the system meaning that if I change my software release I just have to kick the system. I don't have to rebuild the entire VM It just rebuilds the software on top of it and I can obviously set the software release of each of the components that I want Now if if you go into a production environment that we have You will see that these environments are actually quite big. There's a whole set of instances here It's it's very difficult if I want to create a new environment to do that So I'm not going to do that. What I will do is I will take that production environment. I will duplicate it I will select a different cloud provider for it and a different data center and I Will just duplicate that environment Which one was it? red So when I click in here, you will see that we have the and the entire new environment And I just need to start it and I have a duplicate it Complete environment. So all the settings all the attributes that are dependent on an environment have been set and have been met according to Now let's go back to our previous I think it was black cloud No, it's not black cloud It's the problem when you start creating all these So there you go Should have another beer. That's usually helps Okay, so Now we're on the chef run. So where open stack stops is with the creation of your infrastructure This takes it to the next level. This means that What it actually means to be active Here is that the application is up and running. It's healthy and that is what Active means you also have the monitoring capabilities associated with it You can see the last events that have happened on it and you can actually Once the machines are active you can download the keys for it if you have the The necessary security Role you can download the key and you can go directly into the machine if needed So each of the machines that we use has a different SSH key So security is governed as well by them. So that's that part. Let's go To the monitoring part maybe a little bit So when I go in here You will see that actually I will get an overview. So we use monet an open source packet as well And what it does is it resolves all the dependencies that the applications have on each other It checks them and only then will the application be deemed to be operational And if needed will make we will take actions to rectify any issues So I will let the Machines actually launch Let's see. Maybe there are already some active So, yeah, so the open stack based one is currently already active and the other two machines are Still building so that's all it takes to build a hybrid cloud environment in a system based on this We've also so this is for a normal hybrid environment We've also done stuff around VPC That was a proof concept that we've done Where we actually use net app and PS private storage that's attached So not a private storage which is on-prem That you can then use in combination with your compute in the cloud So It's this one, I guess See if I can actually play the video. So what you will see happen here is that You go into the configuration and you go you have to import a VPC. That's an Amazon VPC Once that VPC is imported You will be able to use that in your environments So here you see the actual creation of that So you can actually see that it's the same value as what the net app NPS Manager shows you Once that is imported you can then spin up machines that are actually going to use volumes of that NPS part in your environment Here you go. Now. It's as easy as in the environment selecting the facets that you need When you create a facets, you're going to say that you have the VPC capability and That you're going to attach the volumes accordingly So a storage type NPS We set the number of machines. So it's a Cassandra cluster So Cassandra cluster It makes sense to be able to take snapshots of your Cassandra as a backup and to be able to restore from it So that's all it basically takes for that Now Let's first check on our environment So for our environment two out of three of the services have already becoming have already become active The third cloud provider is still provisioning the machine and will deploy Once it is done that again, this is an application point of view That means that active means the application is a fully functional and running on top of the platform so in conclusion why use a flex pot for your private and Hybrid deployments you want to have an enterprise great solution in place in order to Achieve maximum efficiency behind it if you use commodity hardware Usually you're going to run into problems In terms of manageability of that infrastructure in terms of that you are going to have to work around the fact that There is always in contention between the amount of Resources that you have in a private cloud as opposed to what you actually need Also, it becomes a lot easier in terms of customer compliancy data locality And you have full control over your private cloud infrastructure and data Okay That's a little bit in a nutshell what we do and how we do it So if you have any further questions I'm open to suggestions No questions What are some of the benefits of having using multiple cloud providers in an application like you're in your example? Okay, the benefit of using multiple cloud providers is that you're not tied into a single vendor You can easily switch over or you can actually combine them In terms of resilience a certain cloud providers Have less performance Definitely the more immature ones and you want to be able to go to one of the bigger ones to Make sure that your high availability is covered like that Obviously, there is a cost associated with that as well in terms of latency and Architecture around it. So you have to be very careful how you do that But that's the main benefit behind it Any other questions? No, then I would like to thank you and enjoy your lunch Thank you