 Hello everyone, thank you for being here as I said second day people are really tired a lot of topics a lot of Things to think about so thank you for choosing this one We are going to be speaking on machine learning on the edge. So just check that you're on the right talk Hopefully you are too late to leave now. Stay please I'm going to be here with my colleague Carlos de Huerta. I'm Pablo Perez digital advisor in Microsoft consulting services here in Spain Carlos de Huerta. I'm working with the manufacturer of customers here in Spain as architect Because the talk is in English. I'm going to call him Charlie. Okay, Charlie from the garden Charlie for the game I guess you don't mind So this is the agenda and we were looking at the agenda and we with like what do you think about the agenda? What do you think about the introduction? No one likes introductions No, we actually we ran a machine learning algorithm to predict your emotions during the talk And this was the result all the forecasting You are going to be bored to death for the 10 first 10 minutes And we know that because of Microsoft we have this software that I loved because they told me that I was 25 years old and of course I'm not that looks at people and understands emotions And we ran this algorithm and this suggested that this is the right Outcome and we believe algorithms right so we say okay. Let's change. Let's start with the demo Let's start with the demo so you are not bored, but you still need to go through the Valley of the death of the slides in the middle of the presentation There will be some slides then we have another demo to keep you awake until the end and we will finish with a video Of course as I said we are believers of machine learning and the in the machine learning algorithm also told us the recommendation for the titles You cannot call it demo demo. What is this talk about? So we are going to start with just a short demo very very short But to for you to understand when machine learning on the edge really makes sense to be applied Okay, then Carlos which are Charlie will speak about the use cases and for you to think about where you can apply this tomorrow If with your customer or your internal departments or whatever And then we will go to the what is the edge and what is the matching learning by then? Hopefully you you grasp what we are talking to about talk here. Okay So without further ado, let's go to the demo and then hope the connection is working fine that it was not a few minutes ago Can you see this no Can you see my browser? It will be nice Close the PowerPoint Okay, now better Vulcan is it's not a contortion company This is a real company in the States in the Midwest. There are still producers the Manufacturers of the still and what they told us they told Microsoft is well They wanted to have a dashboard to control the health and safety. Oh There is an alert for that one the coincidence, right? So they wanted to control the health and safety situation of the tracks and they have around the states Okay, this is a really serious issue because every day in the States for people are death because of health of safety reasons Okay, and it's not only that is the fines and everything related to having those kind of problems So the idea was to be able to create the dashboard that in a real time And this is the important part here in a real time will notify the people about Others or risk situations when they need to act. Okay, so suddenly we have this health and safety bridge bridge And we see here that in these steel beams instead of having a one high level That is what the policy says they have to high level. Okay, and that's not correct So here what is the algorithms doing is bringing an alarm to someone that is looking at the window and say, okay This is not correct, but also look at this guy here. Can you see that guy? He's not wearing a helmet and that's not correct. So this is the reforms learning So he's okay. This is not correct. We need to change the algorithm We are going to retrain the model And how we are going to retrain the model just by saying that no protective helmet is detected Here, okay, of course now we need to retrain this model. We click on save and We have coming from all the tracks in the world They have images that have been captured and they need to process the images and retrain the model to be smarter and easier So we can click here and retrain the model, of course, I can choose as any Pyrton machines that I want the model and I train This is a simulation, of course. Nothing has happened and not not depending on the Wi-Fi that was too risky But now the idea is that the that the model is being trained with the images and and everything that has changed around the Glove it's not important that is specifically here and this is the key point I have a fleet of 100 tracks around the world I cannot go with an USB to every single track and deploy the new model I need a system to deploy this model to all the tracks And then that's the system. I click on deploy and I'm deploying this of course if we go and we will go a little bit deeper in the Technology behind that this is using containers on the old algorithm is running while I'm deploying the new algorithm Okay, the new algorithm is deployed. So I click on test the same image now is saying hard hard not detected She'll be loaded. Okay, as I said, the Wi-Fi is not the quickest, but this is good for this demo So this is what we are going we're trying to get here is Having these tracks Offline or with a really bad connection being able to have real-time responses on the tracks with the cameras But all the information that is gathered from all the tracks going around the states take that information and put it into the cloud Process that change the algorithm and deploy the algorithm back. Okay So I will just explain quickly What has happened here? Yeah, yes So here we have basically what is going on behind the scenes. Okay, so we have the cloud where we have the IoT hub What is the IoT hub is the thing that is getting all these messages from all over the world and it's processing those images Then we have the insights actions. Okay, don't underestimate the insights part. That's the big data part That's all this all that's what you know how to do is getting data from all these different sources and finding patterns Okay, that's why the cloud is important because there you can going to compare Data from one place to another then you think about the actions and the important bit is this Those insights and those actions those can run on the edge on the track Because at some stage your connection won't work and you need that circle to be processed right where the data is generated That is in the track So that's what we meant by Azure IoT edge Basically three things an SDK that is going to manage all the connection is going to manage the protocols You don't need to know all the protocols of PC UA Modbus all those protocols. Those are not relevant for you So I th is going to provide that SDK is going to provide a modules that you could customize for your problem That is where the insights and the actions are going to be stored And of course you need something on the cloud to manage all these devices that is not one ten or ten can be thousands Okay, so that was supposed to be the last demo. That's what we changed in the order Hopefully now you can understand models less what we are speaking about when we speak about matching learning on the edge and this is it Now Charlie will explain some examples about use cases where we can really use this Yeah, and here I want you to start thinking in three patterns that we will see and even in this demo from from Pablo What's on up is the three patterns that is important when you think about use cases for for the edge first is The edge is a cloud endpoint. So at the end you're able to leverage capabilities from the cloud at the edge But not only for capabilities but for For for features that for security you need to manage all those Potential tracks or we'll even see other scenarios where you need to manage that security As well the management and deployment at scale of those tracks or all those edges of those endpoints all around the world And for that the cloud would be an enable Second the point that you need from the edge to connect potential Sensors that are not connection capable at the end for example the manufacturing and or any Scenario that has sensors that have information that you need to move to a central endpoint Those are not directly connection. So it needs to have a gateway that would be the edge and third And in this scenario you will see that maybe that information Needs to be run quickly quickly in the in the sense of I need to act In a machine under the second so I don't I can't wait Things to move to the cloud and come back and as well. I need to be able to run offline so at the end those three patterns would be interesting to To put in your mind to review for example this type of use cases We may you may have heard about the same autonomous car at the end a car needs to manage and to handle a lot of sensors and For that information that may be interesting to upload Without the end point that I was sharing as As well I need to act locally of course from A point of view asset management drawing review of Assets as more agriculture you see many scenarios that Afterwards we can we can talk but we wanted to focus on some examples that leverage that Patterns and that idea that we want you to think about it that how could I learn and And deploy it's a scenario in in my in my company and my customers in this case is I think in a hospital where I need to maybe understand and anticipate medical full and And rise resolder in this case is focus on vision. So the idea is you'll see at the in your Right in your left. My right is the live video of in this case the hospital at On the outside you will see which other ads are rise based on Different patterns in this case are trained for classifying the sense of we will see What's happening inside the in the bed in the room and then rise those those alerts I Want to put one of those in my house also is to check when the bed is empty You see there for the first is saying this is occupied. So it's understand that there's a man in this case in in the room and It's important. For example for older a man or woman that they have them They call the role. So it's down. It has detected and jumped at that moment that Alert So the as well that this has it's sitting. So it's it's moving or it's empty They got there in the bed. So at the end is trying to Think about those scenarios are talking about the edge in this case. I send some Some alerts in in the sense of the hospital without really moving out Out of the cloud second for example in the manufacturing space Even in some Multiple scenarios sometimes we think about machine learning from predictive Maintenance or or potentially improving anticipate the error that many times And in the manufacturing and even the energy in a space We are moving a lot of but the preventive maintenance so that a people it's a moving Inside the in the the assets and the buildings to find out this is there's any problem So sometimes as you gather information to build a predictive by a predictive maintenance scenario You may start doing preventive automated Maintenance so in this case based on Reginalm convolution network network and java variable the fast fine potential errors in this case cracks imaginary and then being able to Do for example corrective maintenance while you are moving into our predictive a scenario We're more in the retail in this case is banking airports. It's Maybe Seen as an easy one at the end when we are accused and we are waiting to to be at the end of this and In the airport in in bank It's important sometimes simply To count people a check-in area to notify the staff based on on the queues and the number of the people and at the end we are able in this case to manage and anticipate and in real-time Leveraging capabilities from the cloud that at the end running at the edge have been able to to handle this scenario in this case is even focused for us As an ordinary of looking at the head and soldiers for the people Moving more and this is I wanted I like because it's in the vision zero from Bellevue in United States and is focused on trying to reduce to zero any damage for people Environment and in the country and the city so here it's It's a model running on top of building to understand which is the the traffic in a specific traffic and be able to manage Potentially the traffic lights in in this area. So sometimes You are able to understand which are the flows. It's it's a crossroad But which is the people doing at the 9 a.m. Or at the 4 a.m. Or maybe based on the event today Of this week and the big data spain event all the the parking was full based on The people coming here. So sometimes you're able to anticipate and run locally some some activities for this scenario and Think and last but not least and when we think of course traffic and daily safety will So with you the bull can scenario there's more for health and safety security and at the enterprise But looking at the and the public sector Here just you're seeing three different scenarios the first On your left is the big alone soldier in the sense of being able to anticipate potential problems this attract in the middle of the of the of the road that Leverage a pedestrian on the road so anticipate and send alarms in the geofencing a scenario not really Remote leak that as well it could happen and this Learning could be serenade are completely different now that this gives detection that's Well, you can detect someone with a helmet but in this case is there are Thief they're trying to to move with a motorbike into a jewellery is to drop the the jury and And in this case the idea to anticipate that is not normal that someone in the helmet with a motorbike Near the the building in this case moving toward the the shop in this case Not working. I guess so at this stage. We are wondering or the ideas What is the edge? We're talking about the business cases We can talking about how to use it, but now the next question is where is this edge? This edge is everywhere. Actually, you know Some people are saying that the the world is becoming a computer itself and to think about it a technology It's really a breakthrough when you don't see it anymore Like the cables of the Wi-Fi or things that you don't notice and we don't notice anymore But we have more computing power today in this smart watch and the telephone that the day the first day in the keynote They mentioned the power that they use the computing power to put the Apollo in the moon And we have more power these days on our devices that we are wearing So if you wondering where is the edge think about all these places where you can really compute? I know the autopilot is really probably a cliche and you think it that's really complicated But one of the purpose today is to say you can use even your phone and that's the next demo Hopefully will work you can use your phone to deploy an algorithm and use it disconnected That that's the key point you can do have an algorithm that you are thinking about the challenge something that you want to do You have the skills that that's why you are here You know how to do this kind of things now the idea is where we're to deploy that and how to use it How to how to reach the maximum amount of people by being able to deploy that into for example a phone But there are no examples and there is always a but the edge has some problems. What about the latency? What about I need connection at least to connect once and deploy the system and I need security So the edge can be anything But you need to be sure that you are using the right edge and you are using the right technology to connect to that And we are going to speak on some examples from Microsoft and some partners about this edge In detail you have seen Many devices that maybe are more the home or who automation or in this case for a consumer space There are a catalog this is for from Azure Focus on edge for industrial scenarios in this case. We wanted to deliver it to not just really edge from Just the devices capable edge at industrial but focus on the AI enable as devices that you can You can start working for example one that does Usually the deep learning as soon as moving faster in the vision and speech use cases this is a Qualcomm device that's as ready enable for a vision so you can deploy remote Models inside that that device and then be able to run whatever algorithm you want to To be done there at the edge This is more from a developer and even test Prototype even use cases of course you can go deeper in this case and you know You may know the Kinect Device this is Bringing the capabilities and moving a hundred times better from from from the Kinect in order to integrate in potential industrial or other Manufacturing warehouses and retail scenarios, so that you are able to integrate that hardware into your own Products or services Moving more into they say speech. This is another device enabled in a sense of I can integrate from a smart home scenario to a Car Any other potential edge device where I can integrate and deploy models for speech device Speech scenario recognition or language understanding or whatever Use case. I think it's valuable for our customers and in this case is with robot everything is It's really so you can just bring it after that and last but not least the drone Evolution here with the DJ DJI In this case they release an edge device in this case a drone that's able to Deploy inside that drone a model a machine learning modeling and through the camera being able to Make potential Routine or even detect some potential things for example to to review electrical towers or Whatever a scenario you may be thinking about and then of course enable your own edge device This is not something about this is These devices have built and you may use it or not, but at the end as Pablo was saying is this software open source a project that you can embed on your solutions and and use based on a specific use case Something that you observe from manufacturing retail banking and health or whatever scenario you may be thinking about Because we predicted that you will be sleeping by this time We have another demo to wake you up right now Okay, and speaking about the drones while he is preparing the demo actually I want to tell you about the client in Belgium that they were spending a lot of money on putting a right for G For controlling the harbor with a lot of drones and we spoke to them You don't need to have coverage on the whole harbor Why don't you use the edge to process things in the drone and whenever you have the connection you get the data and you come back I said well, I've just spent a lot of money putting the best connection in the harbor I should have known this three months ago. So for you, maybe it's not the late to To find scenarios with the edge can save a lot of money as well in this case we have used a A service in in Azure is a custom vision that where you you you can just drag and drop images make a leveling for in those images and from a Algorithm does already train and make some sort of transfer learning to your own use case So in this case based on initial training For custom vision with a train To models one is for printed board boards so that the the system can learn when A board is broken in this case the the lines One are defective in the sense that maybe when we build those boards Once they were defective and ours are right. So here the good point of the accelerator on of these services that you can even train with a few data just in there we are just I've just trained with 10 images and Based on that you are just simply add images as I have done and Click the the train button to be able to to to have a Model ready for you. This case is pretty fast in the center 15 10 minutes, but the idea here is that after that you can export that model and Run at the edge at the end this service can can be query and run from the From the cloud But of course as we are in the machine learning on the edge the days that that model can be downloaded and can be It can be run in a mobile the NS an edge through some sort of Have you been in the in also there was another talk today about machine learning from our colleagues from go Google Have you been there in that talk? Okay, so more or less. This is the same It's up to you who were to choose but there was we want to go one step further It's one you have the model that is trained. Okay with this process that is easy click upload the images transfers learning What do I do with the model? Demo no the idea is what we are going to do here is to deploy one of those models into a phone And this is where I bring my phone up People that know me try not to send any messages now because you're on a screen Okay, and we are I'm going to use an application and it's called intelligent kiosk Okay, this application is available for laptops in a Windows Store Okay Here you have some demos that you can by just getting your keys from Azure put your keys here and use it to demo Some of the functionalities one really simple one is one that recognizes fruits There is a very basic model hopefully with this light and this is not nothing complicated But she'll be able to recognize that this is an orange with 97% of accuracy. Okay, nothing impressive But the impressive part here is Okay, now if I want to I know what to do with the orange If I want to Download well, I can also disconnect this this is going to run also without connection actually I'm going to do it But I want to add a new a new model I just need to Okay, hello Click on plus and the models that Carlos has shown Basically, if you want to explore the model I come here now you cannot see the screen but basically I Scan that and by scanning that this is complicated What I'm doing is downloading that's more that model into my phone. Okay Hopefully now is downloading the model and it's in my phone It's important when you are creating the model to understand that the RAM and the Computer power that you're going to have is this. Okay now that I have the model here I can do another image classification. Hopefully There was not that difficult that is recognized cats or dogs That was the model that he was trained. Yeah, he trained in the cloud I guess the connection is not working the world. This is really small and it's basically downloading that, okay I think you need to trust me in this one, but we have a backup here We always have a backup that we also have an application that has already the model That is the big data application. We were not really That was the name that we put that basically is recognizing and we can do that the Yeah, if this is effective or not effective. Hopefully the light is good enough This is the fact if they saying this is defective one and to show that this is real What I'm going to do basically is put the airplane mode and then It should also say that this is defective, okay Basically what we are showing here is that you have the knowledge already to create a model train the model We want some applications that you have already the models They are just to tune the last part of the model get that model and install that model everywhere or anywhere And we also have this application real-time crowd insights It's the one that I wanted to show and this is supposed to tell me my age if I Is the one that I love and I can put that here to you also Alberto You can be a volunteer because I have a cable and they actually this should say that I'm 25 well 30 I'm not that young anymore. I was 25 the last time I used this Okay, but Basically what we wanted to show hopefully the demo was illustrative enough and we come back to the presentation is that You can today already start developing these algorithms and Deploying those algorithms to a software or hardware that doesn't have doesn't need all the compute power Okay, so just to quickly recap what why we are speaking here today Why are we are speaking on the IOT edge today? And what to you because we have these waves of innovation We have the cloud and limited source of data and power We had IOT getting all the data from everywhere and put it that into the cloud Now suddenly the artificial intelligence was ready to process all that but the important thing among any others is the last mile Where are you going to deploy those algorithms? How are you going to use those algorithms? Are you going to put that in 1000 devices all over the world to really democratize the usage of that? Agree in that model. That's what we want to do We want to be able to give you the tools to be able to do that Deployment that management of the models that you can already create these days Of course, there are always barriers But the good thing is that if you go together the barriers of the cloud are the benefits or the edge Because what is the problem with the cloud high volume of data? That's that's something that maybe you don't want to put all the data in the cloud You can use the edge to filter what data you want to put up and what data should stay wherever it is generated Also privacy if you cannot put your all your data in the cloud You can train the model with all the data and some of the data will stay on premises You don't want to send all the video what you were saying with the with the guy in the the tracks and also the guy at the Hospital, you don't want to maybe send that you want to send some screenshots whenever something has happened So it's really important to work both of them together So the question is when should I use the cloud and when should I use the edge? And the answers are pretty simple if you are doing remote monitoring and management It won't do you when you want to do is compare a huge amount of data Then the cloud is the perfect place to do that If you are merging data from different sources Also, you can do that in the cloud and if you need the infinite computer the cloud is the best part to do that But if you know low latency then go and apply those algorithms to the edge if you need protocols translation or you need to Basically, you don't know about the motics and plcs or pcs the protocols that are there No, no, there are not many people that know how to program for those protocols This is one of the good things about it that we are putting a layer on top of that So if you want to program something that works with the camera in your house Do you don't need to know all the protocols? You can use Python or you can use any of the tools that you already know So this symmetry is really really important to be able to do something in the cloud or then something in the edge And those things are synchronized to give you the capabilities to really deploy models anywhere But when we speak about the edge we need to think that the edge It has an spectrum as well It can be something as small as a microcontroller with we call it as your sphere and it was the first Linux distribution Deployed by Microsoft, but you also have it device as I said You have Windows IoT you have Linux as a creative system Raspberry Pi The Azure stack and then you have Azure as well Then the intelligent goes up and you can use the features that are in the cloud to do these these things that we are speaking about So now that we explain more or less and we have five minutes left What is the edge? The next question is why much in learning? Okay, and then I like this picture But you know what is this? This is the data set when you go to the client and they have they say I have all the data That's what they have and many talks have been speaking about it is important to have meaningful data Because when you have meaningful data, you have something similar to this And if the data that you really want to have what you're expecting and the client first time They think that they have but maybe they don't this is what they have So you need to be able also to take decisions to to to fulfill your data set with the right data So what kind of decisions do you need to think what data do you need almost real time? We call that hot path what data you need that can be processed in a batch mode. We call that call path and Even more and more these days where we have is a worm path This is a lambda architecture explaining a very basic concept So it is hot or or cold and today the technologies most of the times Allowing us the possibility to do it warm as the speed as you want And also, there's another question Do you need to cook a little bit the data like the CIS survey before you put it on the cloud? You need to you need to do something like putting together Few signals and create one signal Do you need to look at the window to see the average temperature in that window? You don't need to do that on the cloud Maybe you can do that on the edge and you have the average temperature for the last five minutes When you got that value you put it on the cloud Okay, so with all those decisions you need to add this this tool that is the IoT edge to your Machine learning pipeline You are here is because probably you know this you've seen very a lot of similar slides during these two days What about the pipeline people are speaking about and even more data scientists some of them and this is arguable They need to use the software engineering Practices and one of the things that I want to add is in your data set and your data tools As a data scientist you need to add where do you want to deploy that you want to deploy that to the cloud on premises or to the edge So once you when you are taking a decision You need to think that now you have the power to do a flexible deployment and use all the algorithms that as I said We are putting there and create something that is not as complicated as an autonomous car But that you can use on the real day. Okay, once you have done that decision The next decision is what cloud to use. Okay, of course our recommendation is to use Azure Why because basically you have all the sophisticated Models already that you can use you have frameworks and you you can use any framework that you are comfortable with Then you can use tools like machine learning studio You can use data bricks or you can use a virtual machine when you want it when you need the the power when you need The process power you can choose between using CPUs GPUs FFGA and at the end as I said Where do you want to deploy that you need to choose where to deploy and these are all the options This is happening today and as the last slide as I said, we want to finish with a video So one of our clients is speaking about this. This is just in the internet connection. So Yeah, this is a net scenario where a manufacturing customer is leveraging vision machine learning at the vision to be able to Hopefully it's running to detect defects at the Electrical boards, which will have used the edge you'll have the case study but here the idea is that they they run in just in time a production and They want to reduce scrap material and inventory at the end. They don't want to have to stock So as soon as they find out that there is a problem in the production line they are able to to to fix it and and and work for better Deliver it with to the customer. So at the end this would be running vision with a process called project brainwave that's running FPGAs that are a primal Controllers it's hardware that it's running at the edge and that has a Performance from one to a thousand times the the normal GPUs or CPUs So you need to believe that that's true because the video is not loading So we we're going to give that part you'll help but that in the In the presentation afterwards And basically that's it. We're going to show just the last slide with the takeaways it's done and Some references if you want to read more about this we invite you to go to Azure IoT Edge website where you have a lot of information Okay This is what happens when you don't have everything on premise Okay It's downloading Well, the others is that references are material. So we'll put that line Hopefully It's a no cargo. Well any questions now we have 20 seconds. Thank you very much and so thank you for being here Thank you. Hopefully we will manage to explain what we are trying to say here It was a pleasure and if you want to ask something we are going to be here and also ask the expert The last the next 10 minutes. Okay. Thank you very much There's a question and there's a question here Microphone Hi, thank you for your presentation Just a set of questions All the examples you've mentioned are projects that has been run by you by Microsoft by I assume that Microsoft is not needed as Partner for the integration. I mean just as your components and Whatever the cost is and okay. It's not actually needed the involvement of Microsoft Yeah, it's not Needed we can we can help that we can engage with with you But it's you can run through our partners or the partner channel From us directly or just for well from the information that we have online and okay And the second question is I assume that the only Additional products from Azure that we need besides the one for machine learning for ingestion and the like are the IOT hub and the IOT SDK for the actual devices. Do we need anything else? you need to think about the Reading from the sensors sometimes usually you need some sort of implementation of the sensors and protocols Yeah, and It is right the the endpoint that we were mentioned is that I have in this case So it's at that point that would be fine for start running. Okay. Thanks. I want more thing the IOT Edge has an SDK that is open So the code is available in a GitHub and you can download it and start coding from from there So it's open. Okay