 Good morning. So okay, so today's topic is machine learning for IoT at the edge. I won't go by my introduction. It's already done Now this topic is a Is a connection of various other topics as you can see We'll cover the concepts of IoT edge computing cloud computing and then finally deploying machine learning with it and Training it as well as influencing it Big topic indeed. I'll try and wrap it up within 30 minutes Hope you'll grab something and then you can ask me questions later on as well. So starting from the very beginning We'll go by the definition of what's IoT most of us have heard the word. It's a very buzzword We all know right so interconnected things, but two things that needs That requires us to know is each of these things. We are also about like humans and animals We are also a part of IOT each of these things are uniquely identified has a unique IP address Has a unique IP address and is uniquely discoverable over the internet Right much like our homes. We have a mailing address That is exactly how we have an address and people can find us so similarly IoT the things the devices have their own addresses and They always need network connections and they require high availability, right? And since they're always connected to the internet and there is Continuous to and fro of data over the network privacy is also required the IOT life cycle it all starts with IOT devices the sensor connected to the devices connects collects data and Then we have the IOT infrastructure because without infrastructure nothing works, right? So we have the IOT infrastructure post that post the infrastructure We the data collected by the sensors are stored in the cloud or You can have on premise but in case of industry use cases You cannot store it on premise because it requires huge amount of computing resources if you can afford Prem on premise servers, then yes, definitely you can but otherwise Since the topic will be relating around edge and cloud So in this in this case will be talking of cloud computing once you store the data Data is not information data and information varies, right? So to get information from data You would need to Am I audible? Yeah, okay, fine. So to get information from data You would need some kind of processing to it analytical functions and all those processings, right? And then we have the clients which basically forms the connectivity circles Moving on the concept of edge computing So what is edge computing? Can anyone from the audience tell me? Okay, so edge computing is computing closer to the device or basically If possible on the device So if you can see the architecture at the very base of internet of things we have the we have the devices And then those devices are connected to the embedded CPUs, right? the central processing unit and Edge computing would enable those low-power CPUs to do the processes that actually can be done on the cloud Okay, and then beyond edge computing we have the cloud computing so Again as I was telling you what does IOT require since it's internet of things it requires continuous connectivity It requires high bandwidth. It requires zero latency information. That is information without any delay We need information then and there and also the data since it's over network. It needs to be private otherwise We are we smart watch taking the example of a smart watch if we are transmitting our own data We definitely don't want to want someone else to discover those data, right? So it needs to be private. What does edge computing provide us? It actually helps us enable all these character all these requirements for IOT So for when it comes to latency we have zero latency information Why because the the all the processing is done on the device or closer to the device? Bandwidth we don't require much bandwidth because we are not sending the data to the cloud in order to send it to the cloud We require bandwidth otherwise if it's on the device we can just do it without the internet connectivity Availability we don't need to be bothered about continuous availability because again It will be done on the device or close to the device for cloud computing When we when we send those information to the cloud we would need availability and privacy Well edge in edge computing what happens is you process most of the information and Then send only those information Which is essential to the cloud and reject the rest. So yes privacy is maintained that way use cases Continuous driving cars and driving cars. You would need this application Why because say if your car is moving on road and it encounters an obstacle then You you want the car to act immediately the car will stop But then it has to start on its own if it waits for the cloud connectivity or internet connectivity to come back Then it might just wait for another one hour or half an hour and if there is no internet then that's a problem The car might not start again, right? So that's one thing Predictive maintenance is another big use case for industrial IOT where we require immediate Informations on the device health and the remaining as output a remaining useful life of the device health So we need it then and there itself There are various others as well collision avoidance as I said for Self-driving cars or for different vehicles. We need immediate actions there and the rest you can see and there are a lot more Moving on Since we have talked about edge computing so much. Does it make the cloud irrelevant? No, it is always edge and cloud works together. How as I said edge computing does the necessary Necessary computation close it to the device and then it moves and then it pushes the rest of the information to the cloud Right cloud stores the history data so that we can use it later for our trainings of ML Machine learning algorithms, right? Okay, this is a typical high-level end-to-end process in this to Worked walk through the demo will be using AWS capabilities for both cloud as well as edge so in reference to that Below is the end-to-end high-level common general process So what happens is I told you based on the IOT life cycle first the sensor collects data and then With the help of infrastructure it pushes it to the cloud once it's on the cloud we process information then we deploy the required machine learning or deep learning model and then utilize a ML framework like tensor for play torch or everything and Then once it is trained and deployed we derive inference out of it and then we can also expand the capabilities using Any kind of accelerators such as GPU or that is already present in the cloud computing environment So to do that for the collect and collecting and moving data. We have IOT AWS IOT core For processing we have AWS SageMaker and the rest of the functions are done by green grass on the device, okay? so as In the previous talk you have heard about machine learning or deep learning in a typical business scenario This is the entire pipeline as I just showed in the previous slide So here you can see again in a business environment the first and foremost stage is collecting data Once you collect your data you process it you clean it clean those raw data All you need to figure out is what kind of problems are you solving based on that you choose the correct info you choose the correct ML algorithm for your problem statement and then once you decide that you set up to Set up the entire pipeline you need to decide on the infrastructure needs as well once your model is trained You see the accuracy if it is not as you require you need to tune your models Right and then again further deployment and if you want further if your data in keeps on increasing You need to scale and manage in the production environment as well Now as I told you we'll be using AWS capability. So it for cloud computing Training your model Okay, training your model has two phases. I mean especially with this topic It has two phases one is training your model and the second phase is inferencing from it So the training would be done using Amazon SageMaker. What does AWS SageMaker provide us? It provide us a Complete end-to-end cloud computing capabilities to manage and deploy your ML solutions So it also has pre-built notebooks. It has pre-built algorithms as well and it also gives you the flexibility to develop and build your algorithm and it also has in built Frameworks such as MXNet, PyTorch, TensorFlow and all these things and then once you are done with the training of your algorithm You deploy it using SageMaker as well and you can scale So again training as I said, it's one-click training if you are using AWS SageMaker on the cloud Then you can you can just it's just in it has an inbuilt Jupyter notebook And if you have used Anaconda on-premise, it will be almost similar and you just need to click a few couple of buttons and then get the job done and Then it's it also provides you one-click deployment again. Why am I emphasizing on this because once you Have an on-premise model. You would need to create the entire CICD pipeline for your deployment as well But when your when your focus is just data science or machine learning, you would need to you just need to focus on that I mean given rather than the infrastructure. So it again AWS SageMaker again gives you one-click deployment solutions as well And fully managed and hosting as well as it helps you auto scale as well If you you don't need to you know, be bothered about the servers running out of space And then okay as I was telling you what's machine learning on the edge You train a model on the cloud and you deploy it to the edge for inferencing So for that as I was telling you we have AWS Greengrass Which gives the capability of extending the cloud computing capabilities on a very resource constrained device Right your data is processed locally and then the process data is sent to the cloud What does extending intelligence to the cloud means again, you just need to sense that is done by the sensors You need to infer from the data, which means you need to perform analytical functions on the data And then based on the informations that you have received on the analytical functions. You need to act on it This is the typical high level end to end pipeline for extending intelligence to the edge This is what is happening to the edge and these are also the capabilities that AWS Greengrass provides So it provides it helps you perform local actions like your Raspberry Pi device can is capable of performing any kind of It's capable of triggering any kind of other functions as well Greengrass also lets you use lambda functions for triggering those things Like local actions and local triggers data and state sync as well. It helps you sync then and there If there is any update on the cloud model, it gets updated on the edge model as well through over the air updates Okay, then it takes care of your security as your entire data is not exposed to the network As I told you we have over there updates. We also have protocol supports It also gives you supports of protocols like MQTT and similar protocols Local resource access, which means within the AWS Greengrass, you need to create your local resources That gives you the components to access all the other modules that are connected to your Internet IoT devices as well as the ML models that you are using and finally local inference on the Raspberry Pi So this is exactly how AWS SageMaker and AWS Greengrass is helping the real Model of IoT, which is the cloud and edge based model work So AWS Greengrass core device as I was telling you on the other slide We helps you collect and manage the IoT device Settings, basically this controls the IoT application phase post that we have the Greengrass We have the Greengrass which extends the cloud capabilities in between and utilizes and helps the IoT core perform and get actions And in the cloud computing capabilities here you have a model and your built trained model Before we move on to the demo You should also know that there are other various deep learning architectures that is specifically built for low resource constraint networks One is MobileNet, there are three versions of it, the archive link is there You have exception architectures as well and there's another thing called a squeeze net But mostly these are all used for object detections But nonetheless the archive link is there, you can go and see it So demo, I can't show you on working demo because you can't see it But then I have the demo screenshots here and that will help you understand how it is done So first and foremost, it all starts with the device So I had a Raspberry Pi device, any of the sensors can be connected to it So I need to configure Raspberry Pi device runs on a Raspbian OS Okay, so once you configure your Raspbian OS You can on top of the Raspbian OS You need to configure the AWS IoT greengrass core And once there are a couple of steps You're basically building the infrastructure on your Raspberry Pi for model inferencing And then once it's done, these are the commands to start your greengrass settings as well as As well as you can also test your greengrass Whether it's running on Raspberry Pi or not with this command Once that's done, once I have the IoT core on the Raspberry Pi running I had to do the model packaging So there are various pre-trained libraries provided by AWS As I was using AWS capabilities that are already present Which is Amazon SageMaker Neo, MXNet, then there is TensorFlow as well And Chainer So these libraries, these pre-trained models you can get to download it from the AWS dock site Once you download it, you need to push it to your Raspberry Pi device via any kind of WinSCP or SFTP protocols And then once it's on the device, you need to install it Install those pre-trained libraries on the device post your device is configured with IoT core Once the framework is installed, all you will have is a function handler That will be used later when you configure the lambda functions inside greengrass Okay, so after installing the framework, all you need to do is create a package that pushes the model to the greengrass settings And then that package will help you further to go down the road with the model training and inferencing So this is a picture of the greengrass console with an AWS console And this is a resource group that I have created for making the demo So on that again, what is a group? A group is A set of configured settings that actually helps you access other resources as well Okay, so once you create the group, all you need to do is create and configure lambda functions How again within the AWS greengrass console, you have the You have the function to create You have the sorry, you have the tab to create lambda functions And this is exactly how your screen will look when you create a lambda functions Your lambda functions can be one your lambda functions can be many within the group You can you can use your lambda functions to trigger any other activity by the devices Or you can also utilize just to do the training and inference Then once your lambda function is configured, you need to publish it and post publications You need to configure and publish it and post that all you need to do is go to the resource group within the AWS console And create your resource for for this demo purpose. I created two local Resources one is for the modules the sensor modules connected to the Raspberry Pi and other is for the ml ml training ml models Once that's done, I went to Amazon. I used Amazon SageMaker to create my training model And this is just a snippet of the Jupyter notebook that I was using So inside SageMaker, I have this SageMaker IoT edge instances notebook and on that I have trained my model And then post that I have deployed my training model using the greengrass group that I had showed you earlier And this is the Below you can see the deploy button. So now once you deploy it You can Okay, Amazon SageMaker also provides you with two options. You can train your model locally on your computer zip it and push it to the S3 bucket or You can use the SageMaker train model for me. I have used the SageMaker train model once that's done Once that's done and once you deploy it your model your trained model is readily available on the raspberry pi device that I had Inbuilt with a greengrass core And then if I have to test it again, the greengrass core console has an option to test the model and here Once you publish the and subscribe the topic using mqtt because you would need some kind of connectivity between sensors and your device and then because your sensor streams real-time data and You can use mqtt or even you can use the i2c protocols and everything and this is how Once you test it, you will see the results coming out in this console And that's that's the basically gist of the entire topic which is ml on iot edge the reason behind this topic is In a typical industrial scenario. It is very difficult to tackle and handle iot data and Hence the ROI deriving ROI from it suffers cloud computing and edge computing helps each other to make this happen and That is exactly what I showed you how edge and cloud Computing using aws capabilities happened That's it. Thank you Any questions? There's a question Hello Yeah, first of all, thank you very much for this amazing talk. Can you hear me? Yeah, can you hear me now? Yeah, okay. First of all, thank you very much for this amazing talk That was really informative My question is The all the iot on the edge which talked about is on an OS based system. That's a raspberry pi I need to know what's the state of iot on the edge for The device that is not running an OS like bare metal microcontrollers Okay, so you can also inbuilt various other deep learning architectures that I just showed you or maybe eventually even if you have to run Microcontroller without any OS you would still need to have an infrastructure built on top of it, right to even connect with the sensors so you would Why did I show you a raspian because Once I'm doing it on a business environment I would prefer any of the infrastructure that supports that has interoperability If you are trying to build it from scratch on your own Then you would need to take care of the interoperability and the infrastructure needs as well So you would need to design your own infrastructure as well Any any further questions Yeah, hi hi Like what are the use cases for iot on the iot and edge computing right default computing. What is the difference like? Uh, I mean what are the security measures we have to take care if you are going for iot or other things See, okay, so for iot there are various security standards as well Okay, but the that's beyond the topic of today. I can have an offline discussion with you There are various security standards from owsp as well, but Consider when you asked about use case consider a scenario in a healthcare environment where you would where a patient is Being continuously streamed and monitored and the data is being sent to a server, right? Now the server is capable of flaws and vulnerabilities and the data can get exposed A hospital cannot afford to do that. I mean They cannot expose. I mean They cannot afford to as in from a business perspective. They lose a lot and the name of the Organization also goes down. So when you're doing something on the edge, I'm not saying you eliminate cloud but When you're doing something on the edge out of the 100 data, the uh, you are processing the data, right? You are performing analytical functions and out of that you say have 60 percent meaningful Informations and the rest are redundant data So you send those meaningful informations in an encrypted form to the cloud and store it Which can be used further when you push when you push one ml model from cloud to the uh Edge you would you reiterate in in a typical machine learning scenarios Well, you reiterate the models right based on the data incoming data So once you push it to the uh edge And edge is also giving you new sets of data That is getting stored to the cloud and the model is being reiterated with new sets of data And the if there is any further Upgradation on the model then that upgraded model is sent to the edge With over there update. It's much like you do it for a software update over there Okay, yeah Thanks for the talk Sukanya. So, uh, my question is like you're told there are some struggles related to iot based Data gathering and security measures. Can you throw some light on that? Like what are the real struggles that this industry is facing? Okay, fine. So, uh, in uh industry Okay, so when it comes to normal data and iot data the first and foremost problem is it's time series in real time Right and If you have worked on No, even batch Kind of data You still have something stored But in a real-time data you need to take actions immediately otherwise debugging later on becomes difficult right and When it comes to again time series it it can also be batch data Like you can cache a portion of your data and try and stream it At a regular interval But again iot if the connectivity Data is not only related to the thing the data that is collected by sensors Sensors would also have to have the connectivity to stream it Right, so basis this Problems edge computing is something that comes into the picture that actually helps you Do these things closer immediately when the sensor gathers data consider us as human beings. We have sense organs. We We gather data right and and our brain processes it immediately Then only we take immediate actions Right, so similarly iot in a say remote area say in a disaster prone area where You want to know the predictive case of a disaster or a natural calamities You would need to have the informations immediately so that you can alert the People right you can't wait for a cloud connectivity over a week and then Let the disaster happen. So that's the thing Okay, all right. We've run out of time. Thank you. Sukanya for the talk. I can take questions offline. Yeah. Yeah