 Hey, hey, how are you? I'm doing good. How about you doing doing great? So thanks Thanks a lot, Ajit for for accepting to speak on this on this event and guys Let me let me welcome Ajit Ajit is a senior technical marketing manager developer evangelist Docker captain and you know what? Arm in a waiter you can just you can just add whatever you want to add because he is everywhere He's everywhere on social media. He's running collab mix a great great community platform I am also there by the way if you guys begin to share some some links to on the chat But yeah, great great community. He is he's speaking at a lot of various events within India outside India as well So so Ajit what are you going to talk about today? It's about a Cloud native computer vision applications using Redis. What else we have? Yes So the talk is going to be around IOT artificial intelligence and cloud native three topics fantastic Fantastic as you know, all right. The floor is all yours. So so walk us through with your amazing talk Yeah, sure Yes, your screen is live Yeah, share my screen Okay, thank you so much. Can you see my screen? Okay Okay, fine, let me go ahead. Yeah, so thank you current and the team from divination. I think You guys have been doing great inviting a lot of speakers I have been going through all the sessions and I find it really interesting So today I am going to talk about building end-to-end cloud native computer vision applications at edge using Redis because I have been Doing a lot of stuff around IOT cloud native and artificial intelligence and I thought that I will go ahead and share it with you guys So let's get started a little about myself I am author at collabniks.com collabniks is my personal blogging site with around 300 plus blogs Which I publish almost every week on Docker Kubernetes cloud native and as well as an IOT So if you are really passionate about learning about Containerization, this is the place to be I'm also a Docker captain since past close to six years now I started using Docker since the time it was born So so you can just imagine that all those 300 blogs are basically they talk about all the new features Which has been introduced in the Docker ecosystem. I'm also Leading Docker Bangalore community group which has around 10,600 plus members We do meetups regularly. So if you are really passionate about speaking in any of these meetups, do let do let me know I'm also an arm innovator. It has been the last one year. I have been being part of the arm community and I was also a speaker in the last arm dev summit which happened in the month of October where I talked about deep streaming and Docker and IOT right and and talking about my professional Especially career. I have worked in the companies like Dell VMware and CGI in the past and right now I am working as a senior technical growth marketing manager at Redis in Redis I have been handling a developer community where I go ahead and maintain Developer Hub site, which you can also visit www.developer.redis.com and If you are really interested to be part of the Slack channel so launchpass Com slash collaboration is the link which you can click and you can get connected to around 6,200 plus DevOps engineer Okay, so as I promised This is going to be an exciting talk. Why because you see all of these things in the in your screen We are going to play out with drone We are going to play out with NVIDIA deep stream Which is again an open source tool and then IOT edge devices like Jetson and how all of these Dots get connected to build a kind of a powerful solution So for anyone who is more interested in any of this technology I know that there has been a lot of questions in the past saying like how IOT edge can help You know in artificial intelligence as well as how it connects to the cloud, right? So this is a talk You you should not miss about Now let's talk about each of these technologies. The first I will go through the artificial intelligence then touch upon IOT edge And then we'll show you how it can be integrated with the cloud native technology So let's talk about the rise of artificial intelligence. Now if you look at AI, right? Everyone is talking about AI today if you talk about especially there has been an explosion of solutions right there in the market which Which which which basically talks about the deep learning machine learning and object detection and analytics So think of it is like in a smart infrastructure where you you know You are able to track especially the traffic based on that you are able to route your The you know the vehicles in the right direction, right? The other way round isn't that retail if you visit a kind of a hospital's airport You will see a small robotic platforms Which is helping you in guiding in all the queries which you have related to the infrastructure talk about the healthcare how you do Crowd face mask detection system. So AI is being used there Especially about the robotics in data warehousing Think of it is like a video conferencing when you are in the zoom channel and then you are speaking at the same time There are there has been a kind some kind of a box which are running in the form of transcript which is captured, right? Those are in one or more like an artificial intelligence, right in the call centers If you see there are automated Automated voice recognition based on your preference, right? Those are being handled So in one or many ways, there are some kind of a deep neural network, which is being used across the industry and If you think of like if you if you are an end user You don't really have to care about it But if you are really building such kind of a solution say AI workflow This is how the typical AI workflow look like you have some kind of a pre-trained model library So right now there are a lot of them I would say like object detection face mask detection You have a can image classification You might know about AWS recognition system. That is one way of how you basically recognize the face You do the face recognition. You have a pose estimation and all of these things, right? These are basically being powered by few of the pre-trained models which are available Right there in the market, right? And usually what we do we have few of the optimization tools Which we called as in the form of a train adapt and optimize So what we do basically we optimize it so that we can get results Now think of it is like if you are doing some kind of a face mask detection system There is a lot of data which is coming out from those IoT sensor device Now you need to get a result out of it. You have to optimize it so that you have all those Data which you can play around with and then that can be basically That can take place in terms of analytics. How you plot it over the Grafana How it how you How you count a number of people who are wearing the mask can be one of the thing and then how you detect the object So one thing which I wanted to show you is like I have seen in a few of the factories like like the water border factories if How they are able to detect whether the bottle is Really broken or it is not in the proper shape So that could be one of the example where you do an object detection and try to find out the data And at the end of the day, you have a complete report saying like how many how many of these objects are are not in a proper shape And how many you are counting the number of people who are entering them all right So these are the few of the examples where you have a valid data then Now the challenges with these kind of an AI streaming application is definitely the design because you might have seen Face detection is entirely different from the other type of object detection You need to have a model which has to be defined think of it It's like you have an IOT device with a camera module connected in your parking area And it is able to track whether the vehicle is a car or is a registered car or not right So those kind of each and every solution has to be designed in a different way There has to be a different model and that is that is where you're all the design complexity few of the solution are More based on tensor flow. We are using the you know Pi torch framework and all the so we have a multitude of these frameworks and Which comes again with their own set of programming languages, right today You can build using Java Python and all those AI based programming languages like you can use it in the second way We are also talking about the cost or think of it is that you have an IOT edge device You are in the and you with the camera devices and you are able to track all those devices, right? There is always a total cost of ownership which comes into the picture. It has to be accuracy accurate Definitely, there is some kind of performance benchmarking is involved and those tools has to be purchased to run those benchmarking, right? And the other way around is you have a multitude of these IOT edge devices You might be deploying it on your Raspberry Pi You might be running over the cloud you might be say Amazon recognition system This is like first 5000 APIs are free But after that what you need to pay for those APIs and each API API is cost you, right? If you are running on the edge devices, it also depends upon the cost When you talk about the Raspberry Pi which cost around say Close to around 30 to 40 dollars and then till the point you have a Jetson Xavier kind of involve Environment which which might cost you around 500 bucks 500 dollar bucks, right? So those kind of devices are still available, but We you have a more complex software stack sitting on more complex hardware stack and that is the challenge today Now if you look at the implementation side, this is this is basically the overall Graph architecture how it look like say you have a sensor device You have a sensors which is collecting the data and then you try to capture it decode it Do a some kind of a preprocessing on top of it and then you do the inferencing part where at this is the point where data itself You you make the machine learn things right and then after that you do some kind of a post processing stuff now right to AI inferencing whatever you see like a tracking business rules and all this thing it again It again needs some kind of a GPU capabilities where you do some kind of analytics and to gain insight, right? Right now we talked about AI now Let's jump into the edge computing and that is an important part because the AI analytics is Something which you run on the edge device. So how it is going to look like. So let's get started Now one of the things which I am really Excited and you know wanted to share because I have been working in this platform last two years Ever since the Jetson Nano has come into the picture and the reason why I'm talking about Jetson Nano because it comes with AI in build The other than that it also supports all those cloud cloud native applications So think of it as like a docker docker is by default Installed so when you flash nesty card on raspberry pi you have to install docker, right? But in this platform docker comes by default and that too with the latest version of Docker, right? So so If you look at the features The moment when I say AI computer then you might have it might strike your mind that what kind of an AI analytics it does One of the beauty of this product it it comes with a GPU in build So you can run all those AI based application on top directly on top of edge You don't really have to go ahead and buy a server try to run a full-fledged application buy a GPU So it is it already has a GPU in it. Look at the configuration. This is now a very minimal I would say Small form of Jetson Nano, which is like a 2gb which cost around around $59 And it comes with say 2gb of ram gigabit You know ethernet and look at the gpu like 128 core nvidia maxwell gpu it what it comes So I have been so if you really want to Try out few of the application if you go to colab mix.com you will see like there are number of Examples which I mentioned where I have talked about the face much detection object detection and all of this thing And that primarily uses the docker container and today Docker supports gpu so you can run container and that container will be able to leverage the gpu device Okay, you can also specify the number of gpus. You can also Go ahead and try to use application if you really if you if you really want to go ahead and Specify a share the gpu share that is also possible today Now if you look at the Jetson one of the thing I think the nvidia has done a really great job here is Looking at the cost point because it comes with 2gb and 4gb But the other thing is nvidia provides you with a kind of a free open source Kind of a stack or on top of it think of it is like a now This is this is how the software stack look like you have a Jetson on top of it You have some kind of kuda driver which you have to install it and kuda is basically your interface where your Operating system talks directly to the gpu right and on top of that you have a jetpack sdk This is completely free. Okay. You just have you just when you install an sd card when you flash an sd card on Jetson nano these are the things which comes by default And these things are really important if you see the deep learning multimedia isolated computing all of these things you get it just On top of your sd card and this sd card will be minimal like 16 gb to 128 gb Uh, it supports more than that But we have a support matrix which talks about it and on top of that what you do basically you go ahead and deploy your Uh, sdk is like deep stream sdk is which is based on g streamer and it helps you to To have an end-to-end ai streaming application And that is being used heavily in all those robotic platform today if you see the most of the food delivery robotic thing robotic Devices which is being used they heavily use jetson right? So this is how the jetson software look like so what we don't really have to go ahead and start something Start compiling something from From the scratch if you have ever installed the open cv on raspberry pi or any of the jetson device You will you will find that it is so tough right the thing is you really have to You know take care of the python libraries and all this thing but with jetson and the deep stream you have everything in place Now the most exciting talk on especially the same section of this talk is especially the dji tello drone now Now i'm not here to market this product But this is the first product which i came across which basically supports python And that is that is something which is really exciting with just around like 99 dollar you can get An indoor quadcopter quadcopter is more like an unmanned helicopter right? So it has around uh intel 14 core processor. It has a five megapixel camera ATG in weight so it's very lightweight drone and uh the main Motivation of having this drone is having coming up with more use cases where You make a tello talk to the these kind of an iot edge devices and build a powerful solution The tello drone is uh today is available in india. You can always go ahead and buy it and uh the The beauty of this product is it supports both the scratch and the python programming It means i'm sitting on my laptop. I can go ahead control this device There are different flight modes which which i can use it and and look at the maximum flight Distance of 100 meters, right? So this is this is this is a really good For basically an indoor testing and that is that is the reason i started, you know Because most of the drones which you see those are not non programmable, right? So not every drones are programmable But this is the first drone i came across and then i had a one other thought in my mind where i can go ahead and test few of The object detection and i'm going to talk about it how i built it from the scratch And uh, let's not forget that it supports wi-fi which means that you just have to You know power on the moment you power on this tello drone It basically the wi-fi hotspots was created and your laptop can you connect it to that Now one of the last thing which i want to talk about is how this edge devices and how it integrates To the cloud in to the cloud and this is where you see here the cloud native deep stream application Now deep stream as i said, right? It basically provides you with the end to end ai Edge streaming ai streaming application and that is where the moment you have a sensor data, right? Or you have a you have a data coming from the video frames of each of these camera models come to it You do a lot of a bit of pre-processing inferencing and iot messaging and now These devices has Has a very tight integration with the cloud So think of it is like an atl s iot green grass It um, you might have used some kind of data dog You know plugins to basically send this data from right from the iot to To the cloud, right? So today the integration with the cloud can happen in multiple ways One of the example which i am really interested about is basically the redis now one of the thing which If you have attended my earlier talk, you could see like how the sensor data Collected by the jetson nano has been pushed to the redis enterprise cloud As in the form of the redis time series modules like we have a redis time series model at the cloud And then these data is fetched through the python script and send it to the To the remote redis database now today they have been support of multiple message broker library And i'm also going to talk about deepstream 6.0, which is the latest release from nvidia which talks about Again and free software which you can use it and its support redis for the first time Which means that Whatever the data or the video frames which you are getting from these iot devices can be pushed directly to those Databases it supports kafka mktut and then you have a green grass All of these have a very tight integration with the cloud and the reason why we are sending to the cloud is based on Because of the analytics when we want to plot it over the grafana. We want to we want to have and cloud analytics because that is the 24 path 7 which you can run it right even though these small devices have very limited in terms of size and shape Because the sd card maximum it can go to 5gb not more than that But at the same time what you can do here is send those data to the cloud and and use the cloud tools basically the analytics tool to run some kind of Analytics on top of it Now this is the other picture which talks about because this talk is more on the redis side I am going to focus on. Okay. How redis fits into the picture, right? So the moment you have a video frames coming from the camera It is captured decode and and then you know the ai streaming is handled But what are the things which happens is these datas are coming in a real time, right? It has to be it has to be feeded somewhere It has to be saved somewhere now your sd card is not going to save the data Which is coming say for months and months of time, right? It has to be stored somewhere And this data has is a timestamp like every second the value changes think of it is like a sensitive device Or think of it is like you are doing some kind of a face mask detection And this data has to be has to be feed into the cloud, right? And that is where your redis pops up and redis time series coming to the picture now redis time series is basically a database It is basically a redis modules which stores the data in terms of a timestamp And that can be fitted into the Grafana to showcase the entire Results thing like how many people are have been captured by the camera or how what the if you do object detection It it also tracks the objects, right? And so the python application is it could be running in the cloud or it couldn't be running on on-prem But the only thing is it has to be sent to the cloud and definitely your The other type of databases which you can use is like if you are even for the primary database you can Look at the redis But again, you can always use other kind of a databases also to store those applications Now Coming to one of the most popular use case, right? So this is this is a part of I don't I don't have a full-fledged demo because it requires infrastructure But I can go through the steps which I went through Yeah, definitely. I'm looking at the time here. But so let's let's let's talk about one of the use case which I picked up This is very powerful use case. It's already been implemented by a lot number of people But what I was excited about is How it can be implemented using the deepstream 6.0 which supports redis So most of the solutions which are available over the market or in the github which you see It's more like with deepstream 5.1 when redis was ready support was not introduced so this is a kind of A kind of a demonstration which I want to do especially the wildfire detection Which is you know the drone flying over the forest Or especially, you know is able to track whether there is a fire or not and based on that It should be able to send the message to to the nearby Location or it could be you know For my demo purpose, I use the jetson nano and then after that it is being tracked over any of the analytics tools So let's get started here So the hardware requirement is going to be quite simple because I already talked about it We have dji tello drone which cost around 99 dollar Jetson nano again, you can use 2 gb 4 gb 4 gb is recommended because the deepstream takes because even the sd card takes around 13 gb of 13 gb of space The recommendation is at least have a size of 128 gb in your sd card And then we have an sd card which might cost you around like 20 dollar These jetson nano doesn't come with a wi-fi module. So you need to buy a wi-fi module One of them is like it comes around 25 dollars. So So moreover, it is it will be It will be close to around 250 to 300 dollar If you are just going with the 2 gb of jetson nano and then you need a 5 volt 4 ampere power supply and the deepstream 6.0 Which is like freely downloadable from the nvidia site And the whole solution which we are looking out is something look like this. You have a tello drone You know with the wi-fi module enabled basically it Supports it it basically streams the data in the udp. Whereas you have an nvidia jetson With a deepstream is able to capture You know through the rtsp and then after that the data is being you know pushed into the redis database for further analytics So the step one the first thing is you need to run a deepstream container So the moment you flash an sd card on your jetson nano device Docker is already there. You just have to go ahead and pull and now this is image is not available in docker hub It is available on ngc, which is nvidia google cloud repository sorry nvidia cloud Repository where you can go ahead and you know pull this particular Image this image has all the runtime libraries. It's a bigger is a gig gig in size G steamer plugin on based because that is a primarily platform which the deepstream is based on You have a models. You have a configs. You have a sample application which you can Which is already there in the deepstream and that's why it's more fatty The second step is going to be setting up the tello drone and I can show you quickly an example here Which with this very simple code. I am able to fly You know, it will basically take off and land So you can also use the move left rotate and move forward because I was just like testing it in don And the way you see it is very uh, it went up Stared in the air for around like Less than one minute and then landed right? So this is a simple code which you can run directly on your Jetson or on your laptop and test it Now the moment it is done then most probably the the third thing is you need to able to stream the video, right? So you take the picture you stream the video and send it to your jetson device and this is the code Which basically does some kind of an object detection It will detect it and then it will try to Write it to some kind of a file, right? And this is just for a test But in our in in our testing what we did is especially the real-time streaming It's not like saving in a file, but it is more like a real-time streaming The fourth step is will be like cloning this repo. Once you clone this repo the Uh, it will basically utilize the live stream of the camera again We need it as some kind of udp to rtsp, which is a real-time Streaming protocol so those kind of a conversion and you will see tello control.py is going to take care of that And this script will basically start your tello stream on the following url So what we have done here is run some kind of a video directly on top of your laptop because this was an indoor testing which we wanted to do and Definitely the deep stream today Basically uses some kind of a yolo v3 model detection file detection model Which we call as the deep stream Hermes and that will basically detect the fire and it is going to Show on the screen saying like, okay, this is the other places and the way how it is going to do is basically It is going to count the location of where the fire is happening fire one fire two and all this thing and And at the end you need to push this data to the redis and that is where this kind of You need to go ahead and write this kind of a python script where you are importing cv2 But at the same time you are using the redis queue Providing the redis ip address port and the db name Okay, so by default if you don't you have if you haven't created a database it basically takes db zero But definitely if you are using redis enterprise cloud, you have a database name You have a module you have if you want to really go ahead and integrate with the modules So by default these are the modules which can be installed and if you are using Some kind of say if you are using deep stream 6.0 You don't really have to do all these things you just have to do as some kind of a configuration inside it because it already supports that But if you are using 5.1 then most probably you have to install redis server Separately, so that's the beauty of deep stream 6.0. So i'm planning to Go ahead and write a blog on top of it. But but definitely this is one of the use case Which you know which strike my mind and I thought that I will go ahead and try to do it But this is primarily a kind of a simple python script which you can write it down Now coming to the last of the session These are the future works which i am planning to do I have been sharing about it You know in the social media because I want people to get excited and then try to contribute But but these are the these are the things which I have been planned like the first thing is integration with the grafana So I want to do the grafana integration in terms of counting how many how many number of fires The second thing is I don't want to restrict restrict myself with the one Teller drone I want to have a multiple input sources and try to test it So i'm planning to buy a few bunch of them and then pushing these Times time data to time series so that because today the grafana supports that It would be great if we we can go ahead and you know try to do Try to test some kind of a solution So I think i'm done with this Over to you karan if you have any question or I am on time Hey, thanks a lot ajit. Thanks a lot for for for this great presentation So we were already in the chat. We were already shipping shipping pizzas And finding new startup ideas based on drones. So so thanks for you know thought joking presentation I would say Yeah, thanks. Thanks I'm looking at more from the contribution side Because the reason why i'm sharing on the social media and about what are the things which I have been doing These things are more exciting, right? I know that people there are There are a lot of talks which happens around the implementation cloud native But I think this is one example one good example Which we can see the power of the cloud native application how we are fetching the data from the iot and sending it to the cloud for analytics Yes, definitely definitely and and you know, uh So so so drone is also kind of a think about this is kind of an edge, right? Where you are running some compute at the edge and doing you need some some processing power You are limited with respect to the form factor. You can't do a lot of I mean you can't do more more ram and more cpu those kind of stuff So you need you need things like like like, you know containers running on Maybe on a drone chip or maybe on a on on jetson that you are showing or raspberry pi For that sake and running running red is to store some some memory And then once they are in they are in contact back to the back to the central location or the hub They can sync in the data get the new data information exchange information things like that So this is definitely going to be a very interesting Interesting topic for another five to ten years to come to come down and let's hope once we have You know medicines or pizzas or food food delivery Being done by by drones. I guess. Yeah, I mean at sun at sun is would be At sun is still if you're still I don't know if if amazon is shipping something Or maybe some other startup is shipping something in in the us with drones Is it is it real thing there? Uh, well, not in North Carolina. I've heard that they're trying to deliver something in Seattle, but Um, not here at least not here. Okay. So yeah, we should we should actually ask I should ask our friends in in in the Seattle area We are if they're doing something with respect to this but yeah interesting technology and and we can we can see that You know edges is is meeting with with compute and and technologies like dockers and Kubernetes at the edge And you know reddish reddish cash is at the edge and you know 5g coming up So, you know the real the infrastructure and and the software and the services are there It's it's just about you know permissions permission from the government Who can who can give us a green flag on this? So so fantastic. Uh, Ajit, uh, that's a great presentation