 Hey, good afternoon everyone. My name is Gavin Adams. I'm a Specialist Solutions Architect with Amazon Web Services and I'll be presenting deep learnings at the edge. I hope everybody is staying safe out there and welcome to the Embedded Linux Conference in a virtual environment. So a couple things here is just going to go over the agenda of what we're going to be talking about today. So there's a lot of interest around edge computing or edge capabilities, which is taking, you know, what we normally see is either data center-centric or cloud-centric and taking it out to the edge. So we're going to go through some of the different use cases for why you might want to use the edge. We'll then go through some of the implementation patterns. What we have seen with our customers and just, you know, through user experience, how these things actually can get deployed, some things that are good and some things that are bad so that we can sort of highlight those. Next we'll go through, you know, what happens when things go wrong. These are devices that are out in the field. So unlike, you know, in a data center or in a controlled environment, there's a lot of things that can go wrong, but there's a lot of ways that we can also address that through edge computing capabilities. And then finally we'll kind of just, you know, pull it all back and say that, you know, if you want to go down and start doing edge, how can you leverage what, you know, existing practices you have in your IT or your OT organization and go ahead and do that. A couple things is that we're going to be taking questions at the end, as mentioned, and we'll do this through the chat mechanism. If we don't get all of the questions answered, we do have the AWS booth that you can go to in the Gold Hall and ask other solutions architects. But having said that, we've got a lot to go through, so let's go ahead and get started. So before we actually jump into our edge cases, this one will go a little bit over architecture terminology. As a solutions architect in the IoT space at AWS, I use the word cloud a lot because that's what we work with, you know, cloud cloud cloud. But when I talk about cloud as sort of a holistic view is, you know, consider this to be, you know, either your cloud platform you're using, or if you're not using cloud services, you know, the data center, your on-prem or on-premises environment, wherever you've got sort of, you know, central control, and I've got services that are available 24-7. When I talk edge, I'm basically talking about discrete devices that are at the edge. This can be one or many devices, but it'll be basically a combination of both, you know, hardware and software and then multiple ones, you know, out there. The reason we like to sort of, you know, break these up is that when you talk about cloud, that's where essentially you've got a lot of your elastic capability, where you can have services on demand, and that when you want to basically scale out or scale back in, you can do so in a very fast fashion. When we talk about sort of the edge component, and I'll talk about sort of the hardware and software, essentially it's a combination of hardware along with software that you bring, an operating system, system packages, and then your own components that make up an edge device. This can be either a physical device, or in some instances it could actually be a virtual type of environment. So it could be right on top of, you know, VMware or Hyper-V. So let's talk about the edge a little bit. So I mentioned it, there's a lot of interest into it, and, you know, if we take a look back, you know, what edge is, is, you know, you can see for the term is sort of an IoT, and the internet of things essentially comes from, I've got some type of device that's doing something, you know, in the real world at the edge. The way we like to kind of, you know, say that if you can address one or more of what we call the laws of the edge, is that it might be a good candidate for how you want to do edge computing, or you deep learning at the edge. First one, laws of physics, is that, you know, the speed of light, you know, basically it's not just a good idea, but, you know, it is a law. And that if you've got things that you want to do, such as that, you know, a closed loop operation such as, you know, automatic braking if you detect, you know, an object or a person in front of you, you want those to happen very quickly. You want those to happen in microseconds or even single-digit millisecond latency. And if you had to take that data from a local device and send it out to the cloud, you're not going to be to achieve that. So that's a good example of essentially, you know, of how we can say that, you know, there's operations that should take place locally. That's a good candidate. The second one is around economics. As devices are now starting to go out into the environment, they're leaving the hardwired ethernet and Wi-Fi environments to sell your, or even to like Laurel WAN or other types of low-power WAN environments, if we now have network connectivity that can basically cover the globe, even from satellite, you know, communication, but it comes at a cost. It's either that you're going to have a significantly reduced amount of bandwidth. You're going from, you know, megabytes per second or megabits per second to potentially bytes per second, or that the cost of transfer data is going to be significantly more expensive. Transmitting one gigabyte of data on your broadband provider's network is going to be a lot less cheap than a one gigabyte across, you know, an LTE or a 5G-type connection. And then finally, it's kind of the law of the land, is that there are certain situations where you may want to have processing and there's, you know, values for it, but from a regulatory or sovereignty laws that there's limitations on what you could actually send out. A good example is that if you've got edge computing inside of a hospital environment, it might be very good to do some of those closed-loop operations inside of an operating room or a patient's room, but for, you know, personal identifiable information or healthcare reasons, that information should never leave the environment. So a couple of examples we'll go through here. Common one from an edge computer is sort of like, you know, a wind turbine farm. We've got a lot of very extensive devices that are close proximity to each other, but the farm might not be in close proximity to the rest of the world, meaning that they can communicate between each other but requires a cellular connection outbound. We want to do some operations on this. We want to do predictive or preventative maintenance on each of our turbines, which requires, you know, high velocity checking of vibrations, for instance, and that we want to essentially emit, you know, small amount of data to the cloud to say, hey, come out to farm one turbine number 22 for maintenance perspective. Comparing that to, like, the automobile of today, the modern automotive environment. Inside of an automobile, you probably have got multiple computers all communicating with each other, and these can be doing things such as the infotainment system, the ability to, you know, adjust temperature and controls, but it also might be things such as, you know, advanced driver, you know, controls through ADAS or things like this automate, you know, like I mentioned with, you know, auto braking if there's people in front of you. Keeping in here is that, you know, you've got a variety of different components, and you can actually include, you know, third-party ones that need to communicate with each other, and they differ. When a car is in a garage at home, you have got a good Wi-Fi connection, but when it's out in the street, you're now going back down to, like, to a cellular or LTE type of connection. And then the final kind of, you know, third example here is like site inspection. Very common where you might have a drone that you drive out to a remote site. This could be a refinery or an oil well or even, you know, a large farm environment. You program the drone to essentially do its autonomous operation. It takes off, it zooms up to altitude, it's checking things for methane, you know, methane discharges or blight on fields, collecting a lot of very, you know, intense data, you know, full-motion video plus LiDAR and other types of things in an autonomous manner and then comes back to you. When it's actually in that sort of autonomous mode, that's where Edge, you know, Compute of the Edge can do some differentiation. You know, you might determine, oh, I see a particular thing that we're not going to collect, but because I've inferred this, I'm not going to collect this. So if we kind of categorize these three things together, the turbine, the automobile, and a drone site inspection against the three laws, is we can see that there's certain values that, or certain things that come into each of those that match up. So the wind turbine is that from the physics perspective, I need to be able to do thousands of monitoring, you know, measurements per second on vibrations. For the automobile, from the economic perspective, I want to capture a lot of video to help train my autonomous driving component, but I don't want to transmit that when it's on the road cellularly. I want to transmit that when I'm back into a Wi-Fi environment. And for the drone, is that there might be a corporate security policy that, you know, any data collected by a drone itself can not be transmitted across the public internet, has to go through a private connection. So when you think of your use case, think of how you can kind of tie, you know, a potential use case into three laws. If you hit one of these, okay, if you hit two or three, that's normally a very good candidate for an edge component. So speaking of that, you know, what does an edge implementation actually look like? So I'm going to be talking about this slide a lot. And essentially what we're saying is that, you know, starting from the bottom up, we've got hardware components, we've got an operating system, system packages, and then our user packages, which are sort of the orchestrated components that come back and forth from the cloud, that then add your business logical capabilities to whatever the device is. Starting with, you know, the hardware is we may need to interact with other things such as GPUs, for machine learning, inference at the edge. You might want to be a user connecting to fix your sensors. You might have different types of architectures that you're running here. But this is your common hardware that you might want to expose all the way up into your edge application. From an operating system perspective, we normally see this in the three different environments. A lot of times, customers will start with sort of a general purpose operating system. You know, they use a vune to a red hat internally. Or they're doing development on a Raspberry Pi. So the Raspberry Pi OS becomes sort of a, you know, the use case. These are kind of general purpose. You're kind of, you know, manually doing updates. They're not really in sync with each other. The other one is sort of the embedded space from Linux is that you can use things like the Yachto or open embedded to build, you know, very specific and repeatable or immutable type of operating systems that can run on your particular devices. We also see customers are using more lean or thin operating systems such as, you know, Alpine or Busybox. Or even cases where you're going to sort of abstract out the entire operating system environment and use third parties such as Belina using the Belina OS as an underlying component for then deploying your edge capabilities. So moving up from the operating system into the system packages, there'll be a variety of these things. I won't go through, you know, much detail, but just to call out is that, you know, if you're running Python applications or Java apps, you're going to have to have a Java runtime or a Python interpreter for that. If it's compiled applications, you're going to have to have shared objects potentially or certain versions of G-Lib C. Or you want to take advantage of, you know, very performant things such as, you know, NumPy that's been, you know, written with, you know, underline C runtimes that are then available to other languages. For the machine learning and the deep convoluted neural networks, these might be full blown ones such as, you know, MxNet, Caffe, you know, TensorFlow, etc. Or ones that are more refined or slimmed down to run in constrained environments such as, you know, TensorFlow, you know, light or, you know, such as, you know, a Neo, you know, the deep learning, the deep learning, your runtime environment. Call out also from a management perspective, ensure that you have, you know, something that's built into there that allows you to do remote monitoring and management as necessary. Your edge capabilities and orchestration provides some of that telemetry, but by having the ability to create a tunnel that can be outbound from the device to the cloud and then the cloud back to the device, is it allows you to do that type of remote monitoring or troubleshooting is necessary along with doing OTA updates. And then from the edge framework, this is going to depend on what you're doing. If you're a container-based shop, you know, Kubernetes or Rancher might be in a sort of, you know, environment to use. It might be EdgeX or Kuvex or Kuvej or, you know, our AWS IoT Greengrass, which is a fully, you know, managed environment for doing new edge computing capabilities. So operating systems and system packages. And then on top of that, we have essentially the user packages. This is where, essentially your developers or you are adding your specific value-add business logic to the device to do what you want it to do. You know, machine learning at the edge, detecting certain events and taking actions. And we normally see this happening either through, you know, native applications that you develop and your SCP or copying over to a device. This may be that through it's an orchestration of multiple containers that, you know, you're doing through, you know, Mezos or Kubernetes. Or go back to the Edge frameworks where you now have, you know, a rich environment where you can do things like as containers or server-less environments such as functions along with other things such as, you know, APIs, queuing or stream management, all capabilities at the Edge. A lot of times you, you know, want to interact with that Edge device locally. We'll see that in a demo a little bit later on. So you might want to be able to build the capability of having, you know, local APIs, you know, either restful ones or just quick request response or endpoints that other types of devices can then interact with if they're also working with the Edge device. And then for the machine learning model is the ability to be able to have machine learning models downloaded and operate in the device. A long ability to have some type of new closed loop or round-trip operation for doing retraining and detection of drift so that you can ensure your models are running properly. Let's take a look at an example of all these kind of in operation. So we have on the left hand side, you know, physical camera, we have our Edge device, we have the cloud. So the first thing we do is we have a local process, a collector, that's taking in a certain type of format, you know, h.265, and it's using something like OpenCV to convert this into a frame that our inference model can actually consume. It's expecting it to come in a BGR format and resize to 300 by 300 pixels. Our inference could be a Python-based application, you know, maybe using ResNet 50 as our model and using MXNet underneath the covers. And then it's basically, it's incorporated into the training model that it's loading into. So we take in our video, which is, you know, a standard h.265 format. We're converting it into the format that we want. And then once the inference is done, we're only sending to the cloud the inference capability. So the list of the objects that are detected and the bounding boxes around it. So in this particular case, you know, everything on the device is very rich in the amount of data that's being transferred back and forth. We're having megabits per second from the camera into our device, but we're only sending bytes or kilobytes of information based on inference out to the cloud and only the things of interest. So let's take our basic example here. We're going to take it to the next level. So here's what we're kind of saying, all right, for the local streaming, we can use some of this G streamer pipeline that allows us to abstract out any type of cameras as to an input. We then bring those into our, you know, transformations to BGR in the resizing. And then in the green shaded section are all of our edge components that we're orchestrating. We have our machine learning inference model that's taken in the values and giving back, you know, giving back, you know, values back. This can either make it available to like a local web server and we'll see it in the demo or you might, we might be doing data enrichment which we then want to send to the cloud for a common web platform there. It also might be that we want to take a look at video of interest that might be useful for retraining our model. And then on the cloud side where we have the capability and resources to essentially manage not just one of these devices, the edge, but potentially thousands, is we may have a web server that essentially is an aggregate of all thousands of our devices that we can drill down into each one as necessary or see what the total aggregate is. For the machine learning trained model training component is we might be getting back video snippets from different views or perspectives and then doing the training of a model across, you know, a large amount of instances of, you know, GPU accelerated, you know, machines to recreate a new model which we can then deploy as a retrained model out to the edge to all of our devices as a single deployment over their updates to then have them take advantage of those new capabilities. Apologies there my story is getting dry. Just also call out and we'll go this a little bit further is that there's different types of layers that we can map from the edge device back into cloud capabilities which will help with the orchestration and automation of everything from the operating system all the way up through our machine learning models and compute functions. So with that I'm going to sort of transition into a demo here. So before I jump over and show you this in a YouTube video and sort of narrate what's happening, I want to kind of lay the land from the hardware perspective. So one of our partners Toradex put together a demo of what they call sort of, you know, pasta inference. What you see here is essentially a conveyor belt at the bottom and at the center of that is we have all of our compute capabilities. On the right hand side we place pieces of pasta different shapes onto the conveyor belt and then our central edge device drives the conveyor belt forward at a particular speed and as the pasta comes underneath the camera it does object detection determine what type of pasta it is and what type of confidence level is there. The HVMI for the local user interface is used for a local interface that like somebody on a plant floor would be able to see and then we put the Wi-Fi for cloud connectivity which is sending that other data out to the cloud for a central type of presentation. So with that I'm going to stop presenting here and go over to a YouTube video and I'll do a little bit of narration on that. All right now I'm going to let this settle. So what we're essentially seeing here before I start this is one of our partner managers that was going through this. So as you can see here and I'll start the video up is that we see the pasta on the conveyor belt and at the center you can see the camera that's facing down with a light which is taking a look on that. What you're showing right there is a system on module or system on core of an IMX-8 processor which is running beside the green grass and the inference components. And you can see that the the pasta comes through and I'm going to skip forward a little bit here so we can take a look at the screen on the left. So this screen here is that local interface coming off the HDMI. So as you can see on the right hand side of the screen is that the conveyor belt is moving at you know certain frames per second and we're actually seeing the pasta there and then the left hand side is we're getting the inference results locally. So as the machine learning model says ah this is a piece of fusilli pasta or elbow pasta it can then basically you know increase the account and show what our confidence levels are for this here. So this is all the capabilities that's happening locally on the edge and this can be at you whatever speed that we want you know we can take a full advantage of you know HDMI full motion video these types of things. Let me scroll forward a little bit here and above this is another display which essentially is showing the cloud side so all of that data that's being collected locally is that we're only sending back a small JSON object or small amount of information about the current state of the situation or of the device and you note there on the left hand side as we'll see the pasta on current frame bump up when it sees a piece of pasta and then there's the other portions of you know what the the certain the inference things are. So this is a full you know web application workload that in real time is seen values coming back and then that is basically doing an update on the display here. I'll pause this and go back over to our presentation. All right so that was basically the hardware that you saw here. If we were to take a look at the architecture this one is very specific and opinionated for AWS but it kind of demonstrates the same capabilities. Left hand side is we have our device right hand side is the cloud. On the device side we have AWS IoT Greengrass which is our edge compute capability which is orchestrating all of our functions and our machine learning model. The little circles in there which have got a lambda symbol are our lambda functions and this is sort of serverless compute that we're running you know on a local device and this allows it to interface with the actual camera and the machine learning model to do the inference. The the lines between the device to the cloud a breakout into two. The top one goes into SageMaker and that is our machine learning set of services that allow you to train models and do machine learning operations and this is where we can take back that retraining data but then all those other boxes below IoT Core, Cognito, API Gateway, CloudFront etc are all the components that make up that that second display that we saw which is the you know the the overall view and the interesting thing about this is all those boxes we have got databases we've got compute we've got you know static websites reactive websites these are all able to be deployed through a single click through the CloudFormation template. So just you know just the demo here and I always like using YouTube because the demo guides are normally very good about this is that this kind of demonstrates how you can have local capabilities on a device and that if you want to interact with it locally you can do so but that you can also do those three laws still interact with things you know from the cloud side where you're on premises environment. So we've spoken about you know some of the use cases for for edge and also what some of the patterns are but because the devices are out in the field I can guarantee that there will be problems with them as soon as you don't have the capability of touching a device to either do something to it or even reset it that's when things go wrong. So the first one that is probably I would say 80 to 90 percent of issues that we see with edge computing is internet connectivity or intermittent connectivity. It's very common to hear that a backhoe has taken out you know a certain amount of internet links for the the northeast United States or for the southwest United States. It's very common to see you degraded you know capabilities if there's ZDoS attacks or you know bad routing tables and these types of things and what that basically means is that from a cloud perspective you may have a thousand devices out there and any one time you may only have a certain portion of your fleet that's actually active or available. So how do we have to deal with those devices or how should we deal with those devices that go from an online to offline state? Our CTO, Werner Vogels, has got a saying that everything fails all the time. So if you take that that particular statement you can design for failure and you should design for failure. Assume that whatever you put into the field everything potential is going to go wrong. If you design for that then you've addressed those types of failure points. So from a connectivity perspective is you know consider things that you know if the device is in an online state what happens it goes offline and what happens it goes offline for a couple seconds a couple minutes or it's offline for a few hours due to a some type of internet outage or networking outage. Understand the the interactions between the local components to the cloud back and forth so that you know if you are normally sending data to the cloud and you can't because it's offline how do you queue that data locally? Conversely think about also if you're in an offline state and you go online you imagine from the cloud that you've got your deployments you deployed version one to all of your devices. Then over the course of a couple days you've done versions two three and four but when your devices have been offline for that period of time so when it comes back online at version one and reaches out to the cloud do you then want to go ahead and deploy versions two three and four to get it up to date or can you go from version one all the way over to version four to complete that process. So those are some of the things to consider and from a timing or temporal perspective think about this in matter of minutes to potentially even years like kind of push yours out there but we do have customers for instance that have got shipping containers with edge devices on them. When they're actually in a port and in a shipping container yard they've got good internet access so they can say you know what's the the current condition of a cooling container. When those containers are actually put onto a ship and a ship is out at sea they normally have no zero connectivity but they're still running local edge computing operations. This might be to adjust thermostat values tracking conditions of pumps and compressors. So when they actually leave one port and enter another port that could be a two to three week period. How do you then sort of flush that data that's been collected? Do you basically just blast it down a pipe and get it out there or do you sort of throttle it so that other operations in the background can happen? So think about things around your intermittent connectivity from your device to the cloud. The other key when we see is that security constraints. A well-designed edge device will always reach out to the cloud. It will never open a listening port and looking for an incoming connection. It will always be the one to establish an outbound connection to a web to a always available service in the cloud. This could be a full variety of ports depending on your protocols and the actual end points might be different. A lot of times is that you'll have a fully qualified domain name such as you know myservicename.it.example.com. In this case here is that if you go to your firewall people and say can you please open the port port port three to myservicename.it.example.com. A lot of times I say we don't do that. We don't do fully qualified domain names. We only do IP addresses. So what's the IP address associated with that name? Sometimes it might be a single IP address. In other times especially for cloud services that might be thousands, hundreds of thousands or even potentially millions of IP addresses that that endpoint can change over a course of time. And most times the security people don't say no way we're not going to allow that to happen. So one way to address that is to actually reduce the access surface area. So instead of the attack surface area for like doing security testing is if you can reduce the amount of end points that a device has to get to and through using like this the VPN tunnel you can help address that from a security perspective. The tunnel can be something that's site to site like established from a firewall back into the cloud or data center environment or it can be a client side VPN application that takes everything on your local device sends it through a VPN tunnel out to a set of addresses maybe one or two IP addresses for resiliency and then only one port. But by doing that this allows you and to to convey to security people that here's a set of addresses and a specific set of ports which are easily accessible. There's a couple of design considerations that come into play here too. On the device itself design for your functionality. A common thing that I see from customers is that you may have your machine learning folk running Python 3.6 for all of their machine learning frameworks. NVIDIA has a good example of this with their jetpack environment. Jetpack today everything is designed out of the box to run under 3.6. That's the frameworks like the TensorFlow, the CUDA environment, etc. But you also might have some of your other developers that are writing other functionality that we're using Python 3.8. So in this case consider what applications or languages need to be available on your actual environment. If you're doing this from the Linux perspective directly you may want to say all right let's go ahead and install Python 3.6 and 3.8 and give specific paths to the applications that need this. And that's when you're basically doing sort of packaging for a common type of environment. When it comes to container-based approach you may say all right for our machine learning-based containers let's build in or put in the layers for Python 3.6 to support those people that are developing from that perspective. And let's use Python 3.8 for the developers that are writing other containers or microservices for other applications of our edge device. Other thing is we're going to be doing updates of these devices. These initially you know these will originate from the cloud. There will be a deployment or a manifest of all the changes that need to be made to the local device. And these can be things everything from operating system updates all the way up through our orchestrating component. Just to share that you have a ability to not break the device it's very it's very hard to unring the bell once you've deployed something that you're like oh darn this this is going to to break this that that does require a truck or somebody you're visiting the device. There's a lot of mechanisms out there to essentially help achieve this. A good example is Mender.io for instance has got the ability to basically have multiple partitions so that when you deploy a new version it goes in a different partition when it's rebooted either you see or grub itself can simply say did I start up correctly yes or no and if I didn't roll back to the previous operation and also try to consider if you can to use atomic operations or transactional changes. Instead of you know deploying a command to do like an app get updates on a variety of devices even if they're all the same version and the same patch level from a previous app get update there is a potential that they can slightly get out of drift but if you can build all of your updates into a single operation that if the transact can take place completely it's committed otherwise you do that rollback component it's always good to sort of consider that also the device itself is going to those varying types of environments the car the automotive example is good that you start off in a wi-fi connection your garage so you might be doing some updates to the operating system on the car itself when you're on the road and you're doing autonomous driving or other things you may want to actually collect data that is then used for your training purposes you know further so make sure your device can understand when it goes from one state wi-fi to cellular or back again from cellular to wi-fi you want to comes back into the garage and can do the upload of all that particular data also devices are going to be operating with other things in the automobile this might be with other interacting with other ECUs in other environments you may be talking to manufacturing this you know devices that put all of their data onto a windows files here just ensure that you've got the capabilities of your device to be able to interact with other either systems or sensors you know inside of the environment that they're deployed and the other one also and I should have and when I redo the slide I'm going to put this to top security is paramount every device out there should have a unique principle that associates that device to something in the cloud so that you can identify it so when it does make a connection that it can go through a authentication and authorization process to identify what the device is and what capabilities it has from the cloud side these are normally you know supported through either X509 certificates or other mechanisms so if you can ensure that you've got some type of hardware-based security integration this could be through secure elements TPMs or other ways that if you do deploy or when you do do deploy your certificates and credentials onto a device that you can actually ensure that they are you're protected as much as possible and that they can't be exploited but if a device does get exploited or compromised and you've got detection capabilities in the cloud for that have a mechanism to either you know reset the device to a factory state or the ability to rotate credentials to give it you know a new set of of principles that it's to interoperate with all right so that that pretty much goes over you know what we see from sort of the the design patterns and how to work it you know when things go wrong but one of the other key things is please leverage IT or your OT colleagues if possible this is a common thing that we see is that if you're in a shop that is using let's say a lot of Red Hat Linux under the covers and as an embedded developer an edge developer you're using a Raspberry Pi and you're using the you know you know Raspberry Pi OS which is a DB type of environment great for development but if you're looking to actually you know manage and deploy device to the edge see if you can leverage what is currently being used internally from your own IT organization you know commonly we'll see things where you know that you know you're you're set on a specific set of Linux and you've got those skills or capabilities but it also might be that you also have got you've built the capabilities for doing you know updates consider you know are you reaching out to public repositories or are you going to be reaching into to private satellites or PPAs for doing your updates also you know if you're a JavaScript you've probably got you know a set of Java JREs that you're going to use and deploy if you can align with what you have from an IT perspective that really does help from helping manage those components now once you have those sort of identified is we also say that you know we separate sort of edge capabilities from the underlying you know OS orchestration so you know extending things like using salt ansible stuff puppet etc for managing the underlying operating system devices is is a good thing then on top of that you can do things such as you know potentially using containers that you you want to use your rancher at the edge or kube edge or some type of your mesos environment for managing your container operating environment and then if it's a serverless type of capabilities you're deploying is you know you'll use things such as you open with help take those you know servers or functions you deploy in the cloud and deploy those out to the edge so if we move up to the next level which is sort of the assets these are sort of the the value add or the business logic that you're you're adding you might be deploying you know might be developing functions and micro service based architecture or it might be a combination of doctor images and doctor composed files that deploy all of your functionality at the edge it's also the the output from machine learning so the machine learning model that you want to deploy to your your your devices along with operating system updates firmware etc normally from this you'll have an end points where your devices can reach back into to say hey do I have a new deployment it gets the manifested deployment information back and then inside of that manifest will say oh I'm getting my functions from this particular URL or from these objects my machine learning ones from this one etc also manage you know fleet management if you have the capability of sort of your managing and seeing the health and state of your devices overall if it's got a solution for that you know please use that same for your status reporting and dashboards and then as mentioned discreet device security identity every device out there should have a unique certificate or principal and there should be nothing shared from that perspective from the machine learning this normally depends on the organization but I normally see the customers will have an IT organization and sort of a data science group that may be part of IT or separately but if you can use some of the same tooling and approaches that they're using for doing data science and machine learning using those of the edge is also great if you're using your tensor flow internally great maybe use you focus on using tensor flow light or LT at the edge the same thing is that you're if you're using mxnet or cafe etc is you're using those same types of outputs or models will help you understand the you'll help you and the the machine learning people know what's you know how things are happening in the cloud and then as you're actually deploying same thing it's like you know having a process for that I create a model I deploy it to the edge I use the inference on this to actually collect data I push some training data back to the cloud and at that point I can do the retrain into the model and using you know a lot of the scale out capabilities then taking that trained model and putting it back onto our devices so essentially having that sort of you know closed looped environment for doing model deployment model or data collection model retraining model deployment etc and then finally is that and this is where it gets a little bit sort of hand wavy it depends on what your development cycle looks like in the cloud is a lot of times if you're using like you know a continuous integration continuous deployment process you're doing a lot of deployments on a daily basis but if you can sort of incorporate that when developers are you know writing code and making commits that that kicks off a pipeline that goes through essentially the entire process of taking that code creating the assets going through a testing on either you know virtual environment or on local testing devices and then getting back essentially for the faster failure of a build is that essentially can demonstrate you know a pipeline that you normally use for like you know cloud-based for web-based applications that can be applied to edge compute components too we normally see this only when you're talking from the the the assets on up so the the blue box into the green boxes but this also you know when you get into like full regression testing to be all right we're making some significant changes with the underlying operating system and we want to do a full test of of OS system packages user packages etc and of course you have different types of mechanisms for having deployments you can be one box in this you can do in blue greens fleet deployments but have the capability for for how you're doing both over the air updates to send out things but also a rollback if necessary as we kind of mentioned on the what things go wrong slide all right that's you know 40 minutes then we've covered a lot here but what I like to sort of go is this you know sort of you know wrap everything up together and sort of summarize this so we'll go ahead and start with you know should you use edge or is edge a good use case and I'd say that the sort of litmus test is if there's a capability out there that addresses one of the laws of either you know speed or physics cost or you know from a jurisdictional or sovereignty perspective laws of the land you might have a good use case for that normally what we see is that if there's any type of machine learning that's being done at the edge where somebody says I want to put a camera out there and detect when somebody walks in the door or they're not wearing their personal protection equipment or when you know things like a manufacturing line look properly do I have the right posture or not those are always you know sort of your good indicators that you've got a good edge case once you do have that you know either define and create your own or adopt existing patterns to sort of build out that full stack component so you know think about the hardware think about the operating system the potential additional operating system you know repo packages or distribution packages you want to include then along with how you're going to orchestrate and deploy all of this business or edge logic that you want to put onto the device and then think about how these things are going to be deployed you know it's very common to see these in a your test or lab environment where you connect them to a hardwired ethernet or to a very consistent Wi-Fi these work fine you then put them into a remote terminal unit on an oil well in you know Northeastern New Mexico and things don't so it works well because when the sun's up it degrades the signal quality to the point that the device can barely get a connection and it you know it goes in and out so think about those sort of intermittent capabilities and then some of the things that we mentioned in the gotchas component for addressing the device and you know I really can't stress it enough but you know you know stand on the shoulders of giants take advantage of all of the hard work that you're you know you know IT or your operational technology the OT departments have done for for how they're deploying things and how they manage things too by doing that it helps significantly reduce the technical debt that you might be creating for a sort of a bespoke or a custom edge use case but at the same time is that it allows you to also have access to other resources for for troubleshooting or for potentially you know new opportunities and also keeping up to date for when they make changes that you can actually take advantage of those also and this is sort of the the things you know from my from AWS is you know AWS IoT green grass is the edge compute capability that we have that essentially addresses a lot of these these things we spoke about so essentially addresses the orchestration the ability to do container management at the edge along with you know lambda functions are sort of compute everything that can be deployed onto a variety of different hard work components and it also handles the machine learning where you can train you know a machine learning model in the cloud with SageMaker click a button and then have that automatically deployed through a set of you know prescripted connectors that will do that without you having to write this actual code and with that I'm going to open this up to Q and A I normally if we're together I would just you know point to you or point to you and it has a question but one of my colleagues Lisa Ralph is going to send questions over to me and I'll simply reread them and give you answers back here but first with a quick drink here all right so the first question is what will be the future of edge computing will there be any more evolution and in computing platform and I think the answer to that would be well first of all I don't really know what the future does hold but if we take a look you know historically what has happened is that things that are currently out there in sort of you know an innovative type of space that you're looking from one particular angle or another but there's sort of you know a one-off type of solution or one one-off type of situation normally can either evolve into the next type of technology or they they can essentially fade away I'd probably say you know you keep a weather eye on what things are happening in the the compute or the IT space or even the embed of Linux space in general and see where things are going is you know you know 2020 hindsight but but normally you know what we see from things such as you know from from markets or from the the analysts is that the the explosion of IoT devices into the tens of billions that are out there is giving us a lot of capabilities at the edge from a sort of discrete component perspective and now we're seeing sort of that same kind of you know waiver adoption happening from your edge compute or or edge capability which isn't just compute this machine learning other types of things so a good question so the other question is there a way to retrain an object detection model automatically from data gathered from the edge of a device using SageMaker and this is specific to AWS but the answer is yes so we actually have a green grass connector which is called the the machine it's a machine learning inference connector that allows for retraining so it allows it basically deploys the the capability of the edge to collect data and based on certain inference values if it sees probability dropping or or getting you know not hitting like a high confidence threshold to be able to capture some of that data it could be it could be you know video data or other types of you know input data that can then can be pushed up to the cloud and then automated from that perspective does require some work and coordination for how you you look to create your notebook and your training model or your training approach inside of SageMaker but that is definitely doable so the next question I love this one if an app needs Python 3.6 sounds fine you know you know 3.8 is simply just rolling out so you know should 3.8 be fine unless you're saying the Python isn't good with backwards compatibility and that's predominantly exactly what it is we we see customers out there that do this you've installed 3.6 it's like all right we have things that are running 3.6 and 3.8 so let's just do a sim link so we'll sim link our 3.6 to a 3.8 environment or 3.7 whatever is hard coded to use those particular paths and 99 9% of the time things work where things normally break is when it comes into some of the machine learning components in the case like IoT Greengrass we only support Python 3.7 so to get back to 3.6 you can do that we can sim link our 3.7 to 3.6 and at that point things normally work however certain things can break when you're doing different operations with machine learning models or like with the SageMaker Neo DLR perspective so that's why I I always like to say is you spend as horrible as can be sometimes spend a lot of time working with your Python environments specifically to see what is or isn't compatible and that's also sort of a good plug or use case for that that unit testing and that full regression testing of all the capabilities of your your IoT or your your edge capabilities there. Let's see so we have another question in regards to security and identity of devices in an IoT network so the question is are there any ongoing experiments to integrate hyper ledger fabric capabilities Indie Eartha Aries what is the future of blockchain blockchain technology in IoT I am not an expert in blockchain overall it is a question that we do hear a lot from customers normally right now sort of in the innovation phase or steps what I would say or what I have seen is that customers will take advantage of certain blockchain technologies for source data and this can come into things such as like an oil and gas industry where data is measured at certain you know remote terminal units things for like you know natural gas or oil expect then there are certain values that will determine the quality of the quantity of that picture product and that's actually that's actually tracked from forth all the way through distribution so from essentially an upstream all the way to downstream and we have been customers that are taking advantage of blockchain to essentially you know ledger those types of measurements there and then along the entire process of when they're using the technology unfortunately I am not that familiar with some of the other hyper led hyper ledger fabric capabilities out there but it's something that I'm keeping an eye on too to see how that can be sort of incorporated into an IoT environment what I would say is I never I never see that associated with the security or identity of a device it's more about potentially the data that it's actually creating and then sort of the like I said the chain of trust or the chain of custody for that data as it goes through a next in actual process so another question is are there any good resources to learned edge computing and Lisa kind of answered this but go to the virtual IoT booth in the golden silver hall is normally we'd have the staffed with people like myself solutions architect where you can come and have a one on one conversation virtually you can you can kind of do that today and simply bring up your particular use cases and have sort of a you know a one on one or you know even a one on many type of conversation in regards to that and there's also a great collection on the inside of the booth itself on a getting started guide so I worked with some of our marketing people to essentially say if you want to get started with Greengrass or Sage maker or our device SDKs or just IoT in general there's some good sort of links there directing to our developer guides that will start to explain what the services do and how they operate so another question as you know how should I compare the models for inference at the edge is there any tools for that I'm going to guess that the question is more around the quality of a particular model you know is model A doing the best or is model B doing the best when you're given a certain amount of data and what probably is that that comes back to the more data that you can collect from the edge to use as training data for your data sets the better in ML in general 10 or 100 samples is probably going to over train or over fit a model quickly 1000, 10,000, 100,000 sets of data is better and so that's why I'd say is that if you are looking to deploy training to the edge a lot of times the inference itself will happen cloud side when you're going to the training gyrations but then actually when you do deploy the model you could take sort of the the blue grain approach if you've got your entire fleet that's running you know model version one and you're getting back certain values and from the you know from the cloud side you're classifying or annotating the data of either correct or incorrect and you can use something like you say you make a ground truth for that you may want to say all right for 10% of my fleet I'm going to you know deploy model version two to the to the blue you know the blue fleet if you will and see what the differences are that comes that with that if it's better you can then say all right this this go ahead and roll everybody out to version two if it isn't you at least got some good data back that you can then use to make the terminate you know for the for the next cycle and with that they're going to be cutting the software in a few minutes I'm going to take one more question if there are any follow-ups please go to the database booth and also if there are things I'll be on the Slack channel for this this track later this afternoon probably about a couple hours or so can answer questions there and unfortunately that does going to be the last quest actually no here's the last question so coming from a developing country how can one start edge computing easily and teach community to to adopt it in the right services I'd probably say the best approach for that is find something find examples that are out there in the world that have worked and and kind of you know use those you'll play around with those and understand those are but take those lessons learned and see what that actual community that you're you're looking to potentially you know educate or elevate may have problems with for instance I do your mission work in Haiti and a situation there do you think it would be like you know water or power you know types of things when the key things there was they have that solar panels that they actually charge their cell phones during the day and you know that could be a potential opportunity that if you've got cell you know cell sites you know spread across the country they're doing charging and one cell is degraded or it's not pointed in the right direction that could be a good example of of getting some of that that edge data you know the operation of you know how many phones are being charged the conditions et cetera and be able to to feed that back to go out and make adjustments or whatever so with that I'm going to thank everybody for attending I do appreciate this I really wish this was in person so that you know I give you a heartfelt thanks to everybody but it's been my pleasure to to go ahead and present your deep learnings at the edge of you know what I've gone through you know battle train for the last couple of years and please enjoy the rest of the the embedded embedded links conference and with that I think we'll go ahead and close this out