 Hello everyone, I'm Tony Lin, a software engineer in Azure Edge Devices engineering team, Cloud and AI Microsoft. We are chartered to build and ship Azure Edge devices with the hardware and software stack that covers everything end to end. Today we would like to introduce deploying AI solutions on the Edge with containers. To build an AI solution for Edge devices, the AI and machine learning models can be built using different frameworks such as TensorFlow, PyTorch, Cyclone, Keras, etc. in different formats. In addition, there are Silicon ecosystem partners built various hardware and AI accelerators. However, different individual hardware vendors have their own toolchain and frameworks for their devices such as GPU, FPGA, ASIC, or other hardware. As it's not easy for developers to build a unified inference app or business logic that supports different AI Edge gateway hardware. To address this issue, Onyx Open Neural Network Exchange is an open source model format for deep learning and traditional machine learning. Onyx gives data scientists and developers the freedom to choose the right framework for their task and have their models exported or converted to the standard Onyx format. Once the models are in the Onyx format, they can be run on a variety of platforms and devices. To run inference with Onyx models in different hardware, Onyx runtime is an accelerator for machine learning models with multi-platform support and a flexible interface to integrate with hardware-specific libraries. Onyx runtime provides extensible architecture that enables optimizers and hardware accelerators to provide low latency and high efficiency for computations by registering as execution providers. To deploy the solution at scale, container is a standardized portable packaging for your applications. There are pre-built containers with Onyx runtime and related packages. Developers can leverage those pre-built container and extend with the other packages, inference application, and business logic. Container can be pushed to container registry such as Azure Container Registry and be deployed with cloud services such as Azure IoT Edge. Let's take an example of building and deploying a vision AI solution on the Edge. The container packs Onyx runtime, OpenCV, and GStreamer as abstraction layer. Onyx runtime uses the portable Onyx computation graph format backed by execution providers optimized for hardware. OpenCV provides a real-time optimized computer vision library, tools, and hardware, where GStreamer provides a library for constructing graphs of media handling components. It also provides plugins for different hardware. Let's put those ingredients together. This is a sample vision AI solution with container running on AI Edge Gateway. An AI Edge Gateway can connect to one or more video sources. Where a video source can be a USB camera, an UnViv RTSP IP camera, or a HDDAR264 or HDDAR265 encoded video file. The AI Edge Gateway runs vision AI solution container for AI inference and video processing. Finally, it exposes a RTSP output with inference results and can be consumed by a RTSP viewer or other applications. To conclude, as there are various Edge devices with different AI accelerators, each kind of Edge device may run its own operating system with its specific torches, which becomes a big challenge. Therefore, we show some examples to develop AI solutions for Edge devices with Onyx runtime and container technologies. That provides the abstraction layer of Edge devices when performing AI inference on the Edge. For developers who build AI solutions using these concepts, the solutions may be running using different AI hardware without rewriting the inference application or business logic, which is easier for developers to deploy the AI solutions at scale. Thank you.