 Hi, everyone. Welcome to the top smart cities in the cloud. Today, we're going to talk about two platforms, NVIDIA Metropolis and Red Hat OpenShift, and how these two platforms can be used to build smart cities. I'm Sajid Beswaz from NVIDIA, along with me is Pete Mackinnon from Red Hat. NVIDIA Metropolis is an H2 Cloud platform for smart cities. It is leading an AI revolution, giving you the tools, technologies and expertise to meet every challenge with smarter, faster applications. Red Hat OpenShift, on the other hand, offers a consistent hybrid cloud foundation for building and scaling containerized applications. Together, they bring businesses, cutting-edge technologies and developer ecosystem to create, deploy, and scale IoT applications from H2 Cloud. Here is today's agenda. We'll briefly talk about smart cities and traffic management, Kubernetes from H2 Cloud, Red Hat OpenShift, smart cities and traffic management. Consider large cities like Newark or San Francisco. There are thousands of traffic intersections in a given city, and each of these intersections are equipped with more than one camera. Now, question is, how would one process this amount of video stream and get meaningful insights real-time? One option is to push all the data to the cloud, but it's too much amount of network bandwidth requirement. It's not viable in some places because you don't have the network. The other option is, you can push the computer to the edge. Now, NVIDIA EGX is a platform which basically brings accelerated AI to the edge. So, here is one deployment of DeepStream on NVIDIA EGX. NVIDIA EGX provides the Kubernetes cluster at the edge running on top of GPU, and DeepStream provides the perception layer which processes video stream coming into it. So, if you look at the top left corner, you see a scene with respect to a camera. Now, in order to get meaningful insights from this video stream coming in, one would need to detect all the objects which is done by DeepStream SDK. And once the detections happen, one would like to track the objects like how the object has been moving in terms of space and time. So, the two key functionality which is being done for this reference application is object detection and tracking. Once the object detection and tracking is done, and remember this is done with respect to hundreds of thousands of cameras at a given point of time, this information with respect to a specific sensor is sent to the cloud for further processing. In the cloud, NVIDIA Metropolis once again provides SDK to do streaming analytics. Whatever was coming from the edge was primarily in the image or in the camera coordinate. So, in order to do any kind of geospatial processing, the image coordinates has to be converted to geocoordinates before we can do such processing. So, once it is converted to image to geocoordinates, the next module in the pipeline is trajectory state management. So, what is a trajectory? Trajectory is a sequence of point with respect to your object movement. In this case, primarily it is sequence of point where a vehicle is moving in terms of space and time. So, now we have a formal definition of trajectory. We can do further processing with respect to the data to get meaningful insights. So, what are the trajectory processing attributes we can gain? So, we can move, we can get how a vehicle have been moving, what was the speed of a vehicle at a given point of time, what was the direction the vehicle was moving. Along with that, where exactly it was moving? So, if the entire road network is a graph, let us say the graph is provided by OpenStreetMap, we know exactly on what road segment, what are the vehicles moving and what the vehicle speed and which directions they are moving. This information can be aggregated and presented to traffic engineers and traffic managers and they can always have a meaningful insight of the entire traffic or city traffic. So, once the traffic state management is done, there are further modules which basically enhance the capability to find out if any onwards incident has happened or any incident which is of concern has happened and this happens real time. So, there is an anomaly detection model which basically detects anomalies. The detection model is very simple in terms of the thought process, like if you consider the behavior of all traffic as normal and then see if there is a deviation from that norm, so we call it abnormal. If there is a deviation from the norm, in that case what happens is that an anomaly is triggered and that snippet of the video can be seen by the traffic managers to see if something untowards has happened. So, here is a, if you look at the right hand side, that is the browser-based command center. A traffic engineer or manager can basically go to the command center and see how the traffic has been flowing through a city. Not only that, they can go to a specific corridor and say how this corridor at a given point of time and what is the traffic in that and they can further drill down to a traffic intersection and the cameras corresponding to the traffic intersections. Here is one example where an incident was observed that it happened to be, if you clearly look at the picture, it happened to be a vehicle which is moving in the wrong direction or the vehicle was moving in the opposite directions and it was caught by the anomaly detection module. This is very important information to the traffic engineers because they would like to take corrective action given such incident. The NVIDIA Metropolis Deep Stream provides a perception pipeline. Here is a reference implementation of such a pipeline. As the video comes in, it has to be first decoded. The video itself could be encoded with H.264 or 265. So, before any pre-processing can happen, it is first decoded. In the pre-processing layer, a frame might have to be resized before any kind of inference can be done on that frame. Then comes the object detection. So, once the object is detected, it is followed by the tracking module which tracks object in terms of the space and time. Now, thereafter, we can send the metadata to the cloud in terms of object detections and their bounding boxes or any other attributes which have been detected or we can do further analysis in terms of building a composite layer in terms of the detections and the original video. For the Smart Cities use cases, we just send out the metadata for the detected objects and the tracking ID. NVIDIA Metropolis Analytics is built on top of Apache Spark Streaming. The Spark Streaming pipeline is running on Red Hat OpenShift in the cloud. As data from multiple sensors are coming into the message broker, the first step is converting those bounding boxes which has the camera coordinates into global coordinates. This conversion depends on camera calibration which provides homography matrix. Homography matrix, think about it as a mapping between an image coordinate to geospatial coordinates. Now that we are dealing with geospatial coordinates, we can do all kinds of geospatial analytics. Next step is trajectory state management. Think about this trajectory state management as imagine a city with thousands of cars, thousands of vehicles moving across the city. If you want an equivalent in-memory representation of those objects moving, that would be a trajectory state management. And because we have a trajectory state management, now we can do subsequent analysis of the trajectory thereafter. So the next step in the pipeline is trajectory processing which computes how the vehicles are moving in terms of the speed and directions. And there is also a key functionality which is map matching. So map matching allows a user to map a trajectory with respect to the road links on which it is travelling. That means on the road segment on which it is travelling. The road link definition is provided by the OpenStreetMap. It is basically a graph of road network. And once we have that information that at a given point of time, how many cars are on specific road links, we exactly know the flow rate of all traffic moving with respect to that segment of the road. And once we aggregate this information across the entire city, this information is very valuable to the city traffic engineer or city traffic manager. They can take decision with respect to if they are seeing some congestion in parts of the city and a normal flow rate in other parts of the city. Apart from the normal stats which comes out from the trajectory processing, the next module basically does anomaly detection. This is on a simple intuition that whatever we see throughout in terms of the traffic pattern is kind of a normal behaviour. That means this is a kind of, this is a LSTM model which is basically a sequence to sequence deep learning model. It basically checks if there is any anomaly which is deviating from the norm. So it learns from the normal behaviour and then it flags anything which deviates from the norm and it raises an alert. This helps the traffic engineer to take corrective action. If they see something is wrong with respect to the, look at the snippet of the video where the alert was raised and see some corrective actions has to be taken. Thereafter, the messages are sent to Elasticsearch via LogStash and it is exposed by APR to the user interface which is a browser-based user interface. This is how the Kubernetes H2 Cloud deployment looks like. On the edge, we are running Deepstream in the EGX Kubernetes node which is close to the sensors. On the right, we are running the Metropolis Analytics stack in the cloud. This stack is running on Red Hat OpenShift and if you consider the NGC registry which is basically a repository for all container images, models and hem charts. Now, Pete is going to walk you through the details about how the OpenShift deployment looks like. Thanks, Sujit. As part of this collaboration, the analytics service platform for NVIDIA Metropolis is provided by Red Hat OpenShift. OpenShift is the enterprise Kubernetes platform for modern hybrid cloud application development and deployment. Applications are deployed as container-based microservices into an OpenShift cluster running in the cloud. The cluster is responsible for orchestrating the container deployments, maintaining their replicas and monitoring system health to provide high availability and scalability. This OpenShift cluster is hosted in the cloud and comprised of three master nodes for the control plane and 10 worker nodes each with 16 virtual CPU and 64 gigabytes of memory. The NVIDIA Metropolis software functionality is provided by a combination of NVIDIA developed reference applications and open-source components. This slide gives an overview of the data flow and execution architecture of Metropolis on OpenShift. We see the organization of Metropolis microservices deployed as pods in OpenShift, which encapsulate the application containers. Now that we have seen an overview of the integration with Metropolis, let's talk a little bit more about OpenShift itself. OpenShift is 100% certified Kubernetes, fully open-source and non-proprietary. However, OpenShift extends Kubernetes to provide enterprise-class security features and useful developer tools and abstractions. Many critical features of upstream Kubernetes have been contributed by Red Hat, such as RBAC and pod security policies. OpenShift is built on top of RHEL and RHEL Core OS, the two most trusted Linux distributions in industry. Automated installation and day-to-operations are integral to the platform for smooth upgrade paths. The platform is designed to address the needs of both IT operations and developers. CICD capabilities are provided by Tecton and containerized Jenkins. The Red Hat service mesh, based on Istio, provides comprehensive microservice traffic management and monitoring. All this can be deployed to multiple clouds, be it bare metal, virtualized or private and public cloud. Here is another view of the OpenShift platform. The basics of a Kubernetes cluster should look familiar. You see that the master control plane is replicated and highly available. The control plane provides API and authentication services, stores cluster state, manages scheduling, cluster and application health. Worker nodes run the RHEL or ARCOS operating systems and host the cluster workloads. Those workloads are a combination of infrastructure and application pods. SDN, routing and cluster service network is all managed by OpenShift as is persistent storage. There is an internal registry provided for image builds and image stream management. Finally, operators based on the operator framework encapsulate all this complexity. OpenShift provides the features of platform as a service and also of containers as a service. Applications can be implemented in any programming language you choose, with the only requirement being that the application can run within a container. You can deploy any pre-existing container image built to the OpenContainer Initiative image specification. However, OpenShift developer tooling can also build your own container image directly from your application source code and deploy it for you. The application source code can include a Dockerfile with instructions to build a container image, or you can use a source to image builder, which takes your application source code and converts it into a container image for you. Plus, there are many other OpenShift tools. Code ready containers or CRC bring a minimal pre-configured OpenShift forecluster to your local laptop or desktop computer for small scale development and testing purposes. OpenShift Do or Odo is a fast, iterative, and straightforward CLI tool for developers who write, build, and deploy applications on OpenShift. VS Code extensions speed up OpenShift development by supporting fully integrated OpenShift development and deployment within VS Code. You connect to any OpenShift cluster and create, debug, and deploy from the IDE itself. The OpenShift console is designed to enable both personas and DevOps, the developer and the administrator. For cluster administrators, the console provides a comprehensive view of all operational aspects of the cluster. Every project, every deployment, every pod, root, and service can easily be viewed and managed. The application in the cluster can be elastically scaled based on real-time demand. Machine resources or worker nodes can easily be added or removed. The number of running replicas for any of the components within the metropolis application could be easily adjusted with a simple edit of a field in the console. In both cases, OpenShift immediately and automatically reconciles the cluster state so that the resource modification is fulfilled. OpenShift includes a preconfigured pre-installed and self-updating monitoring stack. The stack is based on the Prometheus open-source project and its ecosystem. OpenShift provides a web interface to Prometheus which enables you to run Prometheus query language queries and examine the metrics visualized on a plot. It provides monitoring of cluster components and a set of Grafana dashboards. It also includes a set of alerts to immediately notify the cluster administrator about any problems. An operator is a method of packaging, deploying, and managing a Kubernetes application. Operators are extensions for Kubernetes. They make use of custom resources to minimize the burden of manually installing and configuring for complex clustered services such as Elasticsearch, Apache Kafka, and Apache Spark. OpenShift includes its own enterprise operator hub with 30 Red Hat operators and over 70 OpenShift certified operators that have been tested and supported by Red Hat and its partner ecosystem. In addition to those, the public operator hub.io today provides more than 80 community operators. Community operators broaden the choices for other Kubernetes distributions across different categories ranging from databases to artificial intelligence and machine learning applications. Now that we've described the Metropolis system, let's look at it from end to end in a video. First, we will start with a visual tour of the application internals via the OpenShift console. The console provides multiple means of OAuth-based authentication. Here we are using simple HC password. After login, we are presented with a listing of OpenShift projects visible based on the RBAC permissions assigned to the user. OpenShift projects are extensions to the Kubernetes namespace resource. Within a project, we can see the various types of resources that have been created. For many of the supervisory resources, such as deployments, replica sets, daemon sets, or staple sets, we can drill down into the pod to inspect the microservice function. Here we examine one of the Kafka pods and inspect its logs. If we are a cluster administrator, we can move over to a view of the cluster nodes as you see listed. For any one of those nodes, we have a dashboard showing us important details, such as memory and CPU usage, as well as file and network I.O. Now let's look at the Metropolis traffic management web UI itself, which is exposed as an OpenShift root. This view shows you the high-level information that matters. The average time it takes to get from one end of the corridor to another. It shows on the map the traffic conditions in terms of average speeds of vehicles on the various links at the highway, color-coded from slow-moving in red to normal movement in green. The collection of trajectories at this intersection are analyzed to estimate the following metrics. The number of vehicles, along with any specific road link, the speed of each individual vehicle, the aggregate speed of vehicles, the rate of flow of vehicles. Hi, Peter. And Sujit, can you hear me, sir? Sirs? Yeah, hi. Okay, great, yeah. Okay, well, I just want to thank everyone for listening to the Semi-Live broadcast. D-Live Q&A is now in progress and we have Peter and Sujit here with us to answer any questions. Thank you. Are there any questions in the chat, or anyone have questions? I don't see any questions in the chat. Maybe, Sujit, you know, we collaborated on this a while ago. Do you have any updates about things with metropolis and smart cities? Yeah, I mean, we are coming with new features, like, given the COVID situation, we are coming up with new capabilities in terms of people movement, social listening, so those are some of the things in terms of food fall. So this was a use case in terms of traffic management with respect to metropolis with coming up with new features which should be available by... So it depends on the product management decision, we are going to make it available, but tentatively beyond March, April 2021. And it looks like we do have a question in the Q&A, and the question is how many clusters and pods were used for metropolis implementation and is Red Hat OpenShift supported for scaling? I'll take that, and you can piggyback on that if you like, Sujit, so there were two clusters, there was the one out at the edge itself where the production layer was running and then there was the analytics cluster that was hosted on OpenShift in AWS and the edge cluster communicated and sent the trajectory metadata back through Kafka. And in terms of I can't remember Sujit the exact number of pods for the overall deployment, I don't know if you have that number off the top of your head. Yeah, yeah, yeah. So when you deploy perception at the edge so the GPU used to a T4 test a T4 and it would take at least on average one T4 consistent up to 50 sensors so we were using four T4s because we were running around 200 sensors, so 200 cameras that's where object detection and tracking and the CPU utilization at the edge is not so high but when it comes to processing Spark clusters and the Kafka cluster elastic search clusters, so the breakdown was Spark running around one driver and we are running four executors so that was kind of a five nodes and each of the nodes were around four cores and the memory allocated goes around 12 gigabytes so that's the Spark cluster configuration and the Kafka cluster configuration was once again we are not running a huge cluster we are running a three node cluster and the elastic search once again we are running a three node cluster but this cluster is obviously for OpenShift is horizontally scalable, Kubernetes in general is horizontally scalable so that means if you want to scale the cluster in terms of the compute in terms of the memory or in terms of the storage storage is abstracted from the storage services available in AWS but you can easily horizontally scale, I think that's the question can you horizontally scale it? Absolutely you can scale it, OpenShift has all the facilities and you can do it. The next question you sort of touched on it, Suja how many GPUs do you need for a cluster for this application and do you have any guidance about sizing and the gentleman asking the question might have sort of got his answer from what you just explained there but one thing about the GPU is that how much compute you need to do the inference, if it's just object detection and tracking you possibly can sustain up to 50 video streams given one T4 but let's say you have secondary GIEs like you have other inference engines which is detecting the color of the car, make of the car and other attributes of the car then you have to reduce your or you have to increase your GPU capacity with respect to how much inference you need at that any other questions? Q&A is empty at the moment and we might be coming up on time I think Yeah, I think we are on time