 Hi, welcome all. Today we would like to share even processing at Optum, the Cloud Events Way. I'm Janani Pathengi, Distinguished Engineer, 20 years in USE, and currently I lead the Optum Functions platform. And I also have Murugopan Sevakanchetti here with me. He's a Principal Engineer and Tech Lead and a K-Native Open Source Contributor. We would like to have a quick introduction about our organization, United Health Group. United Health Group is a highly diversified health and well-being company. It's headquartered in the United States and ranked seventh of the Fortune 500. We have two business platforms, United Health Care, which will focus on health benefits and Optum is a health services innovation company. Across Optum, we are able to serve many constituents of health system and we are grateful to serve and partner with nine out of the 10 Fortune 100 companies, nine out of the 10 US hospital and serve 125 million consumers. From infrastructure and technology perspective, we have an annual spend of about $3.5 billion on technology and innovation. 99% of our workloads run in our own data center with over one trillion transactions annually and these workloads will vary all the way from mainframe, big data, accelerators, distributed SQL containers and now with serverless ecosystem. So this is the agenda we planned to cover today. We wanted to give a quick introduction about serverless at Optum, challenges with event processing, why we started to explore cloud events and some of the Optum use cases with cloud events supplemented with demo and we would like to conclude with summary. So now let's look into the Optum functions. It's an on-prem fully managed multi-tenant serverless platform which supports stateless, even driven, request-based compute workloads. Currently, it's offered cross-DC, active-active with our HHA capability. From an option standpoint, we are increasing more than 1000 functions deployed today with more than 500 developers using this across the organization to solve some of our enterprise use cases. For example, ETL jobs today run as functions. We are able to serve machine learning models and also have some of our infrastructure housekeeping jobs running on this platform. So how did we make the open source stack to an enterprise service? We started with the open source stack here, Kubernetes, Istio, Knative and we added more enterprise capability on top of them. And as you can see here, to the left-hand side, we started with enterprise integration with Vault for authentication and authorization, Venevi for auto-TLS search rotation and with our enterprise jobs scheduler for all the ETL jobs that is running as functions today. And we strengthened our SRE with the well-defined SLO, SLI and dashboards. We have very good observability and we added a lot of self-healing and automation scripts and we also conduct quarterly chaos game days. And to the top, we added self-service UI to abstract the infrastructure APIs and provide simple code to URL experience to our end users. And we also added a lot of training and template projects. This is the reason why we wanted to have to make sure that the use cases are moved in a lift transform and shift from monolith to functions. This is all helping them, the end users or the developers with it. And we also introduced GRAL VM frameworks to improve the Java cold stats. Now, looking at the platform architecture, this architecture is running two different Kubernetes clusters across the data center and we provide auto-HA capability with BGP, NACAST and dynamic DNS. So now that you saw a overview of our platform, I wanna quickly show an example of our function deployment and how cloud events come into action. So as Jeremy explained, our platform is based on K&A2 and K&A2, the artifact that K&A2 expects is the container image and all the functions run as part. And other thing that K&A2 expects is the container image should expose a port so that it could check whether the application is up or not. So the application that I'm gonna show here is just a web application which is going to display some events when it gets it, okay? And it, as I told earlier, K&A2 expects a container image, okay? So for building a container image, you need to define a Docker file and then you need to use that to do the Docker build and run it. Instead of that, I have a Go-based application, okay, there's a Go-based application. To run this, I am going to use a tool called a SCO, okay? So SCO is a tool which makes the Docker build and push layer invisible to you totally. So using SCO, I'll be able to take the source code and then build and push it to the repository without even defending any Docker file, okay? For deployment, this is just a Kubernetes application, right? So I can use an YAML file, I can use the Kubernetes CLI. Instead, I'm going to use a KNCLI which is Abyssin K&A2 CLI which gives a better experience to be deploying it on any K&A2-based platform. And inside our company, we have a separate UI which abstracts KN, K&A2, SCO and Kubernetes which gives a better experience but for wider audience, I thought K&A2 would be for health, okay? So now when I run it, so what I'm expecting is a K&A2 control plane to deploy this in the Kubernetes cluster and give me a URL, okay? So that's what is happening here. So if I have the integrations in the right place with K&A2 and search manager, I can even get an HTTP URL. So now you see what you see is HTTP URL and then slowly it should turn to HTTP, there you go, okay? So within like 30 seconds, you were able to take the source code and then build, push and then deploy as well, get a fully secure URL, okay? So when I open this URL in the browser, I should see a web application, there you go. So this is the web application that I deployed. So when we send some events to this web application, it should display this events here. So when we deploy something to the K&A2 platform, it gives you a URL. And to trigger that, it's very simple. You can use a, you can call it from any other application. You can do a post, put, get, you can call it from postman. But in the real world, functions are like synonymous to event process. So it really comes down to how you're going to deliver events from, say, event source like Kafka, okay? So that is where K&A2 has a separate ecosystem called K&A2 event. It is even like independent of the K&A2 serving platform, okay? So you can just, if you want, you can just have the K&A2 eventing alone installed in your Kubernetes cluster. And in the K&A2 eventing world, there are like two main components, okay? Which we'll be talking throughout this session. One is an event source, and second one is a broker. The job of the event source is to reach out to the actual source, which is like Kafka, RabbitMQ, or another application, take the data from it, and then send it to the processing function. And in K&A2 event, the processing function, and whichever is there, you'll see, where it's called as a sync, okay? The sync can be any addressable service. That means anything that gives you a URL, like a K&A2 service, you saw it get a URL. You know, Kubernetes service, also there's an URL, okay? And then you can create endpoints to link outside services also in Kubernetes, okay? Anything like this can be a sync, okay? And then in the first, there are two types of delivery happening here. One is like one-to-one delivery. That's what we take the data from the source, use the K&A2 event source to deliver it directly to the K&A2 service or a Kubernetes service. In this case, it's like tightly coupled between source and the processor. And instead, if you have a buffer in between, it'll be better for processing. Also, you're decoupling the source and the processing function. So you can do a lot more with it, okay? For that's where the broker comes in, okay? So you have the source which takes the event and then puts it into the broker, which is like an event ingress or a buffer, whatever you want to call, or a state store for all the events that comes in. It will take the kitchen sync of the events coming in, okay? And we can have a trigger that's another K&A2 resource, in which we can specify which types of events needs to be routed to which sync, okay? Suppose I have a K&A2 service which just looks for a Kafka service, I'll create a trigger and then say that, okay, I want Kafka events alone. Then the broker will deliver the Kafka events alone to this K&A2 service. For all these things to happen, they need to be glue or what we want. They need to be particular event type that's the, okay, that is where cloud events comes into play. And this whole eventing ecosystem of K&A2 was built around this cloud events only. So what is clouding? Cloud events is an event specification given by CNCF. It is not a data specification. The data inside cloud events can be anything, okay? So it can be text, it can be binary. They don't have any say over that, okay? So as you can see in the right, that is the data. And on the top, it just asks these four headers to be added, okay? So with these four headers, you can do a lot of things. That's what we'll be seeing in this, our use cases. And the cloud events community supports and then gives a lot of SGKs, like for all the language that we commonly use, Go, Java, Rust, you name it, there is an SGK support for it. And this event delivery is not only through HTTP, it is also through Kafka, I wrote they support different formats. For more details about it, you can look at the Cloud Events IO website. It is K-native that introduced us to Cloud Events, but it happened to solve more use cases for us outside of K-native also, okay? So the problems that we used to have was like, we used to have, and we are having right now also is, our organization is really big, and then there's predominant presence in the data center, our on-premises data center, and there are like various event sources. And we see a lot of redundancy in processing. So there's no sharing across teams. And it's rightly so because there is no common ESB system, and it is not possible to have a common ESB system also. So because the ESB system comes with their own domain-specific language. So everyone have their own processors and then own sources, and those source and processor tightly coupled. So with Cloud Events, what we get is like, for each event type that you saw in the header, you could build a particular processor. So then you can publish this processor throughout this company, and anybody who wants to process this kind of event type, then they can use that particular processor. And you can use the K-native broker that I just showed you. So you can dump in all the events from a different sources, even if they're not within the K-native system, even if they're outside, they can send it to the broker. And we can use the K-native triggering mechanism to route it accordingly without even seeing the data. Just by looking at the header, that's what we call this no-look routing. We can route it, okay? And the icing on cake is like, there is Polyglot SDK and multiple protocol supports, which is not only HTTP. Since this is like the simplicity of Cloud Events is enabling rapid adoption within our organization. So as I told earlier, the sources and brokers are the main parts of eventing. So we need the source to take the data from the actual source and send it into the K-native system, which is actually like the job of the sources to like, take the data, convert that into Cloud Event Format and send it to the processor or broker, right? So K-native community gives a lot of commonly used infrastructure sources like GitHub, Kafka, but within our organization, we have like sources like GraphQL, source, and then there's a lot of file processing that's happening. So we have like sources for S3, we have sources for FTP, and below I have given an example of like, how to write a source in my media article, you can take it. So the sources can be either dedicated or it can be a dynamic source as well, okay? So when we show the example, demo and the use cases we'll show like how to build the sources. So now I just want to quickly show an example or a quick demo on like how Cloud Events is sent and how the receiver function can process this Cloud Events and display it on the screen. So if you remember, this is the Cloud Events application that we built, and then it receives only Cloud Event. If you send a normal data, it will not be able to display it here, okay? Let us try to send a normal data. So this is the data that I want to send and this is the KubeCon viewer. So when I send it, it'll say that unsupported media type. So as I told, so what it means to send a Cloud Event is just by adding these headers. So I'll just add these headers and say send. And as you go, you get the data, okay? So this is the one-to-one delivery. Directly you have the source and then you deliver it to the sync or the processing function. But instead, the most sought after thing is like sending it to a broker and then have the triggers and filters do the filtering and then forward the data accordingly. So let me show you how the broker and the sync look like. So this is an example of how you create a broker. This is how simple it is. And broker is an abstraction on top of a store like Kafka or RabbitinQ, okay? So that is what is storing your data eventually. And trigger, here you can see, I have two functions, KubeConViewer and KNViewer. And the KNViewer is expecting a message type KNType and the KubeConViewer is expecting message.KubeConType, okay? And then this is the subscriber response. Instead, if you want to change it to a Kubernetes service, you can also do it as a Kubernetes. I can just do V1 and then this will be a Kubernetes service, okay? So this is already created. I just want to show you how it looks like now, okay? So now instead of sending to KubeConViewer, we are going to send it to the, through the broker, okay? So and it is sending message.KN.Type. So here is the KNViewer. If I send this message, we should show the message here. And you see the message here, okay? And just by changing the message type to KubeCon, you see it here, okay? So this is how this powerful, just adding this headers and sending it to broker and how this is how the triggers and filters are able to filter the data and send it, okay? So now we'll go back to the deck and then see some optimum use cases. Now that you saw a lot of demo for Cloud Events, let's start to look into the optimum use cases which is using Cloud Events. The first one we wanted to start with is the Infrastructure 360. As I already explained, a lot of our workloads is running in our own data center and Infrastructure 360 provides a 360 view of the application infrastructure components for a given app or a service. Like for example, if an application is using a VM, a database host, container namespaces, storage, chargeback, et cetera. We ran into some of the challenges with the growing event sources with more than 40 sources that we ingest today. And we also have to support multiple data formats and sometimes some of the overlapping formats. And there is no control over the source. So these are some of the challenges. And let's see how we rearchitected that with Cloud Events. Now, the first thing that we did was to identify the different formats and for each format, we defined the Cloud Event type and the build parser and processor functions for it. And the second thing was to build a Knative Event source to ingest the data into the broker. Now with this rearchitecture, the entire process is resilient to any upstream or downstream process failures. Let's talk about another common use case, the ETL use case. This is a very common healthcare use case. The example we have is with eligibility event data coming from various sources as listed here. As the eligibility data flows through the system, there is a lot of hops and business logic that it has to go through before consumption. But with this rearchitecture with Cloud Events, it has helped us to move from traditional ESB and ETL processing that happens with the database to even-driven functions. We now have reusable business logic functions that is shared across the team and we are also able to connect sources and processes across the organization. Now let's look at another use case with IoT. We have a lot of healthcare devices and many apps in this industry. At Optum, we support incentive programs based on devices data. This is a very good use case for Cloud Events to support various formats and the types. Okay, so with this IoT use case, I just want to show a demo of like how we could take from different data sources, parse it and show it in the DB and then maybe show it in the graph on a dashboard. So before that, there are like multiple health devices and apps, like there are like more than 40 or 40 they commonly use devices and health apps. And then a lot of healthcare companies do give benefits to the users for sending their data over. And within Optum also, so we have like various implementations and programs to give benefits to our users. So the one that we are going to show here is mirroring an actual implementation but not the actual implementation itself. And the data that we're using is simulated data. So for this use case, we are showing these three event sources alone and as I told you, there are like multiple event sources. Okay, so first I want to show how these different data formats look like. So this is an example of like how this Trava format looks like. It's a JSON format though, but you can see the data and how they give the activities are totally different. And Fitbit is also a JSON format, but the data names here and there don't even match. And the duration here will be like in milliseconds, there it's in seconds. So this is some of the examples. And then DrunKeeper, the data comes in CSV format. It's not even a JSON. So that is the different challenges. And then for other apps, there may be XML and some devices we need to have a particular SDK. So like each apps and devices have their own challenges. Okay, so going back to this use case. Okay, we have various services deployed. And before that, I just want to show, I just want to talk about this application that is showing a graphical view of the functions to point. Okay, it's called graph. And then it is developed by one of the candidate community members and an important contributor called Scott Nichols. And if you want more details about this graph, you can go to his GitHub report. And then any use case that they're showing here, all this use case are available under my GitHub report, GitHub.com slash it's Moorgapon. So you can take a look at the source code. Okay, now coming back to this use case, we have like various even processors lined up. On the right side, you can see for electrical data, we have one function. For Fitbit data, we have one function. For write data, we have one function. Okay, this is all processing function. So what about data ingestion function? For data ingestion, on the left side, you can see there are various data ingestion functions. So first one is like the Fitbit data, it takes the data and then converts it into four different event types and then puts it back into the broker for which there's a separate data ingestion. So for binding the function back to the broker, there is a resource that Knative provides. It's called the sync binding. The job of the sync binding is just to inject the broker URL or any sync URL into the function. Okay, so they can send the data over. So on the left side, it is the data ingestion. On the right side is the processing function and the middle is the broker and the broker, you can see what are all the figures that we created. So as I explained earlier, if the trigger type is held at electrical type, it's going to go into the electrical data. If it is a Fitbit type, it's going to go to the Fitbit. And all the data that goes into the MySQL, you're going to see it in the Grafana dashboard. Now it's time to run this. So you're going to see all the applications that are going to get created and the functions that are going to create it here. So on the bottom, we just triggered the shell script to start the data ingestion process. So once the data is ingested, you're going to see the functions getting triggered for various processes, like electrical Fitbit, everything is going to come up here. And then once this is completed, so you're going to see the data here. And there you go. So you have data for Strava, you have data for Fitbit, you have data for Runkeeper. So Fitbit, as you know, is the most complete thing that gives the data for most different activities. So that's about this example, the use case demo that I wanted to show. So we are taking various different data formats and consolidating and then putting, loading it into a database in common format for this kind of view or give some incident to programs. And finally, I just want to explain how the event flows through, how the event starts and ends up in a database. In this example, first step in the data ingestion process, we went to the Fitbit API, took the data, convert that into the Fitbit type and then store it in the broker. And we had a trigger to take the health.fitbit.type cloud event and then that will be processed by a particular function. And it will create four events in a inter, like one for steps, one for running, one for walking, one for electrical. And the step type alone is processed by one function. And then finally, we put it into the database for us to view in the Garshanar dashboard. So now that you saw the cloud events in action with various optimum use cases, we would like to summarize by saying why we like cloud events. The first thing that comes to our mind is you don't have to boil the ocean because it's an industry spec and binds to existing protocols and integrates with current optimum assets, be it messaging or eventing stack. Basically, it's just a wrapper around an existing event. And we are currently actively exploring serverless workflow for orchestration of the event-driven functions, which is again based on cloud events spec, open API, GRPC. By using this, we want to reduce our ESB footprint. And most above all, we would like to thank the growing community support. Thank you all. Okay, thank you all.