 Hi everyone, I'm glad to kick off the first in person lightning talk. This is by far the largest room I've ever given a talk in and yeah. So my name is Kevin Hannon. I'm an open source software developer at G research and my goal is to get G research to contribute support and also help provide help in the open source community. We also have an open source project called Armada and happy to stay in 2022. We are part of the cloud native computing foundation and generally the topic of my talk is how we can combine another open source project like airflow with our project Armada, which is a multi cluster Kubernetes solution. So what is G research? Well, G research is a financial trading firm based out of London. It is not Google research as sometimes some people say, but we are they are a trading company that uses math sorry, as an American, that's hard to say and computers to try and predict how the stock market is going to behave. Now, what does that really mean? It means they're running a lot of data engineering workloads, machine learning, spark really anything you can think of. It's running in our farm. We use a lot. We are heavy users of Kubernetes and unfortunately, I hate to say it, but Kubernetes is not perfect. There are cases where it can be improved, especially around batch compute and generally our users that G research love to submit lots of jobs like I'm talking in the order of millions of jobs per per day is what we're supporting. And if you try to do that on top of a single cluster Kubernetes, you can be in kind of a bad day. Some cases we have as a former grad student, I remember cases where if you don't have any kind of queuing solution in your cluster, you can have one guy just take over the entire cluster for months on end and you're hosed if you're a grad student trying to get time on your on your computing. And that's the same thing at G research. We have some quants that really want to do their research. And if you don't penalize them for using all the compute or using all of it, then you are kind of leaving other people in the there. You're they're struggling a little bit. So generally, what is Armada? Well, Armada tries to take this approach of of horizontally scaling Kubernetes by adding we can we can add new clusters to our our farm by just adding what we call Kubernetes worker clusters or executors and generally users interface with an API that goes to what we call the Armada server. And then that leases pods to Kubernetes worker clusters. And then those are what actually run the pods. I like to think of Armada as a serverless platform for running large scale really any kind of job. So our users don't really need to know about things like Kubernetes or what cluster they're actually running their jobs on. They just want to point it at a CLI and say run. And also for the API's, we made we know that our users love all different kinds of programming languages. We have support for .net, Python and go. And that's and these are a proto defined API so we can generate clients in other languages. Why do we build a Python API? Well, for one, Python's great. The other other the other reason is we wanted to start using workflow engines like Apache Airflow. Like you can think of Armada runs compute. It runs a single container or a single pod. And then you want to couple that with a workflow scheduler. You want to build DAGs. You want to and so for those of you that are not familiar with Airflow, Airflow is kind of like this workflows as code. You define a Python file where you have a series of execution tasks that you want your jobs to run in like in this code snippet. I have a hello world followed by an Armada job and Airflow really thought long and hard about what it's like to extend their platform. And they have a lot of they have hundreds of providers that you can do. And one of the reasons why we even we looked at Airflow is because we don't run like we really like Kubernetes, but there are cases where Kubernetes may not make sense. You may want to run on a SQL server somewhere where in there might be cases where we had want to point Airflow to run on a database and then feed it back somewhere else and run Armada and vice versa. So generally, Airflow is pretty cool. Armada is pretty cool. And we want to combine them and be able to get our users to use more open source tooling help contribute to the open source projects and really start trying to think about how we can really make our users lives better. And fortunately, that is the end of my talk. If anyone is interested in this, we are as part of the CNCF fan box. And in general, we have a booth in the solutions showcase in hall five booth K 21. Please stop by. We have a nice I created a nice demo for anyone to have any questions about Airflow. Thank you.