 All right, hello. Thank you for having me. So while we are not Eclipse, or while we're not Linux, just like Chris, I'm part of the Eclipse Foundation actually. But what we do heavily complements the Linux ecosystem and we're looking forward to working with you. So what I'm going to do, just a quick overview, I'm going to give a little bit about us since we're not as well known as some of the bigger companies aren't there yet. I'll define an AI ecosystem and you'll see a lot of relevant themes today that are very similar to other presentations within the speeches today. I'll talk a bit about open source frameworks. So like TensorFlow, PyTorch, for those of you who are familiar with it, if not, I'll give a high level overview of those. Different file formats. So kind of like how Chris mentioned earlier, I think a lot of foundations work together on different things. So there's file formats and specs at all the different foundations that all kind of work together in interop. I'll talk about open accelerators. So one thing kind of missing from the ecosystem today is Kubernetes actually doesn't work with NPJs or customer accelerators very well. It's still a very ill-defined area. So schedulers today are not very open to edge. There's a lot of work to do as kind of illustrated in the previous blueprints. So a lot of it's still kind of in progress. So I'm hoping to maybe present some newer ideas here today. And then finally, another area that's not well defined yet and I think will be relevant to LF AI people is open standards for AI metrics. And so to start, so conduit, we're having a developer of open source software. So my open source life mainly comes from Eclipse deep learning for J, but we also heavily develop standards. We work with other open source communities. I'm also on the chair of Keras. So I'm a Keras chair as well as we work heavily with the TensorFlow community in TensorFlow Java. So that's not a well-known fact, but we're actually, we actually have a pretty decent open source footprint. I'm also the author, I'm also a Riley author for deep learning. And we have a few when we're distributed all around the globe with our team predominantly in Asia. And so to start the definition of new open source AI ecosystem. So basically to start the components necessary to build, evaluate and execute AI algorithms. So AI algorithms, for those of you who don't know, are primarily based on this idea of operations. So operations can be something complex, like something in signal processing, or it can just be two plus two, subtraction, multiplication. You know, there are different hardware, different hardware implement, all these operations. And some, and there are different hardware decays from say, I've just mentioned ARM since they're here today. They have ARM and N, you know, there's other vendors like that with NKL, DNN, QDNN. And basically the, turns out the chip vendors are better than us at writing math for their own chip. And writing math code is a specialized area that only people who generally work in physics or something like that and heavily, or work heavily in CNC plus plus typically do. These are not things that normal software developers will typically touch. You know, there are frameworks for these things, which I'll cover as well. So, you know, that's your TensorFlow, your PyTorch, you know, kind of these Python level interfaces to these low level math routines. The file formats, you know, so there's, you know, we think of protocols. But, you know, we need to store our data somehow. So there's file formats out there, again, at different open source foundations. You know, you have Protobuf, you have Apache Arrow, you have things built into protocols on top of those things like GRPC. So, you know, we use these file formats for storing data, for storing models. Later today, Onyx will be mentioned as well. It's actually a Protobuf-based format, and it's a spec implemented as Protobuf. So you need to know, there's also HDF-5 for Keras as well. So, again, I'll get into all these in a little bit. Open accelerators. So, one thing that's really hard to do is actually do cross-platform math. That's not, you know, each chip vendor has their own way, you know, their own different kind of nuances for implementing things the way they do. So, different checks for different use cases. ARM-based architecture is predominantly for, you know, kind of the edge-based computing. You typically have NVIDIA and more narrow applications like TPUs by Google, among others. And you have other HPC vendors, like especially in Japan, working on their own kind of specialized AI accelerators for, you know, and they're all better at different things. Like so, if you do CNNs, it turns out you're better at computer vision, but other chips might be better at NLP. So, it's not, so there's reason, there's performance reasons and use case reasons why different chips only implement various operations or only implement optimized versions of certain routines. So, being able to have a cross-platformist decay for that would be interesting. So, something that can work with all these different things you don't have to. That's kind of what the deep learning frameworks take care of, but even then they only do half the job. So, it's still, so community efforts like the open blueprints that were discussed in the prior presentation are crucial to actually bridging a lot of things from research to production. And finally, again, metrics, you know, so, you know, we don't, we don't, I know that the general public talks a lot about bias in AI. You know, so racist models is something that comes up a lot and also accuracy. You know, eventually once AI is actually deployed everywhere, then how, how do we know what accuracy is? You know, so you have, you have a lot of programs out there, you know, LFAI being one of them that's trying to build out these standards. But there's, you know, there's also the, there's also the AI, the AI partnership like Google and Microsoft and open AI as well. So, there's, there's several different standards bodies kind of working on these things and it's still, it's still very early yet though. So, what is, you know, what is the notion of accuracy for different use cases? You know, how do we know, how do we know what a general, you know, generalized model actually looks like? How do we know something's robust? So, for example, one area, recommender engines. Recommender engines are inherently based on user preferences. So, how do you know what you consider accurate for a given user base? Netflix will be different from Amazon. You know, so for example, we have recommender engines out there. And as users, we all know that we get recommended the second you buy a vacuum cleaner, every site will recommend you vacuum cleaners. And they just, they'll say buy five more. So, you know, as users, we know that there's work to do in terms of, you know, recommender engines with, you know, with e-commerce, right? So, how do we know what we as users consider to be accurate? It's hard to define, right? So, these things are, it's still an open-ended research problem. For many different kinds of AI problems, whether it's something, you know, something like recommender engines or all the way to self-driving cars. And so, to start, so this is, this is an example of kind of the whole supply chain, you know. So, starting from, starting from the top, you have distributed runtimes, you know. So, Nvidia, for example, they have, they have their own, they have, they have, they have their own MPI based clustering stuff. You have, by the Apache Foundation, you have Apache Spark, which is typically used for different, different kinds of distributed SQL queries in ETL. And then, something, something that's not as well known, but heavily used in finance actually, real-time trading is Apache Aeron, or Aeron, which is used in a lot of, it's by some consultants out of London, heavily used in the ACA framework and a few other areas. And we use it as our primary communication layer for our distributed Spark training and deep learning for JAN. So, our framework obviously is one implementation of this. So, we focus on providing, you know, we, you know, we started as kind of this Java thing. And we kind of, and that's, that's the main, that's the main thing we dominate today. But again, we also work heavily, we also have a very compact C++ layer that interrupts with every deep learning framework out there. So, we can read and write TensorFlow, Keras, and soon to be Onyx as well. And then again, as I mentioned, the file format, so from left to right, you have Onyx. So, that's, that's kind of trying to be a neutral standard for defining neural networks. Keras and TensorFlow. Keras, even though people think it's a part of TensorFlow, it is, but it also has its own file format. Even though it's the same interface, it's actually, there's actually a higher-level file format and a lower-level one. People don't even know, people don't even know that. So, in order to fully use the TensorFlow ecosystem, you actually need two different file formats. A lot of people don't know that, right? So, it's just low-level things like this that only hits you after you wanted to pull your model. So, another thing people don't know, for example, is that TF Lite and TensorFlow are actually fairly different. It's a completely different runtime and everything. And so, the general-purpose one and the mobile-only one that our partner with Google on are actually very different. So, AI itself seems like it's well-defined, but it's actually not, and there's actually a lot of fragmentation still. There's also efforts in the compiler and IR space. So, there's one initiative at the Apache Foundation called TVM. So, it's basically an intermediate representation, which is basically a way, it's basically right before you get to assembly. It's a way for compilers to emit something that's platform-neutral. And then you basically, it's directly mappable to assembly on a particular system. There's also LLVM has something called, from Google, called MLIR, that's being contributed to the LLVM Foundation. That's meant to be kind of basically a way for LLVM to implement machine learning code now. So, and then this will allow you to target different accelerators. Both TVM and MLIR are both kind of competing standards in that space. I don't know which one will win. I don't know if there's going to be a Kubernetes or if there's going to be multiple. For hardware, you know, there's lower-level vendors such as CameraCon, Intel, and then the more mainstream ones, Arm and Vidya. The TPU, IBM Power, NEC, and then I know Fujitsu just has their deep-running chip as well. So, you know, the East Asia I think is, some of them are Arm-based, some of them are Arm-based, but some of them are just their own custom instruction sets. All these firms are implementing specialized AI accelerators, some targeting computer vivid, others targeting, you know, just time series use cases. But there's a lot of different ways to think about this. And finally, packaging. So, packaging is generally language-specific. So, you know, we have our own, you know, so as I mentioned, Google uses us for TensorFlow Java. So, we're the main way of packaging. Any C code you think of as only having a Python wrapper, we provide a Java, we provide an automatically generated equivalent with built-in garbage collection and a bunch of other things. So, we're a way of packaging. It's our format that's also open source. It's been used since 2012 for packaging Java projects. But, you know, depending on whether you go to Python, you know, you have your Python archives, you have Go, which provides you a fat binary. There's all sorts of ways you can package things, and generally it's language-specific. So, again, this is just kind of our overview. If you have questions about other standards in this space, I probably know about it, but maybe just didn't include it here. But I just want to give you, because I'm mainly for brevity, but if you have, because I know like the, I know sometimes architecture diagrams get overwhelming. So, if there's anything in particular you're going to zoom in on, just let me know later. So, starting with open-source frameworks. So, you know, right now we're kind of, we just had an unfortunate kind of consolidation in the space. The PFN folks moved from Chainr to PyTorch. So, now we're down to almost kind of a dual-opoly in the framework space. You know, with us kind of being more of a back-end framework. You know, so you kind of have two ways to build frameworks, to build neural networks now. PyTorch and TensorFlow. So, but what most, I think end-users don't know is that actually, those are actually C-based code bases. So, PyTorch is actually just a Python interface for the old Lua-based, the Lua-wrapped C code base from Torch. The same is true for TensorFlow. It's actually mainly a C++ library that has a lot of logic with Python, but all the core execution happens in C++. You have, and then you have language specific, you have server middleware, which is anything where you write a web server or things, or a system language like Go, like Kubernetes, for example. Kubernetes is written in Go. So, you have a lot of middleware, Java, Go and C-sharp kind of in the enterprise space. So, generally, these frameworks will have SDKs for using models from those languages. We provide one, like I said, we provide one for TensorFlow, for example, in Java. And then finally, you have the, you know, we're actually, most of our developers, even though we look like we're Java, we're actually, like, again, most of our developers are actually C and C++ people. So, framework development is specialized, costly, and generally requires a breadth of expertise to get right, and to implement right to bridge to middleware. You know, in this case, to things like schedulers, you know, like Spark and all that, it requires a very specific and specialized team across multiple disciplines to work well. So, sorry, just backing up a little bit, ETL frameworks. So, I mentioned Apache Spark is one. There's also more traditional ones for commoner data like Pandas and Dask. So, you know, so there's lots of ways to run what I'll just call distributed jobs, like run a large SQL query on my S3 bucket. There's lots of different ways you can do that. Those are some of the popular ones in Python. And schedulers, so Kubernetes is obviously the Linux foundation. Ray is fairly new. Ray started as mainly used for reinforcement learning. It's built by what used to be Ampliab, which created Spark. So, if you're not familiar with Ray, it's kind of open coming. It's not in a foundation yet. But the Linux foundation might want to target this one. Just throwing that out there. But in general, I think it'll be, you know, it does have an open governance model, and I think they will be, I think they're figuring out what to do with it. It's almost out of the lab now. Yarn, so that's kind of the, Yarn and Mesa is kind of the old school Apache foundation. You know, that's kind of the legacy Hadoop days. They're still around, even if they're not growing, they're still around, they're still used. Sometimes you run, you know, sometimes you run these things on top of Kubernetes or you interrupt with it. It just depends on the use case, it depends on what's there. So machine learning frameworks, so not deep learning. You know, there is more than deep learning still. You know, just the more standard models that are actually still be used in the industry, like random forest and things like that. So you know, SK Learn and XGBoost, deep learning frameworks, TensorFlow, Keras, PyTorch, deep learning for J, and then serving middleware. So there's a lot of open source serving middleware out there. So there's Celna, there's TensorFlow, there's TensorFlow serving, and then it's kind of serving as well. So there's a lot of open source kind of serving middleware out there. You know, there's different ways of implementing this. And it's basically taking a model and exposing it as a REST API or something over a network. That's still, it turns out, you know, it turns out it's a hard problem, especially if you're, say, deploying at the edge. So for example, serving middleware doesn't run well on edge chips. And generally these things also don't, generally these things only bundle other libraries. And you don't generally have integration with the serving middleware plus a custom accelerator. That's actually an edge case yet. But if you want AI to be deployed for optimal speeds and minimal latency in production, it turns out that you have to do more than trick the models. You actually have to integrate the high-level middleware that developers talk to. And so that's still kind of an open-ended problem in it. That's actually the main thing we work on. It's just that part. Okay, so here's an example of a lot of different file formats. So again, there's a lot of different foundations where all this work kind of spans. You know, you have Apache Arrow. So again, this is an in-memory file format for just data interchange between, say, Python and Spark. So that was the use case this was developed with. But basically if you've ever heard of a data frame in R, or even if you just know an Excel spreadsheet is with columns, with columns and then add types on top of that, you have Apache Arrow. It's an interchange format. You have Arrow, Parquet. Again, those are both helpfully developed in the Hadoop because systems work as well. So these are all different ways of storing data. Sometimes they're used as configuration file formats. Portabuff is an example of that. So Onyx is Portabuff-based, developed by Google. And then JSON, I think we all know what JSON is. DL frameworks, so it turns out, you know, there's actually a lot of different ways you can save models. So there's Portabuff-based file formats. There's HDFI, which is Keras. And then you have another one now is Flatbuffers, built by the guy who did protocol buffers. So there's N plus 1 file format standards, each better than the last sometimes. Some use for legacy reasons. So Onyx, for example, picked Portabuff just because it was widely adopted, not because it was the best one to use. And then you have other lower-level communications on top of those file formats. So in this case, GRPC and Aeron, as I mentioned before. So accelerators. So, you know, you have basically, so you can imagine application developers taking an open standard like Onyx and then wanting to run it on a target chip. And then that chip will have their own optimized math, the way of running certain kinds of, you know, maybe convolutional neural networks, computer vision, RNNs for time series, you know, or whatever optimized libraries might be present on that chip vendor's chip. So what you might want to do is you might want to have a cross-platform middleware that knows the difference between what runs well on Intel and what runs well on ARM. So that way you, as a developer, can just output, say, an Onyx model and then know you're going to get high performance on whatever chip. You know, one of the biggest, so the reason I bring this up is because one of the biggest problems of most, I think most people aren't familiar with this today, and the Onyx people should talk about is this idea of op coverage. So as I mentioned before, certain chips will only implement certain operations because they're targeted at certain use cases. So whenever you hear about TensorFlow, TensorFlow actually itself is a fragmented thing that runs differently on different chips. Most people don't know that. So for example, if you run on the TPU, they actually implement only a small subset of TensorFlow. TF Lite is a small subset of TensorFlow, which is for mobile, right? So chips don't always implement the latest architectures either. So for example, if you go to use QtNN from NVIDIA, they don't generally have the latest CNN architectures either. So this is a fairly common problem in the space. So these highly optimized libraries are generally behind the cutting edge. So it's hard to build something general purpose, and it's not their fault. You have a lot of time being spent on these things, and so you need to pick carefully what you put in your roadmap to optimize. So having something general purpose that knows what to use where can be really helpful. And so this idea of integrating these things is actually still a hard problem because you kind of have a lot of things happening in the lab in Python, then they build the model, but then it turns out that without a lot of extra pre-processing, so for example, TensorFlow, in order to actually deploy a TensorFlow model, there's actually a lot of pre-processing you have to do. Most people don't know that. So you actually need to massage what we call massaging the graph, so to speak. The graph is kind of the instruction set, so it's like you need to add and subtract and multiply. That's a graph essentially. It's just a list of operations that you need to run in order to perform some target operation in AI, as we call it. So in this case, it turns out when you call tf.save or keras.save, there's actually a lot of stuff in there and actually it won't run. So just because you can save a model doesn't mean it'll run properly. So like I said, there's a lot of noise. And then depending on if you want to run in a different chip or not, you might need to quantize a model. You might need to post-process it in some form. So that post-processing work is a black art. To most people who build the models, that stuff is a black art. And it's not clear what will work in what scenario. So sometimes you might have one model, you build for one scenario, and then a completely different model but what you're doing is you might make accuracy and you might make accuracy trade-offs and then just so it can run. Again, most people don't know that. So that integration problem and actually making AI models run, especially at the accelerator level, it's not a well-solved problem yet. Most people just... So anytime you see an AI vendor, what they do is they just use other people's software and then wrap it in Kubernetes. That's what most people do and they say they did their own thing. Actually, they just bundled a bunch of stuff and they don't know how the underlying silicon works. And that's actually a big problem. If we're going to deploy AI models at the edge and actually go after this ubiquitous computing problem, you need to know how this stuff works. You can't hand-wave it. So you need to work with the community. You need to work with a lot of these efforts and that's essentially what we're trying to do. We're just trying to solve the integration problem. This is the only thing I do. So if you have any questions about that, please do let me know. And I'll end with metrics. So these are Linux Foundation projects. Some of these are Linux Foundation projects or widely used open-source visualization-based tools. So one thing I want to say is depending on whether you're implementing different kinds of model accuracy dashboard, a model bias dashboard, whenever you do that, you typically want to use open standards. So it turns out Prometheus and a lot of the things in the Linux ecosystem are actually very good for this. And so we typically think of these for DevOps. But what if you have the same kind of DevOps-style dashboard for modeling accuracy and bias? And then what if you data scientists could then use that output and say, okay, this is the real world version. This is how well we're doing. Otherwise they would never know. Because data scientists, one thing out there is they want to build the latest model, but it doesn't necessarily mean it's going to work in the real world. Machine learning models, they also have technical data. The world changes. What do you do when that happens? So a small example, that would be fraud detection. So fraud detection changes every day. So what do you do about that? You might build machine learning models to detect this kind of fraud, but you almost always need to retrain the models because, again, the world changes. This is called one way to describe this. This is something called concept drift. So you have the real world as one concept, and then what your model thinks is the real world is another. When those two things differ, how do you know you visualize it? And with that, I'll end and I'll take questions. Thank you.