 Next up, we have Brian Granger. Brian comes to us today as the senior principal technologist of AI platforms at AWS. Some of you may also know him as a professor of physics and data science at Cal Poly. Brian is going to talk to us about accelerating AI and ML using open source. So please a round of applause for Brian. Come on up, Brian. Hi, everybody. Great to be here today. So, yeah, I want to talk to you about open source software in AI and ML at AWS. First, a bit about me. In the introduction, you got a little bit of this. I wanted to add a few things. I'm one of the co-founders of Project Jupiter, and I've been working in open source for the last 20 years, involved in a number of different projects along the way. And let's dive into this. So as we all know, it's an exciting time to work in AI and ML. The rate of change, I think, is far beyond any of us could have imagined we'd find ourselves in at this time, and that makes it really exciting, but also really challenging. So what are customers telling us at AWS? One of the things they're telling us is that they want to use popular open source software for AI and ML. They view open source software as a source of rapid innovation, and they want access to that innovation. So from PyTorch to Jupiter, Pandas, NumPy, Spark, LangChain, Ray, Dask, and many other open source projects. At the same time, customers are telling us that they have challenges in using open source software. It's difficult to keep up with fast-moving open source software. It's difficult to configure, deploy, and operate open source software. We find that customers want to use the open source software with the AWS pillars of operational excellence, security, reliability, performance efficiency, and cost optimization. And they also want confidence that the open source software they're using is sustainable in the long term, stable, and continually innovating. So I want to talk to you about two strategies we have for addressing what we're hearing from customers. The first is that AWS and SageMaker specifically offer AI and ML services that bring the value of popular open source software to customers to address these needs and challenges. Some examples, PyTorch. We offer integration with PyTorch in SageMaker training jobs, SageMaker inference endpoints, the SageMaker distribution, which is a package of open source software, a distribution for ML, and SageMaker hyperpud. For Jupyter, we offer Jupyter integration through SageMaker Studio and SageMaker Studio Lab. For Spark, we have Amazon EMR, AWS Glue for Spark, and you can use those things also from SageMaker. The Transformers library from HuggingFace, we offer integration with HuggingFace through training jobs and inference endpoints, and there's many other examples of open source projects that we've integrated into our products and services to serve the needs of customers. But there's a second strategy that I'd like to focus on in this talk, and that is AWS is also participating in and contributing to open source. This part of it is sort of deep in my DNA as a professional, and so it's really fun to talk about this. One example is AWS is a founding member of the PyTorch Foundation and has worked with Meta to create the TorchServe project in 2020, which is an inference server for PyTorch. And in this talk, I want to cover briefly some of the recent contributions we've made to Jupyter, Langchain, and PyTorch. So let's dive in. Project Jupyter, you may be familiar with the Jupyter Notebook. These days, it's a foundational tool in AI and ML. It's a multi-stakeholder open source project, and AWS has been involved with Jupyter for a number of years. We have a team shown here that works full-time on open source, employed by AWS, and Jupyter is one of their key areas of focus. I want to talk about two particular contributions we've made recently. One is Jupyter Scheduler. So Jupyter Scheduler is an open source extension for Jupyter that enables users to run and schedule notebooks as jobs. We're finding a number of different usage cases for this. One is reporting, running a notebook on a regular basis, and producing an artifact that humans can then read as a report. It's a lightweight way for people to move notebooks to production. It's good for experimentation, where you want to run one notebook many times for different parameters, and it's useful for scaling out, where you've been working interactively in a notebook, but you want to run that notebook then on a larger instance type. So Jupyter Scheduler is governed by Project Jupyter and was contributed by AWS. So if you're using an installed Jupyter Scheduler, there's a number of ways to quickly run a notebook as a job or schedule that notebook. There's flexible job options. You can configure parameters for the notebook, pick instance type, conda environments. You can both run the notebook immediately, a single time, or put it on a periodic schedule, and it's an immutable snapshot of the notebook so you never doubt what version of that notebook was run. It's easy to manage jobs. You can download and view outputs of the run notebooks or the logs, delete jobs, re-run jobs, and then edit the schedules. Most importantly, Jupyter Scheduler is extensible. The default backend for Jupyter Scheduler just runs on the system where Jupyter is running. That would often be your laptop. Amazon SageMaker offers a backend for Jupyter Scheduler that runs these jobs on AWS. That backend is also open source. The user interface is extensible, so these extensions or these alternate backends can offer different job options that are sort of specific to the backends. And it's also not specific to just notebooks. A given backend can declare which file types it's capable of running. Next, Jupyter AI. As we know, there's a lot of work being done on generative AI and a lot of interest in accelerating the work of data scientists and machine learning engineers and practitioners using generative AI. And Jupyter is diving into this with Jupyter AI. And there's two primary use cases here. One, using notebooks as an AI playground, and the second is using AI as an assistant as your writing code and working in Jupyter. Again, Jupyter AI is governed by Jupyter, contributed by AWS. Some of the design principles, and this really follows the design principles of Jupyter itself. It's vendor-neutral, so there's a growing list of model providers that are supported in Jupyter AI. It's transparent and traceable. Everything's open source. You can look at the prompt templates we're using. You can customize prompt templates. You can bring your own models to this. It's collaborative. Oops, went too fast. And also, as I said, very configurable and extensible. The first thing we offer is a double percent AI magic command that turns Jupyter notebooks into a generative AI playground. Any of the providers, any of the models that we support, you can do this with, and we support the rich output of notebooks. So for example, here is Claude generating Markdown, and we're able to render that Markdown in a nice form in the notebook immediately. Same is true for models that generate images. We're able to render those immediately in the notebook. What's nice about this is you no longer have to take screenshots of the various model playgrounds to share with your colleagues. You can actually share a reproducible Jupyter notebook with them showing the explorations you're doing. The second thing that Jupyter AI offers is a chat UI. This is a side panel. You can pick which model you want to use for the chat UI, and this allows you to have a generative AI assistant as you code. You can select blocks of code in notebooks and text files and choose to include that in the context of your request. So you can say, pick a cell. What can you tell me what's wrong or help me debug the code in this cell? You can take code snippets that the models generate, insert them back into notebooks easily. It's also collaborative. So as of Jupyter Lab 4.0 and Notebook 7, Jupyter has support for real-time collaboration. So you can deploy a Jupyter server on common infrastructure and multiple people can log into that and share notebooks and edit the notebooks and view them at the same time. What's nice about this is if you're running Jupyter AI on a server like this, the chat also becomes collaborative. This is just the start. There's many ways that we are finding that we can improve this, but part of, I think, the challenge here is we're all still trying to figure out what are the right patterns for us to interact with generative AI assistants and how does that interact or interplay with the human collaborations we have. Right now, a lot of the pattern is that we'll all interact with the AI individually and then interact with each other as humans as if nothing happened. What we're finding with Jupyter AI is that's really challenging, partially because of transparency reason, right? That traditional model or that model, I think that is the default now, doesn't do a good job of showing other people what artifacts that you're bringing to the table were generated or edited or contributed by a generative AI. In Jupyter AI, we're trying, as I said, to increase transparency so that when code is inserted into a notebook, we're tagging the metadata of that cell with a model that was generated. Again, transparency is part of the picture here. Jupyter AI also has slash commands. There are slash learn and slash ask slash commands that enable you to ask questions about your local files. Let's say you checked out a repo from GitHub and you'd like to get a sense of what is in that. You can say slash learn that local directory on your hard drive. We will go through, parse all the code, parse all the markdown files, call the embedding model of your choice, and store it in a local vector database. I should note, we support many different model providers. Data privacy we're finding is really important to users. If you want, we also support local models. You can do this with local models and your data will never leave the system where Jupyter AI is running. Once Jupyter AI has gone through and indexed all this content, you can use slash ask to then ask it questions about that content that you've indexed. The slash generate command allows you to generate an entire Jupyter notebook from a natural language prompt. This is an example shown there of a notebook that was generated. I guess here the prompt was a notebook that explains how mapplotlib works. We're using Langchain underneath the hood to break that down into a set of subtasks to have it generate an outline for the notebook and then fill in the code for each of the sections. It's not perfect. It depends a lot on what model you're using, but we're finding that it's highly useful and it enables you to go from zero to having a lot of code written very, very quickly. We're really excited about the opportunities for things like this. In an upcoming release, you can add your own slash commands. One thing we're finding with both learn and ask and the generate command is that users want to plug in their own sources of knowledge and their own language models. It's not that you're going to have one vector database that you want to ask questions over. In a large organization, you might have dozens or hundreds of different vector databases or other sources that you want to retrieve content from and dozens of models. One way we're allowing extensibility is through slash commands. If you want to have a set of slash commands that talk to your models and your sources of knowledge, you can do that. We're also working on natural language routing. Even without a slash command, we will be able to route commands based on the natural language prompt to the appropriate vector database or model behind the scenes. Again, all this is extensible. All of its open source come and work with us on this. We're really excited about this. I'd also like to talk briefly about our contributions to LangChain. LangChain is a recent but has become a very popular open source tool for building applications with language models. It provides a full set of abstractions for building applications. It's become very popular with customers of AWS who are building LLM applications using AWS services. We've been contributing since the very start of LangChain just over a year ago. Here's some of the integrations we've built with LangChain. You can use any of the models that you can deploy or use on Amazon either through Bedrock or SageMaker Endpoints as part of LangChain. We've also built integration for Amazon Kendra for Enterprise Search in RAG, OpenSearch, RDS, MemoryDB for additional vector surf, Neptune, GraphReachEval and then integration with TextRack, Comprehend and Personalize as well. One last thing, a more recent contribution that just came out at Reinvent a few weeks ago is a new Amazon S3 connector for PyTorch. A lot of the data that all of you are using to train models or fine tune models is on S3. Because of that performance and your ability to load that data quickly, in addition to save and load model checkpoints is absolutely critical. This was launched at Reinvent 2023 at the beginning of this month. It's a new S3 connector for PyTorch. This is based on a new runtime that is being built for high-performance, read-focused access to S3 at the core. There's a Rust library. It's wrapped up into Python and then adapted to the PyTorch usage cases. So for loading training data from S3 you can load data up to 40% faster than you can through other approaches to working with S3. It offers both iterable style and map style data sets. It's compatible with Torch Utils data and also Torch data. And then for model checkpointing you also see a similar up to 40% performance improvement for that. And importantly you don't need to save your model checkpoints to the local disk. You can go straight to S3 with this. So to wrap up, open source software provides a significant value in AI and ML. Rapid innovation, open collaboration and best practices and open standards. There's many many opportunities to participate in open source communities and contribute. And now is a great time to get involved and join us in working in these open source communities. Thank you very much.