 You are in Santa Rosa, you have two hours more by car to get to know our first speaker that it's probably one that you already know. Paco Nathan is here again. He has been in big things conference seven years, so he's a veteran. Hi Paco, how are you doing? How was it going in California? Oh, wonderful. Thank you very much. Paco, I know, well, we all know this is not an easy year for anything, but I think for cider is not that bad. How's the gear going for this cider business you are making there? Oh, fantastic. If I can turn the camera, I'm actually in a cidery right now. It'll be hard to see, but our crop is maybe not as much yield, but the flavor is really good. Okay, okay, so it's not everything bad this year. Exactly, yeah, not it. This is not the only thing going on in the States nowadays. Do you already have a president? Are you working on it? Any news on that? Oh, it's painful. We're trying so hard, but we need to learn some lessons from elsewhere in the world, I think. Yeah, okay. You will have, you will have. So, Paco, thank you again. I think this is, as I have told, this is the seventh time you are here with us. You are by far a veteran, but of course you will be here next year. Then there will be even much more fun because we can be here, have a beer, have some jamons, some tapas, and all this stuff that I know you love. I miss being in Madrid. Yeah, yeah, yeah. So, this is a good reason for having to come back next year again. So, Paco, welcome again, and thank you, and it's your time. Thank you very much. Let me cut over to Zoom here and presentation. Great. Thank you so very much. I'm honored to be able to present here at Big Things, and I miss being in Madrid. I look forward to hopefully next year we'll be able to be in person. This talk here is about AI. Oh, by the way, I should mention, if you want to get the slides, they're already online at this URL. This talk is a kind of study guide, and it's really for a business audience, although it is about technology. It's about how to get near-term leverage of AI. What is, what are the real crucial points about AI adoption in enterprise specifically? I'm going to draw some fronting history. There's a lot of different primary sources, different links here. If you want to study them in more detail, it may take a while, but hopefully we can cover this in a few minutes and leave you with something that you can look into more detail if you find parts interesting. Now, I want to go into a little personal history. About 15 years ago, 2005, I was doing some work, and some colleagues called me up and started throwing ideas at me. It was something new. It was something that turned out to be called AWS. And so when 2006 came around and they launched, I was pointed toward where to go to sign up. So it was one of the really early people working with AWS. And as luck would have it, I was also recruited that same time to be a technical co-founder for a new AI startup out of Los Angeles. It was called Head Case Human Manufacturing, and we were doing conversational bots in Second Life and VR. So my team, my technical team became a guinea pig for evaluating AWS to run machine learning. And I went to our VCs, John Callahan at True Ventures, and I said, there's a new thing called cloud. We need to go all in 100%. And he looked through and he agreed. And so we got approval to be 100% cloud in 2006. Now, we were running machine learning workflows, and so we looked toward a new open source project which had just been released. It was called Hadoop. So we began running Hadoop in the cloud in 2006 very early. And through this, I got to meet some wonderful people. One of them, Andy Jassy, who leads AWS at Amazon. We got to do some talks together. I got to be a reference customer for some of his strategic sales later on. While we were doing this, we found a bug in Hadoop. It wouldn't run efficiently in the cloud. So we hired a young consultant out of London. His name is Tom White. He committed a fix. It allowed Hadoop to run efficiently on the cloud. Tom became famous for writing the book Hadoop, The Definitive Guide, and also for his great work at CloudR2, of course. But AWS called up. My friend said, hey, you know, you're the largest Hadoop instance in the cloud. We need to talk. And they wanted us to be a case study. So for the next year, we worked on something that was later released as elastic MapReduce. Because what we were doing, it seemed that other people wanted to do this too. A year after that, I got asked to help out at UC Berkeley, the university nearest here in San Francisco. David Patterson is a very famous professor there. He helped to create raid. He helped to create risk. And he had written a paper about cloud. He was going to formally define what was going on with cloud. So I critiqued the paper and then gave a guest lecture at Berkeley. And in the audience were the founding team for Spark, Databricks, other products too, really famous. It was very fascinating to be able to get to meet these folks early. But in particular, Jan Stoike, who led Databricks, Jan Stoike and David Patterson had focused their graduate students to make formal definitions of cloud in 2009. And if you haven't seen this paper, it's actually really interesting to look back because they were able to articulate early what was happening with cloud. Patterson had developed a methodology for industry research. And also a pattern language to describe distributed systems. And they interviewed a lot of people, Amazon, Google, et cetera. And they came up with what would turn out to be how cloud rolled out for at least the next five to 10 years. And the graduate students, of course, jumped on this. That's what became Apache Spark. Now, 10 years later to the day, they followed up with another paper. And so that was early last year. Eric Jonas was the post doc who was really lead on that paper, but also David Patterson and others. Eric Jonas had noticed that at the lab that created Apache Spark, less than half the students had ever even run a Spark job. And in fact, for the machine learning students, it was more like single digits in percentage. He was also noticing, Eric Jonas, what was evolving in terms of cloud during its second decade? So we have decoupling of storage and computation. We have the abstraction of executing code as opposed to allocating resources to execute code. But most importantly for business, this part, the idea of paying to execute code instead of allocating resources in advance. So if you are an investor translated, this would be that you could place an immediate trade and buy some shares in a stock, as opposed to having to go out and buy a futures contract. It's a very, very different way to approach business. Also a very different way to approach technology. So we're saying that cloud is evolving now to be more economically savvy. And really this was possible 15 years ago, but it had to kind of be dumbed down for a while to make it recognizable for people outside of Amazon. Translated, the impact means that we will see a lot less emphasis on frameworks, different layers of frameworks, more emphasis on libraries, programming, software engineering. And so if you read this paper, you'll find out what they're saying. But there's a lot of layers of frameworks in between the storage and the memory that have come into play over the last 15 years. These are being dismantled systematically. The cloud vendors will push this. It has a lot of business implications. It fundamentally changes our notions of data engineering. And I'm not saying that this gets rid of data engineering, although it will undergo a dramatic change. And that change could be likened to say the shift from doing mainframes for IT to where we are now where we run Linux virtual machines. Very dramatic. Now, I'm not a data engineer. I've been associated with a lot of the history of this field over the past couple of decades. But when I get customer inquiries, I refer them to my good friend, Jesse Anderson. He is the maestro of data engineering. He has a new book out, which is highly recommended called Data Teams. Now, what Berkeley has learned and what they've been pushing for the past few years based off of this research is something called the Ray Project. And this is a library, open source library, either in Python or Java. It is a kind of pattern language for creating distributed systems. Think of it as building blocks that you can mix and match. So there's test parallel and actor pattern and parallel iterators, all these things. What you need to know about that is that it includes a very rich language for describing sort of this grammar of how to build distributed systems. And what this does is to alleviate the tight coupling of your applications with the underlying frameworks. Again, the layers between memory and storage, those layers are disappearing. And now it's becoming more a matter of something that you would program as opposed to having to necessarily buy a framework. There's a great talk by the CTO from any scale and co-founder for Ray, Robert Nishahara, one of my colleagues. He's talking about the adoption where we see it at very large use cases such as end financial in Beijing, JP Morgan in New York City, doc chemical, et cetera. And also just how much this library is moving into what else we use for AI in terms of PyTorch, TensorFlow, Hugging Face, Spacey, Dask, Selden, et cetera. So I want to emphasize, check out Ray, there's a lot going on. Now one of the drivers for why this is happening is because of hardware. And I also want to say something maybe controversial, but I've said it here before, hardware is moving faster than software. Software is moving faster than process. In IT we are taught it's supposed to be the opposite, but it's not. So when we go back to 2005, say about the time when MapReduce started coming out, what we had then were commodity hardware in Linux, cheap Linux boxes with spinning disks, but they served the needs for big data. And that was mostly about working with log files, analyzing them, aggregating them. And that used the kind of pattern that's called task parallel. That's for every piece of data is relatively independent, just run them all at the same time. That led to things like Hadoop, that led to things like Spark. Now, about four years later, the patterns became more complicated, the workloads had more demands, so actor pattern you see being used in Spark, Aka, and others for a lot of data science workloads. And then about four years later, it got even more complex where we had introduction of deep learning in the industry, and that placed a lot more demands on the software and the hardware, much, much more so than what we had seen earlier with Hadoop. And then it becomes more of a matter of differentiating gradients. So you have data, you want to piece apart different cases, differentiating gradients within the context of network data. And so the kind of AI workloads that we have now, they must leverage these really complex graphs or networks, what we also call tensors when we represent them. There's an excellent talk, if you want to look into it, is by Jeff Dean speaking at ACM in 2013, and he's describing the factors within their latest data centers at Google that were driving the design of something new. And of course, what he's talking about is TensorFlow, it was not named that just yet. But this was about how to optimize to train networks, neural networks, deep learning, working with graphs and tensors. It's an excellent presentation. So to summarize, if we go back into the 1990s, when we worked with distributed systems and a lot of data, a lot of that was with something called MPI. And it was very ad hoc. I've done MPI work. It can be very difficult to use, but it was what we had. By the mid 2000s, we had MapReduce, Hadoop, and then Spark. And that leads to a different kind of work you can see in the center picture. But what we're looking at now, and really since the last seven years, is a completely different world, one which requires a lot of work with optimizing networks, with graphs. And it requires a very different kind of topology in the data center, very different economics for how to leverage this. And so you see this progression from MPI through Hadoop Spark now into Ray. One of the interesting things that's being described about Ray about this kind of world is what's called infinite laptop. And so I will ask a question. When you are working, is your focus of attention on your laptop or your mobile or desktop? Or is your focus of attention somewhere in the cloud? Because I imagine you're probably using both. This idea of infinite laptop is that you have a blend. You do focus on your laptop, but the resources can spill out into the cloud. And it doesn't have to force you to change and shift your focus. There's a really great demo of this by Edward Oaks, if you look at the YouTube link down at the bottom. But this is what Ryze Lab means when they say serverless will dominate cloud over the next decade. Now, there's a lot of precedent for this. If you look in design theory, I spent a little time in design school, I love design theory. If you go back into the mid 90s at Xorex Park, you had John Sealy Brown, Mike Weisner talking about calm technology, something which only now 25 years later is really starting to catch. But the idea is that we do not have to shift dramatically between a center and the periphery. Instead, we can design such that we keep focus as humans do. I mean, we're hunter-gatherers, so we like to have attention. But we also like to check our periphery. The analogy here is that the laptop is the center of your attention. And whether you're working hybrid cloud or multi-cloud, that is your periphery. How can we combine these? As John Sealy Brown said, the key is to honor and amplify the interplay between the center and the periphery. And that's what's going on with Ray, an infinite laptop. Okay, so let's talk about AI, where this gets used. And Ray provides a much needed kind of control layer for very sophisticated distributed systems, optimizing workloads that leverage hybrid architectures. But it really keeps in mind about the economics of computing. Young Story and I talk about this a lot in terms of how to teach people to use it, keep the economics in mind. When people talk about machine learning, don't be scared by the equations here, but when people talk about machine learning, usually they're talking about supervised learning. And supervised learning is just one of several different areas of machine learning. But it involves understanding data from the past, generalizing the patterns from the data in the past, so that we can try to predict events in the future. And that's really what supervised learning is focused on. It's a kind of system one thinking, if you will, coming from behavioral economics. But the contextual horizon of making these decisions is typically measured in milliseconds, very fast kinds of decisions at scale. And that's great for e-commerce, but it's too shallow for the kind of extended decisions, long term decisions that organizations must face. I love this talk from Ferro Bostic talking about the cult of prediction. This idea of how, okay, we know how to use supervised learning, but it's not just a one trick pony, we shouldn't overindex on it. We shouldn't expect to have a crystal ball that in business, the only way we use machine learning and AI is by leveraging prediction, because there are other ways. There are ways to make better decisions without having a crystal ball. Recommend this talk. So now we're moving into a slightly different world where reinforcement learning is gaining a lot of traction. I've been working in this lately. And this is over and beyond what you see with supervised learning. And I want to try to contrast that. The idea with reinforcement learning is that you're creating policies. You're learning these. And in fact, you can learn more and update even as you're using the policies. There are ways to do that called the Explore Exploited. But the idea is it's really more about long term decisions. Okay. And leveraging policy where the contextual horizon for your decision could be very long periods. It could be continuous operation in a factory. If you talk to auto manufacturers in Europe, they're doing this. This leads toward maybe a horizon of months or years in terms of making decisions. And this is very important because we've had the basis for this optimal control theory for decades ever since the work to make airframes much more stable in the 1950s for aircraft manufacturers. What's happened lately is that we've taken that optimal control theory and augmented it with deep learning so that we could learn a lot of different edge cases. This is being used in supply chain and factory automation and finance and other areas. If you want a really good example, I would point to this from Salesforce this year. It's called the AI Economist. And I'll tell you from my personal work in AI, I have worked with the people directly who manage the economic data for agriculture in the United States at the federal level. And I've seen what they do. And frankly, the data is too large. It has too many dimensions. Humans can't make decisions about economics at that scale. But this is something where AI is actually really good. And so here's a great example of taking reinforcement learning and looking long-term at the effects in many different cases of different kinds of tax policies and how can we learn from them, iterate on them to be able to have much better trade-offs. And my hunch is that in your business there are also policies that you develop that you need to leverage. This is a good way to approach it. Now one of the downsides though is that reinforcement learning requires even more compute and more data than deep learning alone. And so the question is will this boil the oceans? There is a stark reality and I'd point to an excellent talk from Neil Thompson being interviewed by my colleague Ben Lorica. Neil Thompson is at MIT and he's been doing a lot of work on the economics of deep learning. And the point is this that there's been a lot of over emphasis on creating extremely large models, models that might have hundreds of billions of parameters. They require a lot of data. They require weeks on a private cloud to be able to run. And they have an extremely large carbon footprint. These things are essentially doing overfitting as an art form, as Thompson puts it. But most importantly what he's seeing is that in terms of even the really amazing work on custom hardware right now, we're reaching a point of decreased marginal returns. So that idea of just take a mountain of data and create a really large model. We're hitting a wall. Okay, we cannot do just that alone. And it has severe climate implications if you actually run the numbers. Highly recommended. But there is a path forward. I will point to Willie Sadler at Manchester. I got to be in a track with her last week. And she's pointing to how there have been applications of AI in more knowledge heavy areas that have become mainstream rather quietly. They're not exactly at web scale yet, but they're getting there. On the other hand, machine learning data driven models that's really exploded is extremely powerful. We talked about supervised learning, but it is reaching some boundary conditions. So what we really need to do to avoid boiling the oceans is to leverage both the data driven models with the domain expertise that you have. Do not rely on just having enormous models. And there are good indications. There's something called graph embedding or graph AI, which is being used with knowledge graph. And it's a very good way to marry the domain expertise and structured knowledge along with deep learning. I'm doing a tutorial about that in a couple of weeks. Highly recommend her talk here. Now, my colleague Ben Lorca and I go out and do industry surveys. And then we summarize and reports. We've done several over the past few years. The most recent one is the 2020 NLP industry survey. There's a free download. If you want to look for more information on that, I've got to talk about it too. I gave it last month. But one of the things we found is that the number four top use case for NLP in industry now in enterprise is support for knowledge graph work. And that was startling to us. So the idea of knowledge graph is if you have linked data, you can leverage semantics, which are basically shared definitions across people in your organization or across you and your vendors or however, but shared definitions used as an overlay on top of your data analytics to interpret the data. And you get the benefits of having a combination of structured knowledge about your domain, as well as capabilities to integrate different types of reasoning and inference queries, graph algorithms, machine learning models, embedding, et cetera. There's actually broad adoption. Now, I've been writing about this in terms of knowledge graph, although it's largely private. There's a couple of great talks coming up. There's the knowledge connections conference. I'm one of the organizers. Metadata day in terms of work by LinkedIn, Lyft, Netflix, Uber, Airbnb, et cetera. And also there's an open source library that I'm using in some of these tutorials called KG Lab, which is built on top of Ray and others. So another thing that Ben Lorak and I have found, a stark reality is that there's been a growing divide. In all of our surveys that we've done over the past three, four years, we're finding a consistent thing. The firms, especially in enterprise that realize return on investment from their work in AI, they've been doubling down very aggressively on reinvestment. So they're investing more and more into this space. But yet at least more than half of enterprise are still years away from the required digital transformations before they can even begin to start to do the work to become competitive in this space. So there's this growing divide between the haves and the have-nots. And the reality unfortunately of our post-pandemic world is that this gap will intensify even more. And ostensibly, the cloud providers are at the head of the curve as far as pushing the innovation. They appear to have first mover advantage. You may have seen this book. The numbers game came out a few years ago. There's a really good podcast by Malcolm Gladwell. I've got linked here, which describes it in more detail. And he does a compare contrast between American basketball strategy and European football strategy. And the idea in American basketball, if you're investing that way, you want to buy really good superstar players because they're the ones that make the points. That's what you care about. In football, in soccer, of course, you have to have full team coordination just to reach each goal. So there's a very different philosophy about how to build and leverage a team. And so that's what I want to point out here is that what we're seeing over these past few years, what Berkeley is pointing to as well, is that the the cloud giants may have first mover advantage because, frankly, look at the points here. They've hired the best AI talent. They've locked that up quite aggressively. They're using as a barrier to entry. They own enormous data sets. Andrew Ng and others have talked about the strategy over the past decade for Google to do this. They own these large data sets that are labeled that can be used for deep learning. They also own the public clouds. And you almost have to own a public cloud to compete in the latest, some of the large transformer models, for instance. And so these companies are producing the leading research currently in AI. But on the other hand, when we go out and talk with enterprise customers when we survey, what we're seeing is that, for instance, in natural language, where some of the largest models are, in natural language, the services based on these by the cloud providers are experiencing more than 50% customer dissatisfaction. People come back and say, hey, look, this really doesn't work for my use case. It's too expensive. It's not accurate enough. I can't use it. And so we're seeing this very common mistake where the cloud providers are really leveraging a one size fits all strategy. And I've talked with product managers and they know that they're doing it. And they're leveraging enormous models that we already recognize have diminishing returns in terms of the economics. And for that matter, the whole argument about AGR, artificial general intelligence, that's decades away, frankly, barring some sort of physics breakthrough and quantum computing becoming commodity. So the idea of having a superstar AI or even leveraging superstars on a team, that's a ruse. Okay, in English, we call that a long con. Instead, there's a different alternative. And I would point to what's being called operationalizing AI. There's a really great article from my co author, John Thomas, along with Will Roberts, we've got a book coming out about this. But the point is that to operationalize AI, you have to go beyond even having just a data team working with the operations team, you need to look across the full organization, you need to have resilient long term strategy, full team coordination, not dependencies on fortune telling crystal balls, not dependencies on hiring just a few superstars. Okay, so look at your organization. Where do you rely on developing policy? That's where you can leverage reinforcement learning. Where do you optimize simulations of production, whether it's sales, manufacturing, logistics, that's where you can leverage augmenting by reinforcement learning. How can you combine machine learning models with domain knowledge and use this to augment your people and your business processes? Okay, that's the message here. And it's where you apply AI in your organization today. Instead of playing American basketball, you need to play an excellent game of football. Thank you very much. If you want to get a hold of me, here's how to reach me. Thank you. So thank you, Paco. I don't know if there's any questions from the audience because the thing is that you can make questions and I will make them to Paco. You can make questions in any language. Of course, English is the language we are using, but then we'll translate. So you can make questions in, of course, Spanish, that French, Euskera, Catalan, Galego, whatever you want. And I will do my best to translate them to Paco. So prepare your questions and send them to us. Paco, let me make the first one to me because I really love this symbol you make comparing the European soccer with American basketball. So could you go a little bit more on detail on this, which is the core of this model of this, let's say, comparison between the team play and the star play? Certainly, certainly. So I mean, in American basketball, you have a team, a number of players are out on the court, but you have your star players and they can shoot, you know, they shoot the baskets, they can make the points. They're the only ones who really make the points. So you do have team coordination, but it's not as dramatic. It's really focused on you have to go out and recruit and hire that superstar. And that's how you win. And so what we're saying in AI is sort of, you know, the sales pitch from the cloud vendors is, hey, we hired up all the talent. Hey, we own the clouds. Hey, we own the data. We're going to produce really fantastic things for you that will be able to work much better than your people. Just buy our services. And that's a ruse. That's American basketball. The idea is instead with European football is you need to leverage the domain expertise in your team. They know your business. You need to augment them, not replace them. And so it is a matter of this full team coordination, not believing that you're going to get something magical, a superstar, a crystal ball. Instead, leverage the business that you have, get your people to work together and augment them. And that's why we're saying things like reinforcement learning where you're developing policy. This is something that a full team can go back and reference and say, okay, what do we need to do now? And if it's not correct, how can we iterate on it? Does that help? Yeah, yeah, sure. We have the first question for the audience. It's just plenty of important information being delivered. And he would like to know if you will be sharing all this information that we were commenting. I can anticipate that, of course, you will be sharing it. But Paco, explain how you will be sharing this information. Certainly, thank you. Yes, certainly. It's already online. There's a link at the first part of the slides. We'll be posting it in the comments for the talk and also in various chat forums. I'll put it on Twitter, etc. But all this material is online in public talk already. And I'd say there's a lot better speakers than me. There's a lot of primary sources and really interesting people to hear in their own words, what were they thinking when they made these kind of innovations and how do they tie together? Yeah, well, the idea is that the conference doesn't end in three days. So this is a community and you can share all this stuff and give you feedback and begin the conversation. So I think that's the idea. Paco, how do you see the future? Because you have to talk a little bit about this cloud ecosystem. But how is it changing and how you feel about this possible evolution? Yeah, no, I'm very optimistic about some of these areas. I mean, I think we have to be careful because a lot of the industry is moving toward very large models. They have a very large carbon footprint. If you run a transformer model like BERT with neural architecture search, it has the carbon footprint of running five gasoline automobiles for their entire lifetime. Just to train that model, that one model. So I think we have to be very careful about which direction we're heading to now. I'm not saying that we get rid of supervised learning. What I'm saying is if you take a smaller model and leverage both data plus some structured knowledge, you can get away with much better results with much less computation. And we're seeing this and we need to leverage those kinds of hybrid solutions that make the most out of both data and the domain expertise. So I'm very optimistic on that point if we go in the right path. I'm also very optimistic that when you look at this area of simulation, what this means is if you have a business and you're doing some manufacturing, and maybe you have to structure your factory, what the floor automation looks like, if you're doing any sort of supply chain work, inventory control, or even just even in sales where you're doing forecasts, any kind of simulations work is perfect to marry with this new generation of AI. And it's not to make the decision necessarily for you but to understand how to develop better policy so that when you get into a bad place, how can you vector back into a more optimal area of business? It's much like, I don't know if you know that the history of commercial aircraft in the 1950s, there were commercial flights, there were terrible accidents where airframes would just disintegrate in midair. And so there's a lot of work in the 50s to reinvent aircraft, commercial aircraft, so they would be resilient for difficult times with a lot of air turbulence. That's the actual theory that we're using for reinforcement learning. And it's so perfect for 2020 because we need businesses to be more resilient. You don't know if your critical vendors are going to have a hurricane that cut them off. And once you start to recover, they have another hurricane or an earthquake or a pandemic or however. Unfortunately, so this idea of building policy toward resilience and leveraging AI for that for teams of people, that's what we're trying to say. And I'm very optimistic about that. Resilience is for sure the most important word here. Our government presented today a plan that is called with this name. So it's everywhere. It should be used for this work. So I have a question for the audience. They are asking you, we used to think about that humans were in charge of strategy and machines in charge of tactics. Isn't that what you are proposing? Or do you think this is changing in the meantime? So it's the algorithm that things will be in the tactics and we, the brilliant people, the brilliant humans will be in the changing? I think it's all mixed up now. And I think what we're seeing instead is when you talk about a team and actually some of the large consultants like Deloitte and others are actively doing this, Mackenzie also, when you talk about a team, let's talk about the components which are people and also the components which are machines because the machines do some roles. And there's a really interesting article from Deloitte about their practice a couple years ago where they talk about HR human resources having to articulate how to incorporate the machines into their team and the policy based on that part of it. And you really have to. So yeah, the strategy is partly from machines and people working together. The tactics are also very much. And I think if you're in business, you need to understand for your use cases and for your customer base and your products, really what side of the spectrum does that lean toward? Maybe more people heavy in some areas, more machine heavy in others. But the big problem is about dimensionality because people can really only think about four dimensions at a time. And beyond that, it's superhuman. Machines can handle billions of different dimensions if we need them to, hopefully not that much, but much, much larger. And so this is a way where machines can really do a lot of work for strategy. And again, we can leverage simulation that we already have in industry, but now we're just sort of augmenting with the reinforcement learning. We have another question that will be the last one from the audience. They are interested in knowledge gaps. You were mentioning this in your presentation, that they could like to have a little bit more detail. Yes, excellent. So I mean, when you look out at the way that we manage data almost always the data that we have is linked. So we have this notion of linked data where okay, I have some description about customers, I have some description about products, I have some description about sales offers, they're all linked together. And we can do that in a relational database, but the links are very trivialized. Okay, they're very shallow. And they're also very explicit. It's not something you can really define very well in a relational database. But in general, when we're working with data, we do use graphs. And we talk about data as graphs. You can see this even when you do the diagrams for relational database. So the idea of knowledge graph, it's been around for a while. Some of the first usage was in the 1980s, as far as descriptions of it. The idea is you have your data, but also you have an overlay on top of your data for each piece. What is its definition? Where is the metadata that describes for this date timestamp? Is it represented in UK representation for the date timestamp? Is it represented in ISO or US or however? For this measurement that I'm making in my process line, is it in American metrics, sorry, American measures or is it a metric? So the idea is if you have the metadata that you can link out to, then you can have both people and machines leverage the data much better and be able to make inference off of it. Also by virtue of a graph, you can find things like relationships. The notion is to really leverage the relations in the graph, maybe in terms of language. Let's say we leverage the verbs as much as we leverage the nouns and the adjectives. Too much of database theory has emphasized the nouns only. And so this notion of knowledge graph is to say how can I link out to different organizations, link out to my partners, my suppliers, and still have some consistent data representation. The other thing that's really important about knowledge graph is this. When we talk about supervised learning, we generalize, which means we remove some context. And now we have a problem when we use machine learning models that it's hard to say why they're making this decision. It's hard to interpret and go back to that context. Knowledge graph is the opposite. It's complementary. It's the way to actually bring in more explainable AI and recover that context. So we are seeing a lot of work both in the query side, the database side of graph, but also in terms of the inference and the graph algorithms and really how to marry it with what we're doing with deep learning. And like I mentioned, I'll be doing a tutorial at knowledge connections in two weeks. They ask for the tutorial. So you're already answered. Another one, could you give us some reference or some examples in today's business of concepts that you have explained like AI, AGI or ASI? Could you give some real examples? Certainly, certainly. So sometimes people will interchange the use of saying AI or seeing machine learning. The way that I look at it, machine learning is an area of mathematics. It's an area of optimization theory. There are certain tactics, techniques in machine learning. AI is much more to me about the application side of it. And it's really where you're bringing in also the people into the equation, how to augment them. So AI is, so for instance, if you can have systems that have full people and machines that are able to perform better than just the people alone. And in some cases, the machines can actually outperform the people in certain areas like automated translation, there's some of these areas. Then, okay, we'll call that AI. It'll get better over time. We're not at the point of having some magical box that you go and talk to and it solves all the world's problems by one question. We're not at that point. And so AGI, artificial general intelligence, I think, is a long ways off. There's a lot of components that will be necessary and a lot more compute. I don't think we're going to see that any time in the next few years. But people are working toward it. That's great. But you know, what I do see in terms of these definitions, like for instance, there's a company called PathMind.