 Okay, my name is Bruce Mamjan. I started the conference day with a two-hour workshop about wind and comment table expressions to SQL capabilities and Postgres. I am from Foto Pennsylvania. I work for EDB and I am one of the Postgres core team members. Like my morning talks, this slides are online. If you go to this website, you will see that there are, there's this talk plus probably 25 more talks that also include the slides and the videos of many of those talks. So I'm going to finish the day by talking about Postgres and the artificial intelligence landscape. Artificial intelligence is a hot topic, one of the sort of named topics for today. And I wrote this talk because I wanted to understand fundamentally what is artificial intelligence and specifically learning and how do those relate to a use in the database. So this is not going to be a super practical talk. What it is going to be is to show you exactly how machine learning works, show you how the works, and then finally talk about how artificial intelligence and machine learning can be used in various fields. So we're going to talk about what is artificial intelligence. That's actually a question. I'm going to talk about machine learning and the deep learning branch of machine learning. I'm going to demonstrate machine learning using Postgres and it should be a very even if you don't know Postgres, that's okay. You'll you'll be able to understand the code very easily. Then we're going to talk about how artificial intelligence is normally implemented and then talk about some tasks and why use a database. So what is artificial intelligence? This comes from a Wikipedia article and the quote that I like the best are artificial intelligence are machines that cognitive functions that humans associate with the human mind such as learning and problem-solving. I want to highlight the word mimic because it's not totally clear if artificial intelligence is the same as the human mind or if it's sort of mimicking it and that's kind of a fundamental issue in trying to understand the limits of artificial intelligence. So if it's whether it's the same it just is mimicking something that looks like it. This is my one philosophical slide and I think it's worth considering. I've considered it myself. The question is if the physical world exists, if nothing exists, they can't be measured or isn't physical. The human intelligence really only differs from machine intelligence because it is not naturally developed. It differs only in how the intelligence is created. So if only the physical world exists, then human intelligence, artificial intelligence may be exactly the same thing. It only means that one was developed and one was developed by humans. The interesting caveat to that and correlate to that is that if that is true, then human free will effectively becomes an illusion. The free will that you think you have are really just the neurons in your brain sort of calculating things and that the things you're calculating are predetermined in a way. Inputs based on, you know, your past experience. Anyway, I'm not going to get into that. There's an article there at the bottom of URL right here that talks about the concepts of what the Orleans but it's I think it's worth something worth thinking about. Here's the history of artificial intelligence. Artificial intelligence basically started before the computer was invented. It started in sort of this pre-computer philosophy concept and there are a number of philosophers who were kind of big on this. Then you have robotics as a part of artificial intelligence, the idea that if you mimic human you're intelligent or you're sort of artificial intelligent. That's kind of formed by the wayside, but it was an area of artificial intelligence years ago. Then we had the Turing test around the fifties sort of tried to test intelligence. Our Robert Turing obviously was the margin of that. Then we have expert systems in the eighties where they tried to take the experts would use to solve a problem and codify those into a into sort of a program that could be run. Then we had those didn't work too well either. Then we had what's called AI wind where there wasn't a whole lot of AI going on at all and then sort of back in you know the two early 2000s you have artificial intelligence and machine learning, which actually has some application. It can be quite a success. For me having been around this field for a long time, artificial intelligence is like fusion energy. It's always ten years away. So it's it's something that's always always there but it's the same thing you know dozens of decades ago. Let's get more specific. Let's talk about artificial intelligence and specific implementations. What we're going to talk about today is a specific brick intelligence called machine learning. As you can see machine learning is inside the larger artificial intelligence field and then inside of machine learning there's this thing called deep learning and I'll show you some examples of exactly what deep learning is. There are other branch of artificial intelligence but these two machine learning and deep learning are the ones that are most popular today and the ones that are getting the most activity. See this slide pretty repeatedly. Oh, I'm sorry by the way if you're curious, there's a the URL here at the bottom again has a very good explanation of those fields. So you're gonna see this slide for a couple versions and it takes make sense to spend our time kind of looking at this and breaking it down. This is typically what a machine three stages of a machine learning. The first stage is called the initial stage where you have a machine called a tensor with three neurons inside of it. That's what we call these neurons. Then we have a training stage and then we have an action stage where we're making requests and we're getting results. Okay, so there's typically an initial stage, a training stage and then an active stage where we're making requests and getting results. So here is the the first example of unsupervised machine learning. So in unsupervised machine learning, we're going to walk through exactly what these stages are. Okay, in the initial stage you have this thing called a tensor. A tensor is technically a vector. It's like if you're familiar with a vector in in in math and math. A tensor has a specific meaning to it, which really does apply here, but they just call it tensors. Okay, and inside this particular tensor, we have three neurons. So this is one neuron one, neuron two, neuron three and all neurons, but again, they aren't the same as the neurons in our brain. They're basically just levels. Okay, and I know that doesn't make a lot of sense, but once we kind of get going it will. Then we have our training stage and here what we're doing is we're increasing and decreasing the tensors based on the training data. So you notice that I've got an arrow that has an up and down here. It's kind of got a slash. So this neuron is going up and down based on the training data that has come in that we're feeding into the machine. So that the typical flow is you start with empty to empty neurons. You set the training data and finally you actually use the tensors and the neurons to produce results. Now in unsupervised machine learning, we're feeding in training that has no results to it. All right, so all you basically have is the training data. All right. So effectively, you just have this thing. It's going up and down, but there's no results to it. It's just it's just it's just we saw a lot if you have, for example, pictures of cats and dogs and you just feed them into the system and you ask the system, distinguish between these two sets. I'm not going to tell you anything about the sets. I'm not going to tell you which is a cat or dog. Feed this data in and it's the job of the machine to identify which one is which. Okay. Now, I'm not recommending this mechanism, but this is a mechanism for doing machine learning. The next one is called supervised machine learning. Now this one is a little easier to understand. What we're doing here is where you have a state again starts empty, but now our training data has some results to it. So the example I typically have with this would be if doing let's say fraud detection. Okay, and you're going to feed in a whole bunch of values. Right? And you basically feed in 100, 200, or whatever you feed in false and true, false and true, and you basically are then getting some results back on that. Okay? Hold on a second. Hold on a second. I'm going to have to update my story about that. Let me just stop here and I'm going to get some of my I'm going to get some new slides. Because for some reason it has some new slides. There we go. I'm going to switch over to a new slide deck. Okay. Okay. Never mind. I guess I'm okay. Hold on. I'm just checking here. I was just concerned about something. Okay. So So unsupervised learning has has us feeding the data in with no with no results and supervised learning actually has results attached to the individual entries. Okay. Now reinforcement a little different. In reinforcement learning, we're effectively doing the same thing we're doing with machine learning but we're reinforcing it. So what we're doing is we're starting with empty neuron training data in with results. Okay, but then once we get once we start working, we actually where we feed the results back in to the to the tensors the neurons again. Okay. Hold on a second. Okay. So we're actually feeding results back in. So what we're doing is we're adjusting the neurons once we've gone live. Okay, so we've had the initial state between the training with results. Okay. We're setting the neurons, but now we're feeding the output back in. So why would we do this? The reason we would do this is if we find a case the neuron where we want it to improve itself while it's running. Okay, because if we go back to supervised machine learning, all we're getting here is a case where once we go live, once we're done the training that the neurons don't change the they don't go up and down. But with super reinforcement, we can actually have it learning and improving as it goes. Okay, that's a very complicated topic. I'm not going to really be able to get into it in a lot of detail, but effectively this is sort of the case where you're you've got this thing getting better. Now the problem is that if you if you get too much feed, you can overwrite the test data and the system starts to go wacky. So there's a very interesting sort of philosophy of when you want to do this, when it makes sense to do this reinforcement. This may not, you know, this might be kind of an unusual case, but I think I think you kind of get the idea when you would use this. Okay? What about that? I have to take a break. Okay. For some reason the number of slides was wrong, so I'm going to fix in that. Kind of weird. Okay. So that's the reinforcement one. Okay. Deep learning. Remember I talked about deep learning. I promised we'd talk about that. In deep learning, you have multiple machines stacked together. So here's our here's our here's our original case of machine learning where we've got three neurons and in deep learning, these three neurons feed into another tensor for another stage. Another stage after that. So you've actually have one, two, three stages of machine learning and you basically, you know, feed from one to the other. Okay. Now, why would you want to do this? What's the matter with the where? Well, imagine you're analyzing an image, you're analyzing audio, which is what we do a lot if you're if you're having using something like Google Assistant or Alexa or, you know, Siri, Siri from you talk to it and it has to first identify, okay, what is the pitch of the voice and what's the volume and, you know, what's what are the words there? And then once you have the words, what should I do with the words? So you can imagine when you're doing that, if you're analyzing an image, you have to identify the foreground from the background and then the colors and then maybe the tail or the head and then you sort of work your way through and each phase is doing a different okay, so that's exactly what this is designed to do is to give you different phases where you can kind of walk through the system and get different sort of results through the phases. So it's very popular when you're doing complex things again, where you're using each machine once and you're feeding it in and what you can see here is that each neuron is going into each the next so I have three neurons here, I have three neurons here, I have nine connections here and then I have nine connections here and so forth and sometimes these deep learnings can have thousands or millions of neurons and thousands or hundreds of stages and of course deep learning that big, you're looking at a huge amount of computation that's why AI is known to have to require a lot of computation. We'll talk about that later okay, so what we've really talked about is the concept of machine the concept of unsupervised machine learning supervised machine learning reinforcement learning and finally deep learning, okay so, uh, let's talk let's use a demonstration of exactly what uh, thing is I was going to ask for questions between each section, but I know uh, this the uh, the conference is running 10 minutes late, so I'm going to wait for questions at the end um, and again feel free to type those in and we'll answer as many questions as we have uh, but again, I don't want to take time because we are running a couple minutes late in the conference so with uh, with here's an example of of machine learning that is very silly, okay I'm not recommending this to anybody, uh, but this is a very simple example of uh, of machine learning so what I'm doing here is I am demonstrating uh, how to use machine learning to identify what number has a non-leading zero in it okay, so the first number here 31903 has a non-leading zero, uh, right here okay, um, and this number does not now this is a very stupid example you'd never use machine learning for this because you can figure it out pretty quickly using other methods, uh, that don't need machine learning, but the reason I use this example is because it's very easy to understand it very quickly, okay so, um, first we're going to install pl pearl, uh, pl pearl is a server-side language for postgres I'm going to do this entire presentation with AI using postgres if you'd like to follow along you'd like to run it yourself this URL right here, uh, ai.sql if you download that you can actually run the entire presentation uh, in psql just just run the sql and you can see exactly what I'm going to show 10 slides or so okay, so uh, what are we how are we going to do this we're first going to create a function which creates a tensor we know what a tensor is a tensor is those three neurons that in this case we actually have 11 so we're actually going to have 11 neurons in this particular test the first 10 neurons indicate how long the how many digits are in the number and the last one whether it's divisible by zero so we have basically two, uh, two types of 10, uh, neurons we have number digits and then whether it's divisible by zero or not, okay then we have a, uh, we have to create a training set because of course as we saw before, uh, we the neurons start empty so we have to put some training data in there we're going to run a training set we're going to create 10,000 the numbers aren't technically random they have a random number of digits so, uh, a typical random number generator most of the digits would have eight or nine you know digits in this case we're going to have a lot of short a lot a lot along a lot of different ones so we're going to insert those 10,000 random numbers into a table called training set that's where our training data is going to live so what is our training set look like? this is an example of the first 10, uh, training numbers that I generated as you can see on the left there's a train there's the number and you can see their random number of digits uh, we have a training output which is whether it has a zero or not in it this is supervised machine learning because we actually are telling the computer exactly how many um, how many of the zeros are there how many you know whether it has a zero or not we're telling it so it can learn from that and finally, uh, the last row on the right is the tensor 11 true and false values okay and that's how that's the tensor is going to be generated for this particular when we feed it into the machine learning um, machine learning you have to sort of play with the math I'm going to say uh, we basically have to generate weights for these tensors so I'm going to create a function this function is going to be three pages so this is just the function again in pl pearl, uh, which effectively just creates the weights for us so it allows us to adjust the weights up and down in the tensors based on the data that comes in this is the body of the function takes a particular value and, um, it's tensors and then and then it it's neuron weights and it kind of calculates weighted um, average that tensors and then it computes a, uh, weighted percentage of the basically the counts of how probable how how predictive is this neuron of the result I'm looking for so when you have a machine case you're going to create a lot of different neurons some of the neurons be maybe very predictive some of them may not be predictive at all so what we effectively have here is a way of controlling sort of how how weighted those particular neurons are okay okay we also have a tensor mask here we have to mask out the neurons when we're doing our our learning we have to mask out the ones that match and the ones that don't this is kind of a you know straightforward function another thing that's pretty common in artificial intelligence is that you have to you have to weight the neurons when you run your initial training set some values are going to be really high and some of them are going to be low but what you need to do is you need to even out the tensors so they all add up to one so you're going to kind of take you know maybe you have a 300 there and 500 over there and you want to balance those so it's like 0.3 0.5 0.1 so that they add up to 100 or to one so you have a balanced set of of neurons you know technically adds up to one and that's what that does there's also called softmax this prevents particular tensors from getting too high so again there's a there's a video link in this presentation that our video link that's probably the best one I've seen that walks through exactly how artificial intelligence works with an example I think of identifying dogs in a picture but again it does go through this map and need these particular aspects of the to-do machine learning so we actually have to store the the training data the tensor weights so we created table for the tense it's table for the false tensor weights and now once we once we run our training data through you can see the weights there hold on let me back sorry I missed something here um yep yep so uh what we're doing here is we're um we're what we're but what we're doing in this slide is we are taking our training okay the 10 000 numbers and we're running it through the generate weight function and we're inserting them into the true table weights and the false table weight in separate tables which is fine okay uh so here's where this is our actual training set this is our actual uh uh training process where we take our 10 000 numbers with we feed them in and we generate weights and here are the weights they generate again I said it has to it should add up to one if you add up all these numbers do they add up to one and what it's trying to tell me is how selective are each of the 10 okay so now we have our training data we've run our training data through we've set the weights on the tensors and the neurons we can now use the program so now I'm going to feed the number 100 into the machine I'm going to run it through my tensor weight table and I'm going to generate a tensor I'm going to use max some weight and soft max and it comes back and it tells me the only chance that it has a zero in it well it already has it has a zero in it but it only thinks there's a 20 percent chance of that um is that good no not great all right you you'd like to have a higher percentage uh but again machine learning is not precise okay it's it's fuzzy kind of and you get these probabilities if I feed 101 into it notice the percentage in it for 100 was 22 percent when I go to 100 it's 11 percent so it clearly knows that when I've used 101 the percentage drops dramatically and that's because there's no zero in the first right here's another number a very long number um and you can see it thinks there's a 68 percent chance that there's a zero in there there isn't but because it's a long number there's a lot of a lot of digits it thinks there must be a zero here somewhere there's hopefully a zero somewhere again it's only using the training data we we set in to figure that let's walk through a test of values so what I'm going to do is I'm going to take a thousand values and I'm going to run it through the machine okay I'm going to run it through and generate my probabilities for those 1000 values and I'm also going to add those thousand values for those of you who saw my common table expression this morning that's actually exactly what I'm using here I'm using common table expressions and I get this result this row right here tells me the system is 15 percent better than random it's not great but again it's it's kind of the best we're going to be able to do with this simplistic example the 72 percent likely it was to kind of pick the right number but again not a great result but that's what it is we we didn't tell anything about how to find a zero we just gave it some rules and then it ran the training data now you'd never do this in real life this is just for a demo so you can kind of see it walking through so let's let's talk about what we would do in real life in real life there's really some very popular tools to do machine learning tensor flow is probably the biggest one that I've heard there's also matlab which is open source no I'm sorry mad lib is open source matlab is not matlab and then there's weka and those are all client side tools meaning that they run on the client and they can potentially use a database to store all this server side you can use pl python with some libraries you can use matlab or you can even use something called scikit which is very popular in open source which again runs at the client and the server side it is really kind of nice down there at the bottom I have a URL I'm sorry a video link that you can use if if you're interested uh why would you want to use a as I said before machine learning and particularly deep learning is very computationally intensive you've got maybe millions of tensors you maybe have hundreds of layers and a lot of comparisons so GPUs are very well processing so if you need to do that a lot of GPUs are designed the libraries are used to design to use GPUs to perform this sort of repetitive calculation to have with machine learning and deep learning kind of cases so that's why you hear a lot about GPUs in AI because it it really is required to sort of do that kind of work okay so what tasks would you use AI for well you would you could use it for chess you know when I was a kid we used to have chess computers a IBM used deep blue for jeopardy that was very popular voice recognition is very good anytime you're using Siri or Alexa or Google home or your phone you know you're talking to it that's voice recognition searching search results recommendations for videos for example action very popular and even weather forecasting you can imagine why you would use that I'm going to walk through what I think is the most common example that is fraud detection in fraud detection you effectively have a case where maybe you have a credit card company and they're trying to determine if a if a particular charge is fraudulent or not okay so what would we measure what would our neurons be well we would be measured charge the amount of the amount of the charge uh whether it was a magnetic swipe or a chip or an online charge the distance from the vendor to the address or the distance from the charge to the previous charge uh we could look country we can look at previous charges by the vendor okay so how would we set up AI to perform this task we would probably choose our attributes maybe the ones we would use before okay we would then create a machine to measure each of the things we want to measure then we would feed a whole lot of data into the machine which would have results here's a transaction it has these neurons and it was it was a valid transaction here's another transaction has these neuron values it was fraudulent this is machine supervised machine learning because we're giving the results with the training day right we would feed it through and then once we got it working which we loaded all these old transactions through we could then predict whether future transactions were fraud fraudulent and if we wanted to get fancy we would have some of the incorrect results or correct results back into the system so it would learn it would learn new fraudulent patterns as it was sort of tracking okay so uh database and all this why am I using Postgres and AI in this example uh well of course machine learning requires a lot of data and where is most of your data most of your important data is in the database so learning in the database is clearly a reasonable thing to do it's not super popular right now because machine learning is still a new field but as machine learning starts to become more popular you'll see more and more machine learning probably moving into the database just just like data warehouse moved in and full tech search moved in and json moved in AI event will move in uh to be part of a standard part of database but again it's still in its infancy so it's still being developed externally in most cases ages of doing machine learning in the database of course um you can use previous activity you have seamless access to your data you can take action on these particular entries um you can even check a transaction before so you could say before I commit this transaction what are the odds it's for a joint great way of sort of using that and finally uh there are a whole bunch of uses for machine artificial intelligence inside of databases user applications which is what we've covered but for performance improvement and alerting that finishes my talk for today I'm going to leave you with an interesting picture which sort of talks a little bit about artificial intelligence I think unfortunately we don't have any time I believe for questions but I'll be available uh to answer them either here or in the discussion room yeah thank you mr boost for the really informative sharing on on artificial intelligence and postgres so before we finish the talk can I just uh there was a bit of a rush introduction at the start so I'm so sorry for that and uh yeah so I'd like to rush share again that it's really an honor to have mr boost here with us he has been working on postgres for a really long time and yeah he's a co-founder and core team member of postgres global development group and yeah so he's very knowledgeable and very experienced in this topic so yeah we really appreciate having you here mr boost my pleasure yeah thank you for our share so actually I think we do have one question we can address that quickly here so it's can we have the test url out here please boost I'm not sure if it means url url you can see in the chat notes there okay I'm looking here um it's in the chat public chat okay chat notes below that okay all right okay um look and hold on can we have the test you are here oh the test okay so um so you wanted I think they wanted the url of the actual slides I think you're you're this slide it's actually right here at the bottom this this is right here so if you go to this url right there uh under under extended you'll see this presentation and then you can also download the sql that you saw so all the you saw you can download the sql and just run it and try it yourself that's what I was recommending to people um the actual url uh for the actual test is okay let's see if we can get back here find that for you that was way back at the beginning right there but again if you download the slides you just click on it it'll download for you yeah I think that answers the question great so actually now we have another question yeah I'm reading that question now so uh how do you create a progress with without billions of dollars so the way we can do that uh is because and I have another presentation about that on my website so if you're curious uh there's a there's a presentation called uh democracy and open source databases which I've given at other conferences but not at this one so if you'd like to take a look at that uh what it basically says is that because uh postgres is so efficient uh and because our development team is so efficient we're more efficient than a billion dollar company uh come software developed in big companies is very inefficient um so there's another presentation called well postgres live forever and that gets into the mechanics of open source and why open source development is so effective uh and you really have to read those and I believe there are videos of those as well understand why it's it's not postgres itself it's really the open source process is so efficient uh that uh we can develop you know with volunteers all around the world we you know I like to say we take volunteers all around the world and we we take their intelligence and we make like a lens and we focus it on the problem uh so that's uh the short answer to that any resources to learn about postgres and ai that you recommend I have put a lot of url here there is not a whole lot of information right now about postgres and ai I'll I'll say that right away um it feels like we're still in the infancy where people are still looking I need to do it outside the database uh but again uh as I said before over time as the technology matures it moves into the database there is a nice video on one of the slides if I can find uh which I would start with which is a good summary it's the slide that has all the software listed which is right here right here so this slide has a youtube video that youtube video kind of gives an overview of postgres and ai could you elaborate on sql versus no sql I'm sorry I I have a presentation on my website about that I probably doesn't fit the topic today uh those communities and companies that develop and participate in the community in Asia well uh we certainly have uh we have NTT out of Japan uh Fujitsu out of Japan and NEC out of Japan uh Alibaba out of Chicago out of China I have companies in Indonesia um I have companies in uh India and Pakistan I'm trying to remember I don't know of any in Vietnam right now uh but I will say that Postgres is growing rapidly in Southeast Asia we I was in Southeast Asia a year and a half ago for a conference in Indonesia and it was great we had hundreds of people so uh we're definitely growing quickly in that area region it's Postgres growing manifesto the MySQL yes it is because the community of Postgres is open to development so in MySQL whether it's MariaDB or MySQL from Oracle all the development's happening inside a company or inside a closed organization uh with Postgres it's community developed similar to Linux and Apache uh and that's why there's so much activity and innovation happening in the Postgres space that's I'm going to I'm going to defer to Hong on that one I don't know the answer I don't know of any Vietnamese language Postgres material yet we do have translation a lot of languages I'm Armenian and we just added some Armenian support but I don't know of any Vietnamese language material yet I'm sorry about that the good news is each country kind of develops its own ecosystem so for example China has developed they translate the manuals into Chinese there's sort of a Chinese blogger that's the Postgres blogger for China has a whole bunch of user groups related to that India has its own conferences as well as China and Japan so the good news is that you don't have to do everything in English you can create your own ecosystem inside a country where you have blogs about that are in Vietnamese where you have events that are in Vietnamese where you have materials that are in Vietnamese so one of the great things about the project is how international it is so I actually just checked when you read that question actually there are materials in Vietnamese um yes so so so if people from from Vietnam they want to to google team hill portraits there are a lot of materials written from the different community so there are I think I believe that are also um material in other languages Vietnamese for sure there are many I just google and can confirm that yeah I mean the great thing about the project is so decentralized so there isn't a central place that usually with proprietary software there's one company that's where all the activity happens but with Postgres it's happening all over the world all the time uh so that's great and you can create your own materials in your own sort of ecosystem within that environment the last question I see here we are part of google summer of code uh in fact we just got approved last week for number of code so if you're interested in that please contact the community uh do a search and we can be part of the community that way uh google summer code is not good uh I don't I actually am not part of it myself so I don't know um I know we have people from all over the world but I don't know the mechanics of google summer of code um it's just a project that that work but again there are there are people supporting Postgres uh all over a high go out of China is a small relatively small company but they have a whole Postgres practice uh Equinix out of Indonesia has a whole Postgres uh you know practice and and support for customers uh so that's the great thing also is that you can create your own business and then you're part of the community and you can then use Postgres within your customers and provide support to them you know using your own language with your own materials and so forth uh so it's a very open system and that's one of the reasons we're so popular um okay so that's a great philosophical question why are other projects not organized this way um um I think you have to watch the two presentations I mentioned before uh democracy and open source databases and why will Postgres live for um it actually is the same question of why are all governments not democracies right like some countries are democratic some of them are authoritarian some are in the middle um and some of them are partially authoritarian partially democratic um and it's because there are some efficiencies to authoritarian governments or some efficiencies to democratic governments uh in in general democratic governments are more flexible more easily to change more easily to bring in new people and to use the talents of the population more efficiently and it's exactly the same with open source you're using the talent of thousands all over the world who are solving these database problems instead of trying to solve them inside a small company that only has a hundred or two hundred engineers um even Oracle only has a small number of engineers who work on the database who understand it so again it's that scaling of democracy is it messy yes is it hard to figure out sometimes what's happening in Postgres yes I didn't even know there was a lot of Vietnamese material right because it's all happening in the distributed way uh so I would recommend you look at that maybe next year I'll deliver that talk um but it is a very interesting talk why isn't everyone that way it's sort of like saying why is in every government a democracy it's it has to do with history it has to do with there is a general trend or democracy around the world it's slow but it's happening I would say the same thing is happening in the software world can you help to work for the project better I would say the first thing would be to look at those presentations because I've given those presentations in China okay um I've given them in Russia I've given them in Europe and and North America um and and Latin America as well so again what the way I've been doing it is try and break down why is something successful what things does it take to make something successful and how can you duplicate the things a lot of people look at open source and think chaos and they they don't want it right um so uh open source is chaos in a lot in some ways it's very hard to control but it's actually very efficient and over time what you're seeing slowly happen in the software world is a movement to that democratic open source process but it happens very slowly because there's pre-installed applications there's people are used to using big databases from billion dollars uh there's people so Linux moved away from proprietary unix um and I think over time open source databases are going to move away from proprietary databases but it's a slow process so Bruce um thank you very much uh you know we need to prepare for the next session and I want to thank you for for giving the time within the 40 minutes so we can take like some major question just now and and I see that there are a lot of questions coming but we are like need to finish up the session now and there's a final question dude they asked please come to Vietnam's I guess that uh when COVID uh over you uh you got invitation to come to not only not only will I come to Vietnam but my wife come as well because she told me I definitely want to go there that's on my list okay thank you very much Bruce for your presentation and also for the morning workshop so you've always been a great honor to have you with us thank you thank you