 All right, so hello everyone. Thank you very much for coming. Thank you for your patience We are basically ready to get started. We're ready to learn a lot about machine learning from the It's amazing panel of distinguished colleagues that we have today. I This is part of our Purdue engineering distinguished lecture series Which is essentially an initiative starting from 2018 to bring together a lot of University scholars industry scholars and debate Interesting concepts and groundbreaking discoveries machine learning will be the area today I'm going to skip my 45 minute introduction and Directly introduce professor Stanley Chen who is going to moderate our panel Professor Chen is one of my colleagues in the School of Electrical and Computer Engineering Engineering working actually in the areas of image processing and machine learning All right. Good morning. Good afternoon So welcome to this panel. It's my great pleasure to be here with everyone To talk about machine learning, but before we do that, let me just briefly introduce our superstar panelists today Here we have professor Charles Boomer Professor Boomer is doing our image processing in the School of Electrical and Computer Engineering We have David Glike from computer science David does a lot of work on page rank Large-scale numerical computation. We have our guest professor Michael Jordan from UC Berkeley Professor Jordan does a lot of work in foundational machine learning the theory side We have professor one walks from industrial engineering professor walks. That's a lot of computer vision intelligent systems. We also have Patrick Wolf our Dean of College of Science and he does a lot of things on graph single processing and machine learning types of work Okay, so just let me just jump right into the questions. Okay And I want to make it interactive So I'm gonna ask a few questions and then let them fight Okay, and then and I will let you guys ask questions and ask them to fight as well. All right, okay, so Let me start off by talking about three bus words, okay Data science artificial intelligence and machine learning now I see a lot of students in this room and I realized that you guys all Confused about all these words because you you try to put equal sign for every all these three words and machine learning means data science and data science means AI, okay? And there's a very clear distinction between all these three words So I want to ask the panelists to give their perspective. How do they differentiate these three words? Maybe we can start with Charlie. Maybe I Was a little unprepared for that but so Well, first of all, I think it depends on the context whether you're talking about Among people in the technical community talking about like specific problems or sort of in the general public my own view which is Maybe specific to me which is is that that particularly data science is refers to this sort of broad set of Mathematical and computational and algorithmic techniques that people are using in various problems particularly you know scientific business oriented or Information processing and it's transformative for Society broadly because up until maybe the last five or ten years that people just didn't really think of of algorithms and computation as being part of Engineering or science they thought of science and engineering as making physical measurements. So so it's sort of it has that It's sort of a way of introducing the broader culture and within the technical communities. I feel like it's These concepts are blended also AI being more a set of goals to achieve and Machine learning being specifically directed towards sort of trained prediction. So that would be kind of my view So Okay, I guess I'm up next So so the three terms are data science machine learning and artificial intelligence. I'm glad you see a sharp distinction amongst them because I think You know, there are a lot of similarities and overlaps if I had to draw Try and draw a sharp distinction amongst them I guess I would say that data science is is is a lot of the practice of rigorous statistics Where you're replacing a lot of asymptotic expansions with computation as a first-class citizen instead. So rather than trying to get into You know, how you might do things analytically in some infinite limit You just use the computer to simulate what might happen in a variety of circumstances I when I think of machine learning, I think of a problem with a very specific goal in mind. So you're trying to I Train something to recognize is this spam or is this not spam and when I think of artificial intelligence I think of something that's a very aspirational goal. That's designed to replace us And I'll end there First of all one shouldn't make too much of these terms if your student don't try to define them So first of all is machine learning is really just statistics mostly and statistics in the broad sense statistics includes pattern recognition and Decision-making so decision-making under uncertainty. It's a 300 year old topic and machine learning was a buzzword used in the 1980s I think you developed at Carnegie Mellon by in computer science by people saying wouldn't be cool if a computer could learn Instead of having to be programmed, which does sound like a cool Branding and idea unfortunately, it wasn't at all new and and so once that was kind of Made into a goal people started to develop actual algorithms and things came out like decision trees and nearest neighbor and kernels and Hidden mark almost every one of them had been invented previously into statistics. All right, so statistics is an old field I think I'm mostly a statistician, but with a really strong interest that in all the Procedure side of it and the procedure side in the last hundred years has become dominated with computer science And but there's dynamical systems. There's information theory and so on so forth So whatever you want to call it. I call it statistical decision-making or certainly I say machine learning if I have to But I think that this broader term really captures the spirit more It's I agree. It's certainly about a bit of pattern recognition But banded algorithms that make decisions not knowing what the best thing to do or optimal control problems Which have to make a sequence of decisions or false discovery rate We're trying to make a bag of decisions together all that's part of statistics and statisticians They're very much as today and computer procedures the FF for Brad Efron is the world's leading statistician right into the bootstrap It's not about analytical expressions at all and that's that was present in the 70s and 80s and statistics You know so so computer science is definitely given a shot of energy and as deep in some of the topics like classification was not as deep But I can't think of very much a machine learning that's actually new Really just let me say it plainly nothing Okay Now data science I find even actually a more useful terminology even though it's more boring sounding Here's why so think about a database person Okay, I got lots of colleagues who are database people they certainly work with data They work with very large-scale data and they do all kinds of consistencies issues. They do all kinds of transactional issues and Banks would not run without the database people thinking about all those issues right so they are and they believe they call themselves data scientists I am clearly a data science. I work principles and I'm do data They would never have called themselves a statistician or a machine learning person right So it is clearly a bridging term umbrella term that is useful in this to bring that mix of ideas in and Then there are people I don't think it was just as algorithmic I think that so there are people in say physics who work with data all day you call themselves data scientists Right, they're not developing any cord methods. Maybe or maybe they occasionally do But so it is the first time in academia I've ever seen a term that most of the university agrees that I'm one of those and that's that's useful That's useful and and in industry. It's actually more of an industry term beta science It's a sort of people that do this kind of activity and we'll get a hair give them jobs Okay, so hopefully that's helpful or your your if you're into this field be it you have to learn a lot of statistics But of course you have to learn numerical in your algebra. You have to learn computer science algorithms and all that But hopefully you have a broader point of view. I'm currently taking a very econ point of view You'll see in my title there and if you come to my talk later for me It's all about decision-making in context of other decision-makers and systems and all that so that's not so now We finally I'm going a little bit long, but I've thought about this stuff a lot a I I really resent that this whole thing Is now being called a I that's pure branding. All right, and I can tell you who did it I think it was it was the AI it was the Google PR people And I really mean that so, you know, they had the machine learning, you know The neural nets were working really real well, and they needed to make a big terminology They couldn't call it data science or statistics or machine that was a big enough so they started to call it a I and Had not gone anywhere in 50 years really there's no new if there's been anything not new in machine learning It's certainly in classical AI very very little was do the other planning field has been very productive and it goes slowly It's hard, you know no knock to people that it hasn't really the aspiration Which I totally agree with of having thought inside of a computer and intelligence who is a wonderful aspiration But it's still very much the same aspiration hasn't really changed. We haven't budged on it very much We have data now we can mimic intelligence with a computer or aid intelligence and augment it with a computer But we don't have thought inside of a computer really in any meaningful sense yet Okay, so the AI people were happy because this new exciting technology was being given to their field And they were now in the center of everything again All right, and then the machine or the the the industry machine learning people were happy because everyone loved to talk about them You know we got AI now boy. This is exciting. It's gonna change the world and that level of hype is there's a certain level of dishonesty about it You know, so it's fine to use the term AI it has its history. It's great But I've said this in a couple of times, but I've lost this battle people are gonna call AI all of the above For a while. I kind of think that people will kind of wake up at some point, but that's where we are right now Well, I feel a little bit about that like I feel that that a science would be that I Feel that so think about that AI exists before Google, right? So the PR of Google came afterwards and AI was still there wasn't getting much more attention I have to say but in my opinion they would really start getting a lot of attention or the term In the branding of the term data science when when actually it appear I think that the peer after people realize that These techniques of machine learning that were originally proposed for machine learning like a neural networks Finally, they got a way to get them work In a very successful and effective manner and in the moment that it was so successful And there were a bunch of interesting problems that you know, the community was able to solve They say, okay, this amazing. Why don't we just give a new name and what name can we give? So we can actually teach that right if we can if we're going to teach again machine learning machine learning is not new Right, so so they came up with this data science name But of course then you start wondering where where the sciences is this a science or not? We have enough arguments about that But of course I cannot avoid but thinking about what the distinction would be So the way that I see these is indeed still some of the techniques are still machine learning I think that some of the emphasis as part of that the science not only that is a much wider and Broader field but includes also the way that you work with data So whereas machine learning maybe was before more the emphasis was on the techniques and the algorithms Now there is a big question of what type of data do you use? Can we generate artificial data? How the artificial how you make the artificial data look realistic? And if you need to rely on real data, how you how you find ways to to get this this data How you find ways to normalize or deal effectively with these data? So it's also dealing with with the data that you use for the for these techniques. So I Think you know, there's there's not maybe a ton to add to what's been said I think there are kind of threads of agreement through all of this Maybe it's most useful to focus on the facts or or think about what's actually changed over time It's certainly true for example that anything that's called AI right now that functions actually implements some form of machine learning some concrete machine learning algorithm And it's certainly true as as Mike alluded to that Statisticians have been thinking about these types of problems for you know, well over a hundred years. I think what's changed is You know traditionally if you were educated in statistics You were really taught to to sort of think rigorously about how to model a Problem or a physical system often and you're sort of really taught to kind of keep the question of Of how you actually get the inferential answer very separate from how you think about the model So you sort of didn't want to pollute your modeling thinking with with any computational thinking And that's one of the things that's really changed. I mean the disadvantage of that approach is You know if you've never studied The asymptotic scaling of algorithms then you might not as a statistician be able to develop an algorithm That's gonna scale in any way shape or form to a problem that we would now consider to be of a reasonable size on the other hand I Think there's been a risk in the past that if you were trained sort of only are primarily as a computer scientist You know you might think about data more as you know something like kind of toothpaste that just gets sort of squeezed out Of a tube you know and the focus is very much on an algorithm as a function that you know It takes a set of inputs and guarantees a certain set of outputs And that kind of misses the whole modeling aspect So I think the piece that's really changed sort of no matter what we call it is the kind of coming together of these two of these two areas and you know That's something that we all think about as educators You can't just make an undergraduate degree eight years instead of four and teach everybody all of statistics and all of computing we have to make some trade-offs and How we all manage those trade-offs and how you think about managing those trade-offs if you choose to go into this area I think is gonna have a big effect on kind of what you're able to do when you finish I think you know maybe the only other thing to mention is just as was alluded to a Lot of the classical sort of now classical AI and even machine learning approaches are very very old indeed some of the most famous were developed in the 70s and It just turns out that The rapid change in technology over the past decade has really been what suddenly made these algorithms work You sort of need three things you need sort of you know ubiquitous low-cost sensors To collect large amounts of data you need sort of cheap data storage and movement, you know 4g 5g You know solid-state Developments and storage and so on and so forth and then you need you know some algorithmic underpinning some kind of You know mathematical special sauce that's gonna make things work, and it just turns out that you know right now We're still kind of Revealing in the fact that a lot of these things invented in the 70s are actually working for the first time We haven't made a lot of progress yet in developing and maturing You know sort of truly new ways to think about algorithmic algorithmic advances and there are a ton of interesting questions there, you know starting with You know why are some of these algorithms so effective and how do they generalize? Continuing to You know, how do we harness fundamentally distributed computation in a different way? You know cray and super computing is not what it used to be We now have much more of a model of you know sort of computing on-demand Data and computation that are distributed in time and space and that requires a real sort of fundamental rethink of Everything from how we design hardware next-generation hardware Nvidia Intel and so on and so forth through to You know, how do we how do we kind of take good algorithmic and statistical thinking and kind of marry it up to this Way of doing computation So the last thing I'll say very quickly I know we'll get into this more in the talk, but what makes this so challenging and interesting is We don't really have a deep understanding right now into why some of these algorithms work or maybe a better way to say it is They're not particularly transparent If we're recommending books to Amazon customers, then I think probably none of us mind Whether the algorithm is a black box or a very clear, you know multiple linear regression where we can talk about the You know the kind of predictors in a very clear way But if we are making decisions about insurance or insurability if we're making decisions about Public health if we're making the decision about medical care or judicial sentencing Then I think we all have a much greater stake in in kind of understanding a little bit better in a technical way Sort of why and how these algorithms function and what their limitations are So I know we're gonna get into that and I'll just stop there since I've kind of taken us away from the initial question So let me follow up with Patrick's comment that that I'm seeing there are two different views when you You talk to the people who does Machine learning one side would be that I just want to do experiment and let me build 150 layer deep neural network if not enough 5,000 neural network 5,000 layers neural network and then really a million of images And make it work and then and then now this we see that the performance of these new networks this it's okay It's not bad is in some reports. They even say that we these new networks can do better than human But on the other side People who does theory then they will say that look all you guys are doing are just heuristic. I don't I Just don't believe in you. Okay, if you look at our work, we provide you performance guarantee We provide you as in top of limit. We provide you the results Why don't you use us and you go back to this practical side the practical side is that I try to use it but none of those work I I want to build a system that that's working So so these are two really really different views and and I do see there are some great opportunities for them to work together But I want to ask the panel what what are your views on on on on these two sides? We'll go the other direction. I think This is a tension that all of us who work as researchers in this area kind of have to have to manage You know the criticism of a more mathematical approach is we often simplify the problem by several orders of magnitude in order to build a step-by-step understanding you know, it's a little bit like You know baking the bricks yourself and then mixing the mortar and then building the cathedral It's sort of very very slow step-by-step building up an understanding in a very kind of You know sort of cogent way And and and and many practical problems of interest It's just not possible to take the time to do that I can't think of many cases where you know where theory leads practice when it comes to mathematical advances on the other hand, I think the biggest risk and limitation with More heuristic approaches is that we sort of fail to understand the fundamental limitations we fail to be able to predict when things can go wrong and how things can go wrong and And also we we risk reinventing Different flavors of what at one greater level of abstraction are essentially the same thing So it's not just about rediscovering and reinventing old methodologies It's about not understanding that one person's linear discriminant analysis is another person's perceptron and so on and so forth and Often if you lift problems up one mathematical level, you can sort of see across a whole vista and understand that a Number of things you're treating are kind of instances of the same fundamental phenomenon You know having said that I also I don't think there's scope and in everyone's education. There isn't time to kind of become Fully developed in both of these areas I think we'd like to see people who have a respect and appreciation of both sets of approaches But but ultimately I think it's a matter of kind of personal taste and what you want to accomplish that will drive you more toward You know, I think of a mathematician of somebody as somebody who if they can't answer the problem They'll change the problem, right? I think of an engineer as somebody who if they can't answer the problem They'll change the technique they're using so it really sort of depends, you know is a very personal choice as to you know, sort of how you feel about approaching that and We all lie somewhere on that continuum having some ability to respect and appreciate the whole continuum and to kind of move back And forth on it I think is one of the things that distinguishes the the more valuable contributors from others and You know, I think you've got an example on this panel of people who over their careers have Have kind of shown how powerful it can be to kind of not be stuck in either way of thinking But kind of be able to smoothly move back and forth across it So yes, I think that you know in the end of the day You really want to use things at work and when of course you're in your job You need to get things working and then and you're going to use whatever works But of course we start seeing that there are limitations in the sense that you know How much we can take neural networks and increase the layers and increase the and change the configurations and the architectures We are pushing the boundary. We're getting to a point that in my opinion. We will need a something totally new And it's going to be very difficult to come up with this new ideas If there is not the fundamental knowledge to develop these these new techniques has to be really root on fundamental science The other aspect is that these techniques are actually really good for doing certain things But in the moment that we want really groundbreaking results To be applied to different areas. So if I do the analogy We are very good dealing with a computer vision and machine and pattern recognition But if you want to if we move into the idea of AI, right? You want some kind of physical action some execution not just a computation these groundbreaking results are more difficult to be transferred to physical action in robotics and Even that we are trying to merge those two in terms of physical performance. We haven't yet got into that a Beautiful solution that you know machine learning right now is offering for for classification classification type of problems or regression So I think that we need to go back to the more fundamental Knowledge to reinvent this right because in a way, I think that these groundbreaking techniques in a big manner they are relying on methods that were developed 20 years ago or more and just they weren't working good enough because The computational power was this wasn't as strong as much and because the people working on these weren't stubborn enough Now we have luckily we have a few people that were stubborn enough to to keep pushing until really the results got drastically Improved but they think that at some point we're going to reach the limit that we need to look into other directions as well So neural nets are all the focus right now and computer vision So it's just such a narrow slice that I think I think we're too too focused on that Okay, first of all, yes a lot of progress has been made the air rates went down quite a bit. They're far from really that good Okay, if you take a neural net that's been trained on images off the Image net that's all these internet photos of things So there's an entity in the photo typically and you just it's centered and so on take that same neural net Which might be 95 99% and take it out the real world around here and and you know 30% Okay, recognizing actions and scenes recognize what's happened in scenes recognizing have you know more abstract notions of scenes I mean we can hardly talk about it Okay So you know all the progress the other one is people often say way out of vision speech and natural language That's the usual litany because it's human centric stuff and people find that so fascinating. I mean Machine learning has been used for fraud detection for supply chain modeling for recommendation systems So on for 30 years with no fanfare But it's created billion-dollar industries Okay, but suddenly now we have this human-like thing and we're doing better at that So we make a big big deal out of it natural language is maybe one to focus off from it There's a nice little article by Doug Hofstadter who's not far from here Called the shallows of Google Translate Google Translate is just doing this pattern recognition thing. It's taking a huge amounts of data huge huge So you got you know a billion strings in English and somehow a billion strings in French that you kind of match up in some way And it becomes a huge string-to-string matching thing Okay, huge No human being translates in that way Okay, that just I you know I speak a couple of languages if you say something to me in French I'll listen to what you're saying. I'll understand it I'll understand the relations you're talking about the physical entities the abstractions and all that I'll understand it And then I'll say that understand it in English. Okay, so what's computers doing at all has zero understanding zero All right, so you might suspect if you look at actual natural language translation done by a computer like Google Translate It'll be terrible at some level and it is Every sentence has got a error in it at least one maybe some more and they're often really egregious errors They're not close to right and that's not going to change very easily You can get trillions of data points and you'll cover a lot of those cases Same as self-driving cars You could drive the car around if you didn't see this scene ever before or you're not gonna do so Well now you'll get more of those scenes and you'll start to patch it up This is kind of hacking. This is gradient descent you know, so So on the one hand, let's just not focus so much on that let's focus on some of the remaining challenges also What do you mean by theorem? You know, there are tons of theory about gradient descent now It's great if you're focused only on the optimization and the processing of data side actually a lot of theory and that theory has guided practice Okay, in fact, it's not just gradient descent. It's stochastic and it's a momentum based and the theory is guided all of that There are lower bounds there rates. It's it's very very guided by the theory. Okay So why is it we people saying well, we don't know if this works We know that it gets done in zero or with a certain width of network and all that and in other situations like matrix factorization or phase retrieval We have proofs that it really works and it's all very beautiful Okay, well it doesn't work only in the inferential sense if you go into the statistical problem, right? Why does this past data training on that does it work? Well on an out-of-sample prediction that is hard to talk well what how what principles do you need? Well, it's not optimization. It's not linear algebra. It's not just information theory. It's statistics and statistics as notions of complexity Various kinds of Gaussian rather market complexity is the one way to think about it geometry meets randomness And that's an ongoing hard effort, right? And but it's not it's not going to stop and it will start to help to explain some of these things But explain the broader set of problems not just why things are working in computer vision Okay, so back to the question on I think this difference between Fully theory guided versus empirics. I mean, I think to all the students out there. I'd say you need both Right, this is not a one or the other thing everyone has echoed that to some degree Theory does impact practice in terms of telling us when when our methods work or giving us some Guidelines and even when we have good theory oftentimes it depends on totally unknowable Constance so a lot of times people will make a big deal about a theorem Well, if you can't compute any of the constants or the quantities in your theorem The best your theorem is going to guide you on as a heuristic method anyway Principled it's got all sorts of better ways of thinking about it. But at the end of the day It's not like you're deploying a theorem. You're deploying a heuristic And so I think recognizing those and appreciating the spectrum of things is really what we need Yeah, I mean, this is a really interesting question. It's when I was in graduate school Sort of the predominant view was and this was a very long time ago when dinosaurs roam the earth that That, you know, basically if you couldn't write down the solution to your problem in a closed-form analytical Framework you weren't doing real research. Okay, but and my view was okay We have to be able to use computers to do more and more things Not just evaluate the analytical expressions and slowly there's been a sort of a drift and what's happened is in my view More the theory is sort of moving to a level where the theory describes the behavior of the I'm gonna say It's not all-encompassing, but we'll call it algorithms computer algorithms Which is sort of the primary empirical approach in machine in these fields of data science machine learning and so But the theory is becoming sort of further removed from it So so the challenge is going to be you know, can we say things that are a precise and close enough to What really happens because when whenever you implement a real algorithm it never follows the assumptions of the theory and So the students all always say well, we did this theorem, but what we're doing is really completely different I like don't worry about that part. Okay, we'll just kind of hide that in the appendix Um But they say well how come you can hide that but you're not hiding this other thing and I said well because I know that's important The other one isn't so important. So I'm really not justifying my approach I'm just saying that's the approach I take and so it's really this blending of theory and judgment and experiment And maybe it's going to bring us closer to what we see in the experimental sciences where people have been doing like Experimentation and the theory and the experiments sort of move back and forth Eat and sometimes leading and sometimes following So let me take a break first Maybe I should ask the audience. Do you guys have any question for the panel? If you can any question you can ask Yeah So I see that everywhere on the internet. Yes today, so I'll ask you an insider question Okay, is there any neural network involved in the reconstruction? So I watched their TED talk right and there was there is this reconstruction algorithm, right? Is there any deep learning first all I have to tell you she doesn't I'm on a strictly need-to-know basis Okay, so I don't really get all the information disclosed to me So I'll be speculating on anything I say here, but I know that she really was very interested in using dictionary She was basing a lot of our algorithm dictionary learning and broaches And I know she's interested in using deep neural networks But the truth of the matter is and she hasn't told me anything so I have full immunity in what I say Which is that you know from Facebook? I know that you're from Facebook The I'm I'm told that they broke up into four teams Which I thought was a smart idea that worked independently and then the big Sort of breakthrough which occurred like a year ago. I'm my daughter didn't tell me any of this Okay, she she said I can't talk to you about that. Okay, so but But the big breakthrough about a year ago was that when they saw that the four images they produced were similar So that was really reassuring and that's sort of a smart approach That's how the space shuttle used to fly with three computers that were written by separate groups of software engineers But the truth of the matter is and I hate to admit this is that when you're doing these kinds of things You have like lofty ideas of very advanced methods, but in the end you tend to Reduce down to really basic techniques you feel pretty confident about because you hate to predict something Really important that turns out to be untrue. So You know, but that doesn't mean that it sort of Builds on the previous question, which is that you need a spectrum of approaches You need a pipeline of innovation. You need some very deep theoretical ideas that may not always be practical today but But maybe in the future and by the way I'm sort of a casual friend of Yann LeCun Which I think probably a lot of people are here and he was working on those convolutional neural networks for an awfully long time Without I mean it was like an overnight sensation that took 20 years So he stuck to it, right? And so there's a you need that pipeline of innovation Everything that you're doing isn't necessarily immediately useful, but you should see a line of sight to its value Maybe we can take a few more questions Up until now we have achieved some level of intelligence, but some people say that some product like speech assistant are still too stupid like Google Google now or Apple Apple Siri And they are still stupid. So what's the next step for? For machine learning or artificial intelligence Yeah, besides the pattern recognition to make the Our machines more intelligent I guess the question is how do you make the machine more intelligent? So I'd argue we don't have any intelligence yet I really like that reserve that word for something different Okay First of all someone earlier said that some things the computer is better than us Well, the computer is way better than me at the digits of pi Right and all kinds of things You know huge numbers of things. It's way better than me and you right is that make it's more intelligent You know arguably not Okay You're absolutely getting at a core area where it is kind of patently clear. There's lots more to do dialogue All right, so Distinguishing a chatbot and and a dialogue system Okay, so I think this is one of the best areas if you're really wanting to work on AI and understanding and Abstractions work on dialogue and chatbots and things like this are question answering A chatbot is a system basically it's seen lots of strings being typed into the computer before by lots and lots of people and It can figure out a good string to respond to any string that it sees in its input And it's really surprised seems intelligent to us Right because you said something and it says something it looks intelligent. That's false. That's fake Okay, that's not intelligence. It's just copying data or maybe you know adding a little bit of value smoothing out things from the past All right, so that's not so why do people find it tells well they can do it for a while But I can guarantee you it is mostly for like 16 year olds You know to have somebody to chat to after a week or two of chatting with your chatbot You realize that chatbot hasn't understood anything about your life You know you talk about your girlfriend or your boyfriend It says what I don't you know It doesn't have any understanding of the concept that it doesn't remember anything and so when it when it doesn't understand something It just says something else cool sounding Right and you get tired of that after a while and that's where a lot of we are where a lot of things are right now It's kind of faking intelligence and and taking in data from intelligent systems like us All right and mimicking that data, but without embodying any of the intelligence, right? So now take dialogue isn't it? So let's agree the chatbots are not intelligent. What about dialogue systems? Well dialogues are extremely limited and they're very very brittle and because they don't have any much semantics now If you if you don't have any semantics you cannot do a dialogue system. So what's a dialogue system? Well something like I want to get to a goal, right? The conversation has a goal It's not just a kind of bounce around and say cool things All right. Well that makes it way harder So even if the goal is as simple as I just bought a refrigerator and I took it home and it's not working I want to get my refrigerator working All right. Well, that's an area where dialogue systems can kind of help you can start out having a human on the phone The the first bit of conversation can be it's not working. Oh really? Is it plugged in? You know bow back and forth. That's a kind of a little decision tree All right, and a little bit of simple natural language possibly with a little bit of science in the domain of refrigerators Can get you a little bit there now eventually someone will say something like oh, yeah By the way my cat keeps getting behind the refrigerator and the cat hair seems to be a problem And the computer will have no clue what you're talking about It's just not that's not and it's semantic representation and that's just about refrigerators And if you do this it for a scale of 500 million people in the world calling your system Most of the time the conversation is gonna head off in some weird direction All right Google is being it is putting out dialogue systems Let me just bash them a little bit You know they put out one where they have the dialogue system call a restaurant and then it does something like um, um by the way Sounding human right that's PR. That's a stunt. That's faking it Okay, that's making us think that it's something that's me and then when it says something that sounds human like we say wow It's how intelligent it is right? Well, I can guarantee what you're gonna do is you're gonna call a real restaurant and sometimes it'll work But you know big deal right sometimes it'll get you a time But a lot of the time the person on your end will say well I can't get you in at 11 I can't get you in in 30 minutes because you know we just had a A gas leakage down the street and they've cleared out the street and so you know we're letting people in slowly You know again every conversation that we really have starts to head in one of those directions And then you have to have the semantic representation the abstractions of all the concepts They're now being talked about and you've never seen anything like that in your previous stream of Strings right so is it all this up impossible? No, but right now We're pre treating it was kind of a very hardcore engineering problem collect all these abstractions You know make a representation on that we don't know what intelligence is and thought to help guide us in that in that construction So NLP people are making a lot of progress Using all these data and abstractions, but they're not really getting at The intelligence of my five-year-old right one last comment is that forget about just the intelligence of human beings I have alluded to economics earlier You know a system that brings food into a city every day is intelligent adaptive it works in all situations I've been working for 3,000 years. It's online blah blah blah. It's intelligent If you want to use the word intelligence for something that we understand something about and there's theory of microeconomics It's not theory that predicts every single transaction, but it's a guide to understanding the sort of thing And so that's a kind of intelligence why we put more of that in our computers and not worry so much about This thing that we're still very very ignorant about I Think it's I think this is right. I think it's easy to get a little bit distracted or seduced by the sort of more general Concepts like intelligence artificial or otherwise, you know tactically concretely on the ground I mean, I think the three main shortcomings right now are Number one robustness you've heard this talked about By various members of the panel a lot of these algorithms have an awful lot of crutches seen or unseen and Things tend to fall apart pretty quickly when when you move those our algorithms Utilizations instances outside of the narrow comfort zones, you know, for example Take out your new iPhone with a fancy camera. It'll do Perfectly fine, you know sort of portrait detection But if you'd like your cat to be the focus of the portrait, you know, no way only works for humans not animals So there's there's kind of no, you know, no real robustness built in any of these things Secondly, I think transparency I talked about this before as these algorithms become more and more pervasive in society The need to understand the basis for how they arrive at a particular decision is going to become more important And we have pretty limited progress there right now and thirdly I think and I think Mike used this word abstraction Slightly different from robustness, but we just we don't We we've treated this as you said in a very bottom-up engineering way and we haven't really thought I think Enough yet about how to make some of these algorithms and algorithmic concepts more abstract able So that we'll be able to kind of repurpose across and and make things successful in slightly different scenarios than the ones for which They were originally envisioned so I mean those are three very practical Immediate but still kind of big and consequential areas where I think we need a lot of progress So unfortunately we are almost time So let me give you a quick announcement that so the panel we're finishing up But then we will have a seminar at 430 in the active learning center That's a theater So if you go to the active learning center go in the door and then turn right and then you will see a big theater So we will meet there at 430 and everyone's welcome tell your friends to come and professor Michael Jordan will give a seminar there All right. Thank you very much