 Welcome everybody. Thank you so much for being here. My name is Danny Laguna. I'm an assistant professor here in the iSchool. Hello everybody who is here live. And I I guess I've been here for for some years now and I do research in many areas but mostly in trying to understand science and a lot of the things that I will present today or I would talk about today I use in my own research so And I would today I will present about applied deep learning I teach two courses primarily big data analytics where we use We try to understand why big data is so popular and what kind of value can it can bring to your organization and Apply deep learning where we look at software packages and things that we can use directly in and many different kinds of Mods of media images text and things like that so first If I can go to the next slide, please. I'd like to just acknowledge the help of many colleagues and students who have who helped me In my research. I run the science of science computational and computational discovery lab here And and here's a sample of the funding. So thank you very much to the funders And I just wanted to start by just saying the the final message of this talk, which is I believe deep learning offers a tremendous opportunity to businesses and Because of any size and any industry And that's shifted from from some years ago where most of the value Was being Taken advantage of by big companies in stuff today. I think any business of any size can take advantage of this new technology and The idea is that we can use the data that your organization produces I Guess I can remove these I'm just too conditioned to use a mask everywhere. I probably sleep with masks so You can use the data says that your organization produces for for and analyze them with these kinds of technologies and also we have these software packages that allow you to to download models and and Techniques that have been developed by big companies so you can take advantage of that now the key in my view is that your Organization needs to have the right kind of data scientists as the right kind of Understanding that it's just not it's not just developing and getting the data But it's also the infrastructure around using this technology So it offers a lot of opportunities and a lot of limitations to not everything can be solved with deep learning so you have to be careful about that and lastly but Perhaps even more importantly moving forward is that a lot of these technologies that I will talk about today hide complexity and they hide ethical issues and that has implications Lee in the legal dimension and ethical dimensions, too. I don't I don't know what time it is, but please let me know if I run out of time All right, so much you learning and deep learning Historically have been used by big technology Tech companies, you know Google the usual the usual suspects Google Facebook and things like that and If you if you look at what they say about how they use AI and how much AI kind of trickles down into the organization to drive up revenue Recent surveys show that For almost a fifth of them They see an increase of six to ten percent in revenue improvement through the use of AI and mature learning And Kobe just has just accelerated it is because you don't have Interactions you don't have really a lot of times you can just skip many steps When you're trying to sell a product or service and then you can use AI to try to replace many of the steps that Usually took place in the real world. So this has just accelerated over the last two years So let me just give you an example of what I mean by this So I'm sure you are all familiar with the process of buying houses And that's a very I mean traditionally is a very long process where you you have an agent You go and look at many various houses And then you you say, okay, this house is great. I totally see it leaving here And you say, okay, I'm gonna offer this amount for this house Even though the owner is saying, okay, you should pay, you know, $20. You say, okay, I'm gonna pay $19 and Then there is a negotiation process with the agent. This is very complicated and it takes, you know It might take several days. I don't know about these days But maybe in the past usually it takes, you know, several and sometimes weeks But if you think about it, it's a very convoluted process And there is a lot of uncertainty from from everybody like it's a very mysterious process But ultimately what you would really like is to try to and say, okay What is the final selling value of the house? Like after all of these back and forth You want to say, okay, I think this house is gonna be sold at this price So maybe the buyer can say, okay, I'm gonna offer the final value and the buyer will say, okay That seems fair. So they agree and the deal can be closed in 10 seconds instead of being a two-week process Now The process is mysterious, but if you think about it with websites such as Silo They have a lot of data about this about many many places in the country Many kinds of houses that have been bought and sold Of all kinds, some shapes from different buyers. So if you think about it They have a lot of features about the house Okay So you in real estate people talk about location But it's a lot of other stuff when you go and look at a place even to rent you click at the pictures So the picture is giving you for some information about the house How nice it looks how much light you get from outside And that all these kind of set of things that you use mysteriously in your brain For predicting how much you're gonna offer. Okay, so if it looks outdated you say that that asking price is crazy So I'm just gonna offer something lower or if it looks great and it's in the neighborhood looks great You might make another offer. So There are things that many Many features of the of the offer of the place Can inform How much is gonna eventually we've sold for and so it's the pictures the location to and and things about the buyer itself so The next slide the next slide please So So this is usually how how this works and you you get the day you well You formulate a problem you get the data in the case of Z-Lo they get, you know Hundreds of thousands of data points where you have this is the house And this is the location. This is that picture and then they try to predict, you know a two-week process How will how much this house will be sold for and that's what they call as estimate Okay, so you deploy the model But then there are all these processes where you have to tune the model you have to deploy it Make it available for for the users, which in this case is just Z-Lo But you can make it available for other companies and then you have to monitor Maybe it's making some mistakes or you have to update The next slide please now in data science We usually don't think too much about these things when you when you are teaching in an environment like this in the university But I tried to well, we tried to teach it here in the school There is all these of this infrastructure around building these models So not just getting the data, but it's also the deployment of the of the of the models So you have to create API you have to create versions and also you have to monitor your things And this has become very important in the Recently because you might have you know biases you might have you might be breaking the law Because you are maybe you're taking the zip code in in the prediction Okay, in the prediction of the value and you know without you realizing In the US at least You know areas of certain cities are very segregated. So the zip code might In some sense be informing the the prediction and making all kinds of bias predictions About that so next slide please so So data science is this kind of role that kind of feels that is like a full pack full stack data mining role that goes from You know getting the data having a little bit of domain expertise developing the model fine-tuning them and then Deploying them and and monitoring them. So here's a survey of how much time people spend on different on all of these Tasks in data science and you know Building the model and thinking about these things is just a tiny fraction of the rest you need to explore the data logic deploy it and Clean the data so next slide please All right. So how is what what is deep learning? What's what's going on here? So because this talk is about deep apply deep learning So what's happened is that in the past you could do the things that I show you you can say Okay, I have this data set and I can kind of skip the middle man and make me the middle woman and make a prediction About the say this the price of house, but what has happened over the last Ten years I would say or maybe even five years is that in the past? Let's say if you look at this cartoonish plots of How big the models have been in the past like this these models that take the features and make the prediction How much computation of power you need to make the predictions what has happened is that in the past? since we're like slowly Increasing in size in computational power. Okay, but around 12 2012 There was a break in that trend in that to now Get predictions that were really good You needed to start increasing the amount of power that you put into this model by match more and what happened there in 2012 2012 was the introduction of this idea of deep learning. Okay, and I will explain what that means, but basically Now for your organization to take advantage of these kinds of models You need much bigger and complicated models more data more computational power So I will suggest that you look at this economist article from open AI that goes through this idea The next slide please So, okay, so what is deep learning? So if you think about it artificial intelligence is a very broad goal of trying to reproduce intelligence like we what we all think is intelligence in the in the in nature Virtually or artificially in a machine. So that's a very broad Goal that has both like technical implications and applied implications, but also has philosophical implications Now if you if you look if you kind of specialize more you can think of much you learning where you're trying to use data to Build models that learn from that data and improve future performance So you want to take you know the zero data sets and make a prediction so that you reduce the error between the The asking price sorry the selling price and the actual prediction that you are making about the price of the house So that's a very standard thing that has been going on for 50 years But now deep learning is a is a more specialized form of material learning which you're using a specific kind of model for making such Tradition and the specific kind of model is based on how the brain works or at least is Inspired by how the brain works and more specifically about how neuron neural networks work in the brain So here's my favorite kind of Structure that we have in the brain This is from the visual system when you are looking at something Let's say you're detecting food or maybe you're trying to look for a place without snow You you are looking with your eyes and you can think of of the eyes kind of capturing the input from the environment And what's cool about the visual system in the brain is that? That light that goes into the eye Hits an area the primary visual area of your brain and that area through various Experiments in neuroscience. We have determined that they compute like little features about the environment like little you know Borders little colors like very simple things Okay So the first thing that that the brain computes are like these very simple features about the environment What's cool is that once you start stacking this area once this the primary visual system sends The signal what he detected these borders into the next area the next area then combines those features hierarchically So you get you get things now that are more like shapes instead of just being borders It's more like okay. How many squares there are how many circles and then you keep going up and up Until at some point there is an area in the brain where there is a neuron that will fire if you see you know a delicious Sandwich or you see you know no it's not or something like so there is this hierarchical Structure where you start detecting simple things and it goes all the way up to very complex concepts faces and things like that so artificial neural networks try to reproduce that in the computer by creating By extracting simp the three basically simple concepts the concept of a neuron Which is a something that receives an input or a set of inputs and it makes a very simple computation The other concept is that that's output of the of the neuron connects to other neurons and the third concept is that that Connection usually is through layers. So you have layers of neurons that connect to the next layer of your neurons And you don't really have connections within a layer Next slide please. So here I'm showing you an artificial neural network. I'm taking this from three blue one brown a great YouTube channel. So And this is a classic example of how a neural network learns to detect digits Okay So the inputs for this network is an image of 28 by 28 pixel Grayscale image and the output is a label that says okay I think these new and these numbers number two these numbers number nine and so on and so forth and Here I have a very simple fully what's called a fully collected neural network. This is artificial So the impulse are the pixels and then I have two hidden layers And I just initialize the network to just having random Random weights so it starts randomly and it starts just guessing So you pass the pixels and he makes the prediction and then what's cool about this is that we make the network So that is differentiable. So we can compute the Difference in error that will make if I change any of the connections in my network Okay, so through a process coil call forward propagation We compute the errors that we're making and then we can back propagate those errors to fix the neuron Okay, so through these processes by showing examples and labels Show example and labels the network will start learning and the idea of deep neural network Is that now we've gone through just two neurons of any two layers? You can see these layers here up to now We are like hundred layers and that's what called that's what called deep and the deeper you go the more sophisticated features are the next slide please now what I've seen is that in the past To apply these kinds of models you needed, you know very specialized knowledge You needed to know Some special obscure parts of the language to implement this neural network to yourself But what has happened over the last five years is that there are all these packages PyTorch TensorFlow MXnet that allow you to anybody really any organization to take these Models and apply them to any data set they have so and is all very standard and It's backed by big companies like Facebook Google and Amazon and there is infrastructure now for taking these models Deploying them on the cloud. So there is it's all these this kind of support that allows you To use these kinds of technologies in your data set. So next slide, please So what is now what's applied deep learning because deep learning could be researched you kind of are developing new kinds of architectures You are processing new kinds of data But applied deep learning is really just taking what other people or other organizations have done not really created a new architecture By taking these little pieces and combining them So you might want to connect combine, you know images with texts and you might want to combine those things So apply deep learning is really this idea that you can take the big data set that you have you can use them You have a storage for For storing data that you want to learn from you have access to GPUs to be computational power power infrastructures and You have access to this much sophisticated models and methods of training these systems So here I'm showing you an example of something that would have been crazy ten years ago well, maybe maybe 20 years ago where This is a classic task in computer vision where you want to detect objects that are present in an image So you take a picture and you want to see if there are people there dogs, you know, if there are all You know cars and things like that So something that would have taken you know a couple of PhDs to to to build now It takes like two lines of code in Python. Okay, so you can use something like pytorch and say Which is a package backed by Google and you can say okay take this model that was developed by Google It cost them, you know hundreds of thousands of dollars to train them on millions of images. Take that download it And then I'm gonna detect based on an image that I have in my organization I'm gonna take the objects that are inside. Okay, so that makes it very easy for any organization to use this So next line, please So let me see let's review a couple of applications of this So well health care, I think is a is an obvious application because many many processes in health care You think about it about them. They involved, you know, and the doctor making a decision usually a visual inspection of some sort of an exam. Yeah, so here's an example of cancer detection in skin cancer and So if you think about it for a doctor to make the decision of whether let's say a mold It's suspicious and maybe you need to do it like a biopsy They have to they have to train right they go to med school and In somewhere there is like a cancer class. Let's say and they show different pictures You say this is a normal mold is like You know the borders are even and it's not growing and the color is like brownish that color clear brown so these are fine and then these are These are not fine. These are like irregular. They're growing in size. The color is a little darker So this you should pay attention to this. Okay, so the the the dermatologist will will learn this separation of these two classes and It will make a decision once it's in the practice It will you will look at a skin of someone as well. I think this is based on my experience. This doesn't look right So maybe we need to do a follow-up Now because we have huge data sets and here this is the work that was published a couple of years ago in nature They took data from UK. They have the national health system. I think school So they have hundreds of thousands of of images and they have the expert opinion of a dermatologist They can feed these images through computer vision Deep learning systems that can make the prediction from the image just the raw pixels into saying this is okay And this is not okay Okay, and this these kinds of networks are called convolutional neural networks and the idea is that you pass these filters through The image and as I was saying before they the first kind of filters detect simple features like borders and things like that But then once you move up in the hierarchy, let's start detecting more sophisticated things like irregular borders or colors and things like that And now here where I'm showing is is the kind of the true positive rate the false Both the kinds of mistakes that the dermatologists are making and the the closer you are to to here to this corner The better okay, and all these dots here just show different dermatologists where they are located What kind of errors they're making and what's cool is that this deep learning Network actually is making in general less mistakes compared to the experts and on average This is the average here. I know if you can see it on the screen. There is a really a little green dot The average from many many dermatologists is lower in performance than Than a computer vision system Now there are many caveats. There have been many follow-ups of these research So it's not completely a hundred percent true because the data said wasn't like completely Okay, and there was a lot of there was a lot of overfitting but in general we see this like We see the performance of computer vision systems approaching human performance Okay, so next slide please Now the applications of these you know, it's really just this this is just one application with cancer research But you can think of something like precision mason where we're trying to to produce a Program for each individual, right? So we're trying to produce a treatment program based on the particular history of each individual So if you think about mason over the last 50 years because of time and because of expenses We kind of have these rules of what we should do with certain Symptoms this is we should do this and because you know, this is what applies to most people So in general on average, this is good, but for individuals it might be very different So the treatment might be very specialized. So precision mason tries to make that now in the past to do precision Mason you will need a team of doctors looking at a case and kind of Tweaking the treatment for the person But what we could do we could take an AI system and we can take data from previous Judgments on individual cases and say, okay, you given these symptoms What should be the treatment for that this particular person? So we can include AI into that to make that that prediction for each individual so next slide please So recently there was I guess I guess this year there was this breakthrough on another Kind of problem, which is protein folding It turns out that a lot of the functionality in our brain in our body and in our brains is made by this protein That are just sophisticated machines that move inside cells that are 3d structures and they perform various tasks They're like mini machines. Okay, so if you see them live, it's pretty fascinating five minutes. Okay, good. So Now the idea is that there are all these these we don't really understand how the the this 3d structure is computed from the from the initial encoding of the of the of this protein so we need it will be great to understand how that how that process goes and and I Guess this year there was this deep learning Technique that took the sequence of amino acids that predicted very accurately much more accurately any any any before Anyone before the structure of the of the protein. Okay, so if I can move here Now there are more like cute applications of these technologies So if you use Instagram that are always filters that you can apply now this very sophisticated deep learning techniques that allow you to parse an image To parse a face and detect where the where the eyes are and things like that And you can and then based on that they and they run really fast at 30 frames per second and they can start You know understanding the the 3d structure of a face and then put You know different kinds of of things on top of it There are all other apps that can reverse H you they can make you younger or older Because they have learned the progress of the image of someone Translation if you speak another language you've seen the tremendous progress in translation the same thing because they have these massive Data sets and they can learn and do equally well almost to translate now in my own research I I use this model so with text Analysis I am using at these models a bird for example a big model for Processing text. I'm also processing images and it's just remarkable how well I can tell The location of pieces of images of graphs and detect whether that's trying to mislead other scientists So I'm gonna move now. This is a cool new development where we have these huge Models that are trained on different tasks and these are called foundation models And this is the latest and greatest and you can read more about this just Google foundation models And it's the next iteration of deep learning so I wanted to finish by kind of By kind of warning that The problem with these models deep learning models that they hide a lot of bias So here's an example a really nasty example of a bias that was hidden in an API in Google So if you if you painted the hands differently The model was guessing that's you know a white hand was holding a monocular whereas a black hand was holding a gun So what happens with these models that they're so complicated that is it unclear how it's making the decision So it's really hard to inspect what's going on inside. So I highly recommend this book that goes through many of these examples Okay, and with that I will finish we have research in our own lab where we have similar things where some Text techniques are bias against woman and against minorities. So we have to be in general careful about apply deep learning Okay, so with that I'll finish. So thank you very much. So maybe just One or two quick questions. Yeah, also, it's a nice segue to the next talk So you talked about ethics the whole next talk is actually going to talk about ethics as well There's a question that came up. You mentioned skin cancer detection and I actually talked about that even yesterday So it's pretty common one to talk about Does that work equally well depending on your color of your skin or not? No, actually, that's a great question. So he has a real real trouble with darker skins that's that's a problem that across computer vision and Because the examples that they that the systems have are bias and most of the examples are of certain ethnicities And they're from the UK. So it's already bias. So my understanding is that doesn't work too well with dark skin Yeah, and one last question So under what circumstances is deep learning better than traditional machine learning algorithms Or maybe another way to say that is under what circumstances are traditional machine learning algorithms still useful? Yeah, I think you should definitely always try traditional machine learning algorithms deep learning Works mostly when you have a lot of data already or with somebody else train These foundation models in a bigger data set. So if you have small data sets or it's very noisy Definitely don't go with deep learning. Just it will overfit your data