 Okay, then hi everyone My name is Karthik. I'm a graduate student in Singapore. I study biomedical engineering I'm doing my research and computer vision is a very big part of the work that I do And so I just like to give a bit of an introduction. I Did my undergraduate studies in mechanical engineering, so I didn't really have much of an But I wanted to get involved because I wanted to get involved in cancer research and the opportunity was to study and classify different types of cancer to Clinicians with identifying diseases classifying them in medical images Non-invasive imaging seems like a very promising, you know field you can go for an MRI scan CT scan and x-ray and it helps to sort of identify Whether you have an anomaly like a tumor fracture, but it's not enough to diagnose a disease and what needs to be done if in order to do that is to Take a tissue sample from the anomaly, right? So when if you go to a for surgery to resect a tumor to remove a tumor the surgeon essentially takes a few tissue samples from The tumor site and then that's handed over to pathologist But the intermediate step of doing that is to stain the sample and so histopathology is the is the field that I work in Work with so what's done to the sample is it's put you know stains applied It's left for a few for a while and you get a contrast in the image that allows you to see features like nuclei or surrounding tissue This is a very big eosin or H&E staining and What it does is the hematose island binds to the nuclei the nucleic acids the eosin binds to the other elements of the cytoplasm and you can essentially see the features like nuclei blood blood cells so on so forth and Features in these images help a histopathologist. Sorry a pathologist Identify what kind of disease it is if it's a cancer for example, you know cancer is very Proliferative apologies The cells reproduce very quickly and it gets very cellular. There's a lot of You know the nuclei can can look very morphologically Mophology is a very fancy term for shape They can look I'm normal in shape. They can be elongated. They can be oversized And that helps a histopath pathologist determine what kind of disease it is Not just that whether it's cancer or not cancer But what type of cancer it is and it's very important that they do that in order to Guide treatment response You know to to determine what kind of chemotherapy the patient needs or even to determine things like clinical outcomes Prognosis is it real? Is it a very bad? sort of disease is it already necrosis have the cells died And Because it's a human doing the job. There's of course a lot of into observer variability, you know humans We have our own biases there's also You know you can have labs without pathologists available in developing countries and it's become quite popular in recent research to have computers come in and you know train to train them out to identify these Diseases in the images to either help clinicians or to even exist as some sort of in the intermediary between The the patient and the doctor when you need to when they need to do diagnosis when the clinician isn't available and When I got started It was quite interesting. I didn't really have the resources Knowledge-wise to understand how to do these sort of classifications How do I I was essentially using a lot of open-source software and I was applying I was applying it to to identify cells within the image to study their shapes And I was using the the numerical values of things like shape Circularity, you know, are the cells very circular or are they very eccentric? Are they elongated are they large are they small so on so forth? I was using that to do classification without an underlying understanding of what computer vision actually is. How does it work? Because I wanted to produce result. Oh, sorry. Okay, is it better now? Think I've got to come closer to the mic so the you know the more I studied these More I looked at online resources and the more I understood the principles behind The way these things work the more I was able to appreciate these methods and why we do Why we modify them why we use them? What are the problems we might face? One of the things that inspired me a lot was the fact that the computer vision community is actually a very generous community you can actually see a lot of resources Provided to both learn about these techniques and to apply them You know if you don't have the computer computational resources, for example YouTubers like Aladdin Pearson, you know, they they upload videos on Yeah, exactly this They upload videos on you know, how do you use these techniques how to apply them in code? How do you you know how they work? You know and and they make it open to many people and if you really if you want to learn this from scratch It's not exactly very difficult So this is actually one lecture in many many many lectures that you can find online If you really want to get involved in the subject The other thing is to apply these techniques. You need a certain level of computational resource if you're here because you probably have a graphics Processing unit like a new Nvidia GPU But if you don't have those resources available, you know that they've also made that online For example the Google collab, you know gets you get you access through the internet to GPUs GPU resources, I think you can use that for free for about 12 hours continuously So if you want to apply your code if you want to run it if you want to try these techniques Maybe not the really really complex ones like VR that Professor Geogrowski was talking about in the last time, but If you want to apply the basic techniques and see for yourself how some basic AI works The resources are available and I was inspired by this generosity in the community And I thought it would be fine a fine idea to present those a bit of basics at the science circle to get everyone You know anyone who's interested started or at least to start a conversation But computer vision is interesting because it has you know, it's becoming more prevalent in many areas of You know our daily lives or In the clinical context for example, it's being used to diagnose images to to identify disease Where you know you have self-driving cars They they do make use to an extent A computer vision enables thing them, you know cars to see things like other vehicles bends in the roads Objects like traffic lights traffic cones You can identify these things and then you can guide Vehicle in its own automated automated driving It's also used in in things like snapchat or instagram filters that you know the things where you could put cat ears or or your masks Or even makeup on your face and or you could torment your friends with it the idea is You know it is everywhere to becoming more and more prevalent And I think that it's important for us to understand how these things work at the base level And in order to do that I I would probably start by talking about what a digital image is and what what it's comprised of and The first thing I did was I I took an image of a a chair It's from here I uploaded it to mad lab and What I got in response was this description of that image and You will break this down, right? It says 231 by 208 by 3 Uint 8 so 231 by 208 is in the in the Number of pixels in the image and the pixel is the fundamental element in describing an image Um a pixel is assigned a color You have those pixels you put them together you get horizontal rows horizontal and Horizontal and vertical rows or pixels and that together comprises an image So what's the what's the time three then? Now obviously These elements, you know They're not just you don't have a grayscale image in this case It's it's a color image, right? And you want to break that down into the different color channels There's a red channel a green channel a blue channel you can see that Where the the blue channel picture on the right is concerned And if you compare it to the red channel image on the left You see it's the chair is actually brighter in the blue channel image because it contains a lot more Blue blue color, but we can break images down into those three Components so each pixel has about three values if we want to see that in more simplicity, you know, we can Use a nine pixel image, right? you notice that there You know when we represent it in the grayscale color mappings we can see that There are values that are assigned to each of these Color color channels and these are called arrays And this is a three by three matrix matrix essentially, so what's ui and ta then that was the last Uh descriptor for the image. It's basically it basically refers to you know An an eight-bit integer, right and it contains numbers from zero to 255 That's essentially a scale for brightness Zero is black the absence of color A 255 can be totally white, right? So if it's a if it's red and it's completely red, it fits pure red then that that would be 255 in red And then if you look at the green array, it's zero in blue. It's zero If you look if you are looking at the at white for example the bottom right hand Corner you look at those channels. All of them are 255 if you add that together you get you get white Black is just the absence of color. So it's zero in all three of those channels And so we can put those together and we can get different colors and we can describe them um describe an image But um and and there are different ways to represent image images, right? An rgb image is what I described earlier We can of course turn that into a grayscale image, which is just zero to 255 for each pixel and that's like black to white I just like to make Address your comments or questions immediately. I don't have that skill unfortunately. So, um I'll try and do that after the Oh, no resolution depends on the number of pixels Assigned to the same Image so like for example Realize I should have talked about resolution. I didn't You could take the same Image of that chair It's two three one by two zero eight now, right? Let's say we increase the number of pixels assigned to this same image You essentially you're essentially seeing the same thing But you'd get a higher resolution you get better quality And you'd be able to you know, you'd see the edges a lot more clearly you get to see those The curves in the chair would be a lot more, you know defined It would be a lot more interesting. So resolution is essentially just the It's a if you want to look at image quality It's the number of pixels that are assigned to the same image What's that affects the size? Yeah, exactly. It's the same to When they compare to your TV screens Um, no, it's not it's not going to be diagnostic imaging. I'll talk about that later Okay, so I do I'm not talking about the Points function. I better research that Sorry We can represent images a few ways one is We can do it in RGB red, green and blue. That's the one I described earlier We can do it in gray scale, right? That's an easier way to represent an image zero to 255 the we can also You know, if we apply a threshold Let's say we introduce it about half the Half half of 255 We can we can essentially convert an image to a binary image, right? represented in zeros and ones So so that those are different ways to to sort of represent image images The next thing we're going to do is we're going to try and Identify features in an image and I realize this is not really intuitive one of the things that computer vision allowed me to You know, help me do was appreciate how easy it was for me to uh To see something and intuitively sort of immediately determine Assetting what sort of object it is I didn't I don't have to think about pixels when we don't have to think about pixels or pixel values of gray scale imaging or RGB When we when we talk about is a chair a chair is a car a car is that a cat or is that a dog? but when we use When we upload pictures to computers and we try and manipulate them we essentially need to play around first but having this Matrix of numbers this representation allows us to manipulate the images to extract features And that's essentially what this The first part of this lecture is going to cover So one of the things we can use to determine Distinguished images for example, if you're talking about cars or motorbikes or aeroplanes for example you you you want to look you There are certain things that help you determine what sort of vehicles these these are A propeller or a turbine engine on an aeroplane would help you determine that it's an aeroplane Acetylene that it's an aeroplane the wings on an aeroplane You know a paramount to knowing that it's an aeroplane on a motorbike for example You have two wheels on a car you have four wheels Just like if you're differentiating a cat or a human a human has two legs a cat has a has a different A cat has four four legs So you you would essentially be looking at these high high level features We call it in order to determine an acetane for yourself whether or not those you know are those The what what sort of image is it referring to what sort of object it is in the in the image So We want the computer to be able to do the same thing in order Image it is and in order to do that we first have to start from the basics the One of the things that you know, we start from We start from lines edges and dots It's these are called low level features. We want the easiest thing to extract from You know an array of of numbers Is lines edges and dots And these are called low level features. We put those together and we can get the mid level features We can get high level features. Um, I I took this image out from Um An open source sorry a website online, but I think uh, Alexander Amini who's a lecturer at MIT He has a very good lecture on uh deep learning that I suggest you follow up on and he's got a better diagram than I do I didn't want to copy that but uh, what he he uses it to introduce low level up mid level and high level features On human images the low level features are essentially the lines the edges the dots the mid level features are the the facial features for example the nose the eyes the ears So on so forth the high level features are then the face mapping and you know knowing those low the mid level features like where the ears are where the eyes are That facial mapping that would therefore allow us to identify Um, you know those features for snapchat filters, for example, where do you put the ears? Where do you put the nose? That and how do you distort the image to make it more interesting? I suppose But let's talk about the low level features. How do we how do we extract those first? The way we do that Is we apply something called a filter through the to the image And filter is just you know, it's a it's another array. It's a smaller You could it could be a three by three array like this And what we do is we move that filter along the image There might be different step sizes involved But essentially what happens is if you look at it, we multiply we do an element wise multiplication here Look at the red array and look at the look at the red matrix and look at the blue blue matrix We're essentially multiplying the elements in the same position So one times one plus zero times zero plus zero times one Plus one times zero plus one times one plus zero times zero plus one times one plus zero times zero plus one times one All that added together gives us the four on the right and that's sort of like a feature map the We could apply it to the one below right, uh, the red the red box that I've highlighted It's the same thing One times zero is one one times one is one Essentially the the sum is is three and this yeah, it's called a convolution. I'll come into I'll go into convolutional neural networks later on but this essentially gives us a feature map of You know from the original image it helps us to pick out features And you know, this doesn't this may not seem a lot to you like a lot to you when I'm looking when we're looking at it numerically, but If you take the image that I used for the promoting this lecture and the abstract there was There's an edge detection operator. These these are essentially Sobal filters they call it It's interesting because these are edge detection Filters what they do Is either detect horizontal or vertical edges. You notice if I apply the first one look look clear quite closely at the text the sine circle you'll notice that The application of different filters allows for either the detection of vertical edges or the detection of horizontal Edges so we can by changing, you know the values in these filters these convolutions We can essentially detect Different things and the way it works is by identifying the difference As an image goes from gray to black or black to white, you know, as it gets brighter as it gets darker You start to see these these edges and as you apply those multiplications You start to see it starts to become more definite when you get the output feature map and so That's very interesting because we can now make use of those features to identify those mid level or high level features like face eyes and all that and I'd like to Talk a bit about neural networks because the next thing we're going to do is we're going to put these features into a neural network And we're going to use that to apply a simple artificial intelligence protocol to identify to to get the computer to classify an image and This is the basic structure of a neural network First off, I want to make a disclaimer. This is a very simple image that I found online The lines that are going from each input can actually go through every other input in the next next layer Each vertical column of circles is called a layer the And each circle refers to a neuron And the neuron could be an input It could be you know an output of the multiplication and essentially what's done is The features from anything that you're trying to classify placed That x1 x2 x3 Up to x33 x34. This is 36 features in this in this case What's done is these features Multiplied and ended and added You know to buy a certain number of weights The the connecting lines are essentially The channels and they are they are multiplied by weights in order to give us the next layer Which is called the hidden layer the which is you know the features Extracted So what what's done is these multiplications happen over and over and over until We come to the last layer of neurons and in this case it's three Sorry, it's three neurons each neuron corresponds to an output So in this case, there are three classes. It might be cat dog cow, whatever What happens is when we apply these these multiplications we add them together We can either we we send them into something called an activation function And an activation function. It's a sort of threshold that determines whether or not A neuron will be activated or not activated You can see that some of the neurons in this in What that means is after passing the sum of those weights multiplied by the sum of the features It's reached a certain value that has caused it to activate And these activations continue until You know, we reach the output and it determines a certain class And this is called forward propagations the the multiplications are happening in the forward direction But this isn't the end of the story Because you know this the computer is essentially just making a guess the weights are not, you know The weights may not really be optimal enough for it to give a correct classification One of the things that I think George Professor Jokowski has previously talked about that I should have mentioned in this lecture is The difference between unsupervised and supervised learning This is a case of supervised learning what we fed into the computer is not just the input Whether it's an image or whether it's a set of numbers that correspond to a certain diagnosis or classification We've also told the computer what the classification is We've told the computer that this image is a is a is a cow. This image is a cat This image is a car. This image is a plane and by feeding these classifications into the computer We're essentially See what's going on and in order to train the network the computer essentially wants to get these classifications correct So when it identifies by the end of the forward propagation You know that Run That it's not reached the actual class. It means that there are errors involved And it will use those errors to go back to you know to to go back into these Into the layers of the network to modify the weights In order for it to achieve an optimal result that reflects the actual class And it will keep doing this forward backwards forward and backwards until It reach it gets an optimal classification that corresponds with the values that it was fed And I think this is one of the most it's a beautiful thing about artificial intelligence That a computer is able to you know by itself Learn you know back and forth and back and forth. We do have to make some mathematical sort of adjustments How many layers do we use? Do we increase the number of hidden layers? Do we use something like a drop out? I won't go into into that but You know this this Cycle of forward propagation back propagation doing this over and over again. That's essentially what Deep learning is and it's Once we've done that we can On images that it hasn't seen before and that essentially You know would allow us to classify new images and to use it in contexts like healthcare And we and it's it's an amazing thing. That's what I think but this is this is a very this is just an artificial neural network Uh since this year, I think was mentioning something Be called Convolutions, right? So there's a convolutional. There's something called a convolutional neural network If we're dealing with images, we are essentially using a convolutional neural network And this is what that is so What this does is essentially it applies the concepts that we learned That we discussed earlier A convolutional neural network is quite similar to an a and n the interesting thing is What I discussed before the convolutions that are applied to the image, you know those Filters that are applied to the image the ones that determine and and You know extract the features like the lines the basic features like lines Horizontal lines vertical lines edges so on so forth. Those are the same weights Those are weights that are applied to the pixels and Those are the weights that the computer wants to optimize in order for it to perform patients that those classifications and so The purpose of a cnn is not just to One preserve the feature features that are paramount to getting a robust prediction It wants to extract those features as it goes along those low level features like lines and Edges and so on so forth would then translate As it goes down the network it would translate into high mid level and high level features And by the time it comes out into a fully connected layer It you know at the end it would be able to classify the image based on high level features But it also wants to simplify the images into forms that are easier to process. What do I mean by this? You know this training with a neural network. It takes a lot of one memory and two it takes a lot of On my laptop, I remember that my graphics card and my cpu Heated up to about 96 degrees when I was training my convolutional neural network Not the smartest idea And I was training it for about an hour. So an hour at that temperature, you know things were frying I and so so one of the ways You know the cnn deals with it is along with the convolutional layers after every convolution You get the matrix result and What happens is you then apply something called pooling and pooling serves to Reduce, you know the size of a matrix while preserving the features as much as possible There are two different types of pooling. One is called max pooling If you look at we this four by four matrix We've essentially broken it down into four components And we take the maximum value from each of the components and we transfer it to a smaller matrix That's max pooling average pooling of course takes into account It just takes the sum of the components divides it and then gives you an average value One of the advantages of using a pooling technique is it gets rid of unnecessary noise for example in the image so It's a convolutional layer and then a max pooling layer a convolutional layer a max pooling layer feature Feature extraction simplification feature extraction simplification so that it's easier to process and at the end You know We looked at the artificial neural network earlier. It was a single row of neurons. Yes So what we would do is we would take the two dimensional image and we would flatten it Into a single sort of Vector and that would be used in the neural network to classify the image To be This simple right you notice that there are only two sets of convolution and max pooling layers in this image it doesn't This is the vgg 16 network. It's it's uh, it was proposed in a research paper. It contains about 16 of these convolutional layers and max pooling layers all that added together It essentially does a lot of feature extraction and the the term deep neural networks The depth of a neural network is refers to how many layers of these convolutions and so on so forth that you have and we would We would essentially you would essentially apply these to your problems Based on the complexity of the image that you have if you have a lot of features in the image that need to be detected If you have a lot of those if the complexity is very high, then you probably need a deeper network It's not always a case that you are you know, when you apply a deeper network, you get a better result I do i'm working on a research paper that I found out that my simpler network Actually produced better results than this vgg 16 network Basically use these for a multitude of tasks. They're different sort of computer Computer vision classification tasks that are involved a few one of the basic ones is of course image classification the simplest ones the ones that I learned in my Online were like the distinction between cats and dogs for example, right? So if For example, if you You know, if you want to classify a cat and you want to classify a dog you're looking at different features like ears Um, how they're pointed out. What's the shape? What's the morphology? In cancer detection, for example, is that is it a can is it a benign cancer? Sorry, is it benign tissue? Is it cancer? That's the simplest way to think about it, right? Um, but you could also be doing things like cancer subtyping Um, there are different there. There are correlations between the morphology of nuclei in tissue images that correspond to a certain Uh, genomic classification of the tissue. So and that would sort of help us Determine whether or not the cancer the patient has a better prognosis Or will they respond to a certain sort of treatment better? and It's it's becoming more and more interesting to use Deep learning in these applications because they're able to see things Simultaneously, they're able to study a lot of images together for large large numbers of images the I've been using about 10 000 for training and Uh, I I just find the number of The fact that you can feed a lot of images into these networks to train and differentiate them at the same time Interesting But one of the things that's important or one of the challenges that's involved with image image classification is The fact that you know the the same class of image, you know, uh, a dog Uh A dog could be standing they an image of a dog could be captured from the side, right? The dog could be leaping in the air or it could be sitting down on its hind legs You wouldn't be able to see the front legs um in the staining for example in In staining for example, one of the big problems is that there's a lot of stain variants in the images when one of the problems that we had with our pathology images or The h&e images that we were using is that when they were exposed to white light microscopy bright light microscopy The stain started to wear off And so you get some samples in which the contrast was very good Some samples in which the contrast between the hematose island and eosin the pink and the blue wasn't so Clear and when you have those differences in images, you can create You're trying to train a system that has looked at a certain set of images A system that hasn't looked at a test set of images, which is how we evaluate these deep learning techniques we can If if it's not able to take into account these differences, then it's a weak uh classifier So how do we deal with it? How do we how do we solve those issues? Well, one of the ways is uh data augmentation and it sounds like a fancy term. It's not It's essentially just a very sneaky protocol in which we might Take an original image and create You know clones of it But modified clones. For example, you could stretch The image of a dog If if you capture A dog from a certain angle, of course, you know, you would see that it's sort of squeezed Just like the image on the top, right? So we could essentially augment images In many different ways to account for those variations when we can't see them in the dataset that we have We could even apply color color filters or you know, um sort of color modifications that allow for For the computer account to account for things like, you know, when you're driving the Is there rain? Is it are you doing in the in it in the night? And light variations and so on and so forth. So so this is an interest Another thing is if you don't have a very large dataset to who to operate on you Can essentially use something called transfer learning and transfer learning is very interesting because People have uploaded complex complex networks like inception inception v3 Which was developed by google vgg16, which is the one that I showed you earlier. It's a very complex network and If we don't have a very large dataset to train these networks on we are not able to extract features We're not able to see features well enough for us to build a robust network One of the ways to deal with that is to use pre-trained networks And people have generously also uploaded these networks. They train them on something like image net an image net is a repository of How should I say it? Maybe I should use Donald Trump's way of saying it millions and millions and millions of him To put it precisely 14 million images of many different things like cats and dogs Boats and cars and shoes and all those things It's a huge repository of images And these networks have essentially been trained on the image net database and they've already been trained to Identify features in these images You could essentially use the train networks the transfer learning networks That are that have already been trained So if you don't have a very large dataset to train your Your network on to to have it identify features you could use transfer learning methods and You know what you what you do is like the convolutional layers would essentially have the features already mapped to them they'd be looking for those features in the images the end would be the The artificial neural network you could do the classification that way Um, I found that is a very useful in dealing with my studies earlier on When we had very limited samples and we wanted to test the effectiveness to show the doctors that this thing actually works With some level of accuracy The other advantage of transfer learning of course is if you have a lot of data and you're trying to train a neural network with it You need a lot of memory. You need a lot of computational power And if you don't have that, you know, uh, this having this already pre trained saves you from burning your computer and The the next thing we could of course you do You know, once we've trained our computer to classify images is we could have it identify objects within images Right, we could pass a window over the image and we could essentially get it to classify areas within the window and Have it ascertain One of the objects in the image and that's object detected if we're talking Testing one two three Yeah, I switched. Um My My my expensive microphone is not working Jeez It's uh But but let's take it a step further. Um I'm using my laptops mic now. So sorry about that Sorry about that Let's take it a step further. Um, we could we could continue from from object detection and from object feature segmentation from from, um Detection we could essentially do object tracking. We could observe where people are going, you know, in a day to see what they are preferred modes of transportation are um, and we could optimize Uh, the you know transport systems and all that by seeing these images Of course, there's there will be probably big sort of debates about You know privacy and and those sort of things when we when we do this but you know It enables video rate Tracking is also quite interesting for using probes in medical imaging. For example When you want to see live images and Adopters using a probe on a tissue sample and wants to identify tumor within the tissue the the probes moving, right? So if you want to track the tumor as it goes across, that's also another thing that you could do Getting getting these systems to work In video rate techniques is a very interesting challenge that's also ongoing And we could also do things like extract segmentation This slide this slide's got an error. It's not object tracking. It's called object segmentation And what it's trying to do is extract things like nuclei from From the cells if we want to study the shape of each nuclei separately, we could essentially Extract and we need we want to study the boundaries of the shape that's involved. That's an advanced level mode of image classification It's not only identifying what the image is whether it's a person. It's a car whether it's traffic light It's also identifying the boundary of the image in order to understand where its limits are and that could be, you know further help the guidance of systems like self-driving vehicles one affair but What I really want to get get back at here is that You know, I've shown you these techniques. I've I've mentioned To some with some level of simplicity the I've talked about what the basis of these techniques are what an image is what how you process it And how you you know how a computer extracts features from it What I'd like to encourage a little bit if you're interested in these techniques and you want to continue to lose them I need to emphasize that the online learning community for deep learning is extensive for computer vision. It's amazing um I mentioned Aladdin Pearson and uh, Alexander Amini. These are they have Produced some profound lectures really. Um, great lectures about the the topic Alexander Ami, uh, sorry Aladdin Pearson actually gave me insights into how I should install the software in my computer because I had a lot of problems Installing and getting it to link with my GPU and watching his videos actually saved my My butt Actually got me and it helped me to get started. I am I am still quite grateful for the work that these people have done And I think if you're really interested you can actually check out There these these channels I recommend them I've also mentioned, um, open open source sort of things When I started doing deep learning I was learning it on google collab Google collab essentially it's an online coding interface that allows you to use google's gpu's public resource gpu's If you want to perform some basic deep learning classification Or try these techniques for yourself or get involved in AI It's not really that far away and it's not really a niche industry If you want to you can watch those videos. You can use these resources online if you want um Kaggle is another platform that provides open source deep learning You know resources You could use these and use them to your Advantage if you want to you know further your projects to to you to do classification or to study trends or patterns in your In your data sets But the most important thing I think is that AI is AI is becoming more prevalent in in discussions as it becomes more prevalent in industries and our daily lives A lot of articles have come out about You know moral dilemmas, but whether or not these we should be scared of these things Whether or not they are really feasible whether or not they're really useful It's I think I think that an understanding of the basics an understanding of You know pixels and all the boring stuff that I've shown you so far is useful in helping us make objective assumptions or you know objective Or having objective discussions about these topics There's a big Thing about artificial intelligence. It's called the AI black box. It's where a certain network is trained But a person like a doctor who's going to use the network to make a diagnosis at the end Doesn't really know what it's gone into training the network And I think an understanding of these techniques The way they work There are a lot more things that I haven't mentioned the cnn is just one way of doing these this analysis There are things like support vector machines linear discriminant analysis that older machine learning techniques That you could also use But it's an understanding of the basics that will probably help us alleviate The trepidation that we might face with this black box problem and with that I don't really have a thank you slide because I wanted to save 10 lindens on uploading But I really want to thank you for this opportunity. I understand that it's the sign circles 500 presentation And I think I really want to thank Shanta and You know the team like Phil George Professor Professor George and many of the other people who have come thus far to present some amazing topics And to to talk to talk about these and to To deliver this knowledge from the public I hope you have enjoyed this presentation. I really enjoyed preparing it for you Thank you so much Yeah, maybe if there there's some things that we want to wanted to talk about I think Shiloh was mentioning the idea about 3d imaging, right? so I don't I don't I haven't really dealt a lot with the 3d images yet But I do know I would think that a 3d image is actually it consists of slices of 2d images An MRI image for example, you know, you scan along a certain axis You would get sets of 2d images and I would think that if you want to classify a 3d image You would essentially use the classification in the 2d space and apply it to those The slices and then reconstruct the image in 3d There are some amazing things like finite element analysis for example Where an image is broken down into smaller smaller objects and classification and things are done to those to those objects and to those smaller segments of 3d object You do the classification you perform the reconstruction and then you might get a 3d mapping for example A heat map of tumor per se and yeah, that's That's the way I would think 3d images are handled Uh, yeah, I got a private message from sumo that I think is going to make me cry Uh, thank you so much. Uh, it's so kind of the I I really wanted to this lecture has a lot of missing content. I have to be honest with you. Um I think uh ccg mentioned raster Sorry, uh, the point spread function. There's not, uh, I haven't talked about that. I haven't talked about you know, uh Gaussian curves in pixel intensities, uh, I There are there are a lot more topics that we could we could raise from computer vision I could come up with a more comprehensive lecture on this Eventually, but I think I should also leave you in the hands of the many people who have come before me and upload it Even more profound content on the internet uh The you know the people from mit open course where for example incredible people that have Uploaded content Also talk about other areas of artificial intelligence um National, uh, sorry natural language processing for example, uh, the use, you know the study of words sentences computer computers are able to complete poems to some level of You know, uh beauty they're able to produce music these days As a musician, I I do find that a bit scary, but it's it's interesting. There are things that they can do uh processing signals sequences You know, we could have a lot more discussions about AI and the various things that we could apply it for Not just computer vision and I I just hope that this basic lecture continues to start a series of lectures If there's any one of you that's involved Um in in this then we could You know, we could continue A fireside chat would be yeah, it's a good idea Steven Ah, yeah, definitely because okay. I have to apologize. I had an exam this week on thursday um And next week. I think I'm going to try and work on completing my my paper for my manuscript so I'll discuss this with uh shantel and I think about it I think it has a lot to do with the data set. Um You know with patient samples in pathology one of the most One of the biggest concerning things that we have is When we when we train it on a certain patient cancer is a very heterogeneous disease For a certain patient, you can look like a certain the cells, you know They have a certain shape and size you could look at another sample from another patient. It's the same disease It could look completely different and It's so heterogeneous that training it on a few patient samples is not enough and if you want to My one of the biggest problems that we have when you know, publishing is Determining whether or not we've gotten enough stem samples From different hospitals So we we want it from Not just from different hospitals, but different hospitals might have different imaging quality right more The better funded hospitals with a better slight scanners for pathological images. You get a bit get better quality so it's about having the spread of a spread of images with a diversity that's Enough to have it Stem features Of course the training time is longer, but you read the benefits of having a more robust classifier I think yeah, I do think that that samples are actually very important. Oh, yeah, that's actually One of those one of the more detailed things to talk about In training a neural network is the splitting of data sets There is a training a validation and a test data set So a computer can be trained for if you have an entire data set of so many images You could break it down into 80 training and 20 test validation What the computer does is it trains on the training data So it's made a classifier on the training data It applies it to the validation data set that it hasn't seen yet And then it determines whether or not it's being biased or not And so, you know the validation validation data set is also is also quite important It's very important that a computer evaluates itself on Before that's the important thing You need to account and you know the people who help you split the data set. Those are very important the The doctors who who might have knowledge on what future samples will look like who have experience in diagnosing diseases The pathologist the surgeons these these are the people that are important, you know, helping you develop your Your data sets to choose to do the annotations. Where is the tumor? Where is the normal tissue? It's very important to have those people on your side If you're planning to do any project in artificial intelligence or deep learning I strongly suggest that you have those people on your side To to collaborate with them. It's it's a blessing that I actually got to meet doctors and work with them because They are fantastic. You get you get a lot of knowledge about disease From talking to doctors That was Yeah, it's it's and it's fun to to to work with them the experts The people you're trying to help One thing I learned in engineering is, you know, always try and work with the people you're trying to help And always try to empathize with their problems and understand it from their perspective Because the better you do that better you understand your your the problems engineering solutions are Not that I have a lot of experience with it, but in the brief years of education that I've had That was the biggest Lesson that I learned. Thank you very much I'd actually like to I I do invite more people to actually come up to talk about these topics I have to tell you that many of the concepts that I'm talking about now in in in fact this Coming up with this presentation was also very useful for me because I didn't I didn't really think about the basics as much before I I did take a module in computer vision um And I I did learn some some of those basics, but I You know in the midst of applying the techniques to problems and you know having those online resources Or a code readily available. You tend to forget about these principles about the building blocks that make up a classification that And and in a way pre preparing for this presentation was also very useful for me to get back to my roots To the roots of these things and to understand the basics of Me as well. So yeah, thank you. I really great Oh, yeah, um introoperative diagnosis is also very interesting The lab that I work with actually deals with that You can use the Now nowadays, uh, you know, you know pathology is an ex vivo method. What happens is The tissue sample is taken out of the patient And then it's processed outside the operating theater But now the big the big thing that i'm moving to is something called label free imaging While the patient is on the operating theater Sorry, it's on the operating table When you have it open and you can look at At the tissue samples, can you use a probe and then apply something like infrared radiation or Something called stimulated ramens but cross the spectroscopy, which is what The lab I work with deals with can we then you know use that to to sort of on the hospital Or sorry on the operating table Produce the pathological image. Uh, the she's the pathological image. So what you mentioned about You know, we need more pathologists for interoperational rapid diagnosis AI pathologists is actually something that's appearing Journal articles Getting a lot of very nice comments. I I will respond after this. I promise It was and I do encourage more people to to come up. Um, especially if they're if they're graduate students walking coming in I think one of the biggest concerns is that you don't have the expertise needed to You know do these presentations, which is a good thing Because it forces you to be resourceful and to go on to the internet and actually look for those answers So it's it's a it's a good idea to have these talks not just for not just for the audience but also for the presenter I agree Actually, if we're having a computer vision part two In relation to what Phil Phil has just mentioned, we can actually talk about things like general adversarial networks That's also a very interesting thing Um, it's where a computer has already learned how to how to diagnose images and what they look like And it produces images of the of you know, those classes. It generates artificial images Computers can now after learning about, you know, after looking at avatar images and also and so forth They can generate artificial images of humans of people of You know of objects, they can do those things and that that is also a very good A good method of evaluating an AI network It's to have a heavy because teaching is also like showing right what you know And that's also a very good thing to You know of the next presentation But you'll have to wait till my research goes to that level I would very much for my next paper like to explore G&S and other advanced techniques um A phantom it would be interesting, wouldn't it? Right? It takes the inputs from an MRI image. It knows what the components are and then imagine if it starts to 3D print its own A phantom image of a of a cancer Cancer tissue embedded within a brain I think with advanced 3D printing techniques It might it might be possible for a computer to do that That would be interesting I know my I know I think I have some friends who are working in advanced manufacturing Maybe I can ask them about it, but yeah phantoms are phantoms are used to evaluate Imaging technologies as well So if you can produce a phantom, it would be interesting You could probably produce what Shiloh mentioned An artificial tumor embedded within an artificial brain Then we could start asking ourselves questions like the matrix, right? What's real? How do you define real? Because everything everything would be a Maybe asking ourselves about the questions Thanks. There's no Izel What happens when we listen to oh, yeah, I could you could do a functional MRI, right? Because MRI MRI imaging allows us to see a neural function so another another method that's also being used for for the the detection of neural function is Let's see Optical diffuse optical tomography um And so so what that uses is infrared in the Within the brain So when you're you know when you're thinking or you're using a certain a certain part of your brain When neural activity is is going on, you know blood circulation might be altered and it's also able to detect that and You could definitely you could definitely apply a deep learning technique to To sort of see if you get enough data, I think Yeah, I am I'm not directly involved with the the imaging research Deep learning as far as as my research area is concerned. I think it's more of a supplement The real the real sort of Advancements or the interesting advancements are the technologies that are being developed to To you know use the to to to detect tumor for example Those the ones that use lasers or That sort of thing that's more that's actually more interesting what I You know Manage the large amounts of data that we're getting in the present age Or that we have already collected in the present age So it's more of a supplement. I would think of it as a supplement. It's good to have it in your It's good to have it as a data analysis too Yeah, I would I I have a grandparent with dementia and I know how bad it is so Uh, I use it not only to I use it for a picture analysis The the way I use mine is uh, I I take patches from the the h and e sample images and I Yeah, I think after this this question we can round up Thank you So those those tissue sample images what I do is I I actually Oh gosh Once you clip kicking on this it just does it on its own, but Ah, okay. So those uh, by the way, these are open source images These are not patient samples in case anyone gets triggered. These are I got this from a free repository on So I what I do is I I take small patches of images from these These larger samples of tissue and I do Patch thanks ccg for coming See Stay healthy The I I diagnose the I basically send those patches into a cnn But I've also done I've also done cell detection to study the shapes of individual cells Um That study didn't go so well because there was a lot of heterogeneity like I mentioned so um And an ai algorithm and a icnn to recognize cells is uh, it's called unit units the network That's very popular currently to for self cell recognition and segmentation That architecture I've used before Yeah, I think On that note, yeah, um 17 a.m in Singapore The coffee hasn't worn off and but you know, it's it's been a wonderful Opportunity. Thank you. Shiloh coming Thanks a lot. Shanta. Like I don't know how How to thank you for this opportunity. I hope it's been a good lecture It really was fun Going back to and delivering this Even though it's a lot nerve-wracking the first time let me pick up my stuff. So Hello, Phil How are you? I think it was very uh good presentation because I Read the identification of the notes in chat