 Greetings fellow learners! Now before getting into this enamoring world of embeddings, I have a thought-provoking question for you. What is your strategy to increase your vocabulary? So when I see a new word, I try to use that in a sentence that I organically come up with, and this is so that I can retain that knowledge for future use. And this strategy was super helpful when I was trying to learn different words for the GRE exam. So please comment down below your strategy for how would you learn new words in vocabulary, and I would love to get to know you better. Now this video is going to be divided into three passes. In the first pass we're gonna start with an overview of embeddings in general, then we're gonna get into some details, and then some code to visualize embeddings. So let's get to it. Let's talk about embeddings in the context of neural networks. Computers and neural networks don't understand words, they understand numbers. Hence before passing any data into the neural network, it needs to be converted into some numerical representation. This data is converted into vectors, and these vectors are basically n cross 1 matrices, and these vectors are also known as an input embedding. Now while the primary purpose of these embeddings is to get a numerical representation of data, they have other characteristics as well. So they capture the semantic meaning of data, the embeddings are typically continuous dense vectors, and embeddings have a compressed notation to represent data. Now let's talk about each of these in a little bit more detail. So embeddings capture the semantic meaning of data. So let's consider three words king, queen, and apple. Now the words king and queen are more similar to each other than they are to the word apple. For the neural network to understand this difference though, each word needs to be represented as a vector. So let's say that each word is represented by a two dimensional vector. This means that two numbers are required for representing every word. Now let's plot these words onto a two dimensional space called an embedding space. It's a space for plotting word embeddings and you can think of this as the neural network's understanding of the words. So if we humans say that king and queen are similar to each other than the word apple, then we want the neural network to eventually learn that the numeric representation of king is closer to the numeric representation of queen than it is to the numeric representation of the word apple. Now if this is the embedding space of the neural network that we are training, then we are doing well. Let's not talk about another characteristic. Embeddings are continuous dense vectors. Continuous means that the two numbers of the vector can take on any real number in theory and dense means that these values are likely non-zero. So with this characteristic, we can have vectors that don't even necessarily correspond to a specific word. The next characteristic is that embeddings offer a compressed notation. So sentences are represented by n-gram vectors. These are large vectors where each unit represents the presence and the counts of a specific n-gram in the input. Now these vectors are very large and cannot be processed by computers with reasonable memory and CPU resources. Hence a fixed size word representation called embeddings become useful. And now this extends to images as well. Raw images represented by RGB pixel values alone can be difficult to directly process and learn by neural networks. Hence they are transformed into some smaller internal representations. Raw audio too is typically in the form of a stream of samples or data points. This is far too big to process. Hence some compressed notation like embeddings become useful. Quiz time! Have you been paying attention? Let's quiz you to find out. What is the primary purpose of embeddings in the context of neural networks? A, to visualize raw data in a two-dimensional space. B, to compress data. C, to convert data into numeric representations. Or D, to increase the size of vector dimensions. Comment your answer down below and let's have a discussion. And at this point if you think I deserve it please do consider giving this video a like because it will help me out a lot. That's gonna do it for quiz time and pass one of the explanation but keep paying attention because I will be back to quiz you. Let's talk about where we see these embeddings more in practice. So this is a transformer neural network. It is used to solve sequence-to-sequence problems. And in NLP sentences can be a sequence of tokens. So we can give the network an English sentence and have it translate to a French sentence. But the network doesn't understand text. So each word needs to be converted into vectors or an input embedding. At different points in the network intermediate vectors can represent the input word two. These two are embeddings. And while the embeddings of the same word in different layers may be different, the characteristics remain the same. That is that each embedding is a numerical representation of the input. Each embedding semantically represents the meaning of the word. Each embedding is some continuous dense vector. And each embedding is some compressed notation of the original data. Now here's another application, the vision transformer. It's a modified transformer architecture to process images. In this case, it is modified to take in an image and output some classification of the image. And this is done by breaking the input image up into 16 cross 16 image patches. This is then passed through the feed forward layer. And we end up with a vector in this pink rectangle, which is an embedding. Now these embeddings also have the same characteristics as mentioned before. To repeat, each embedding is a numerical representation of the input. Each embedding semantically represents the image patch. Each embedding is a continuous dense vector. And each embedding is some compressed notation of the original data. Quiz time. It's that time of video again. Have you been paying attention? Let's quiz you to find out for this transformer image. What does this embedding represent? A, it's a vector encapsulating the meaning of the sentence. My name is a J. B, it's a vector encapsulating the meaning of the word name. C, it is a vector encapsulating the syntactic structure of the sentence. My name is a J. Or D, it is a vector encapsulating the syntactic structure of the word name. Comment your answer down below and let's have a discussion. Now that's going to do it for quiz time and past two of the explanation, but I will be back. So keep paying attention. Now in this past, I want to load and visualize some word embedding so that we get an idea of what they are and how they look. So to do so, we're going to be using a library called Jensim, which we install, we then import some libraries over here. Now with Jensim, we have a bunch of potential embedding generators here. In this case, each of these, we have word to VEC models, glove models, and fast text models. These individual models are trained to understand word embeddings, which we will now later load and visualize. In this case, we're going to take the glove Twitter 25. So we load the glove Twitter 25 model. We then create some words that we want to visualize. And we're going to obtain their word vectors for each of those words. Now we're going to take the first word, which is King. And this here is the 25 dimensional vector that represents King. So this is the embedding for the word King. Similarly, this here is the embedding for the word Queen. And this here is the embedding for the word Apple. Now, what I want to do is because 25 dimensions is too much for us to visualize, we are going to squish the 25 dimensions into just two dimensions and then visualize that in an embedding space. And we do this squishification using teesney teesney will perform this compression into just two components, which after iterating over all words, we get an embedding space that looks like this. So you can see that we have dimension two on the y axis and dimension one on the x axis. And we can plot all of the words there. And one thing we notice is that similar words are actually grouped together. So we have the embedding for Queen, King and Jack very close to each other, the embedding for the numbers, you know, eight, nine, six, 10, seven, all those numbers are together. And we also notice that the embedding for apple orange cherry plum and banana are also very close to each other. So what this means is that words that are actually close to each other and meaning are visually actually close to each other in the embedding space, which means that this glove model represents what it should represent and it is trained appropriately. Quiz time. This is going to be a fun one. Which of these is correct for the embeddings of apple, pear and person? A, the embeddings apple and pear are closer to each other than to person. B, the embeddings apple and person are closer to each other than to pear. C, the embeddings of pear and person are closer to each other than apple. Or D, the embeddings apple, pear and person are equidistant from each other. Comment your answer down below and let's have a discussion. And if at this point you think I do deserve it, please do consider giving this video a like because it will help me out a lot. Now that's going to do it for pass three and quiz three in this explanation. But before we go, let's generate a summary. Computers don't understand words, image, audio, they understand numbers. So they need to be converted into numeric representations like vectors for input and processing. Now these vectors are also known as embeddings. Now neural networks can internally also learn embeddings to represent the data. The embedding space shows how a neural network interprets this data. Closer the embeddings are in the embedding space. That means closer are their semantic meanings. And that's all that we have for today. If you want to further dive into the world of embeddings and see its utility and natural language processing and large language models, I highly recommend you check out this video right over here. Other than that, thank you all so much for watching. And if you think I do deserve it, please do give this video a like and I will see you in the next one. Bye bye.