 Welcome to this course on applied linear algebra. So before we get started, I just like a quick chit chat or brief introduction of sorts. What are your expectations, except for those of you for whom it's a core course, you have no choice. But I presume there's a number of you who've chosen to credit this course, despite it not being thrust upon you. So I would like to know what your expectations are from this course, where you see yourself at the end of this course, what you should be able to do, what you have expectations out of this course. Maybe that way we can have a clear dialogue about what can be met and what cannot be met during this course. Because quite often this term applied in the name applied linear algebra gives some misconception. So I'd like to clarify that right at the beginning. So if anyone could like to volunteer and say, what do you have as part of your expectations from this course? Is this your first course for most of you in linear algebra? OK, for quite a number of you. For many of you may not be the first course. If that is not so, then it's particularly important that you mention what you expect out of this course. Because if you have done a course on linear algebra earlier, we can assume safely that you have some advanced expectations from this course, presumably. So what would that be? Anyone care to volunteer? Yes? OK, OK. We'll hope you shall not be disappointed anybody else. Maybe some problem that you encountered in while you're doing your research or something that has prompted you to have some renewed interest in linear algebra, and thereby you've thought maybe it's a good time to credit this course. Anyone of that sort? Any specific research problem that you're looking into where you've found an application of linear algebra? OK, even if it's not so, it's fine. But the reason why I ask the question is because when I say applied linear algebra, and since it's offered as part of the electrical engineering curriculum, people tend to think there'll be lots and lots of engineering applications that this course is going to be filled with to burst that balloon. That's not the idea of this course. So then why do we call it applied linear algebra? Some of you may agree. Some of you may not agree with the justification that I'm going to provide now. We are still going to learn about ways of proving things, ways of testing, hypothesis. Hypothesis as in you pose it something to be true, and then you check out whether it's true or not. So that requires a certain level of mathematical sophistication, which we hope to empower you with during the course of this particular subject, E635. Now if you have expectations of dealing with too many applications, the reason why I say that's not a fair expectation to have is because this class has, I presume, people from diverse backgrounds. So if I delve too much into, say, mechanical systems or electrical systems or some particular kind of systems and problems in that domain, then the risk that I run is of switching the others off. However, if we just teach you some advanced proof techniques, some results, which we shall hope you will find useful during the course of your research, no matter what domain you are from, it is agnostic to your background. Then we feel that sooner than later, you will find the contents of this course useful. So that is what we imply by applied linear algebra. Not as in we'll have one or the other application on every particular lecture. We will try to give you a flavor for some of the applications, but this is not going to be a course filled with too many applications at every juncture. Rather, we are going to follow a rather conventional route in the sense we're going to follow a math book. I've already sent out a message on Moodle. The textbooks we shall be following are mainly the one by Hoffman and Kunz and also the other by Sheldon Axler, which is linear algebra done right. We will send out problems every now and then, problem sheets, a subset of those problems will be part of your assignments and you're expected to practice the other problems as well, not just the ones given in the assignment because they'll not just help you in preparing for your exams, but also in solidifying your concepts, right. So that is my expectations from you. You are of course free to communicate your expectations to me by any means during the lectures, after the lectures, during office hours, over Moodle. All those channels are open and legitimate, right. So at any point in time you feel like, you would like a little bit of light shared on a particular aspect, feel absolutely free to reach out. Even though this is being recorded, I want to make it absolutely clear. You're absolutely free to interrupt me at any point in time and ask your questions as if the camera doesn't exist. This is a regular classroom lecture, okay. So that's it. The added advantage you will of course have is you'll have access to the videos, yeah. That's the bonus. Nothing less, nothing more than that. So why do we do algebra? Is it just because it's an elegant sounding subject in mathematics? When were you first introduced to algebra? Middle school, high school, right? What was your idea of algebra? When you moved from arithmetic to algebra, what are the first things that popped up immediately? Variables, symbols, right, symbols. So you are given to understand that this symbol has a particular connotation, yeah. This symbol has a particular connotation, multiplication or so on and so forth. But you learned it in a very conventional manner that this means unambiguously a particular kind of operation being carried out on numbers, which is simple addition, right. But algebra allows you to go beyond these structures and say that it does not need to be confined to a particular kind of operation. Operations could be anything that we want it to be, provided there is a structure. There are certain rule bases to be followed, right. And we want to expand our horizons and not just restrict ourselves to just the conventional operations you have learned. So what are some of the simplest kind of equations that you solved, right? Maybe 7th standard, 8th standard, you were probably solving linear equations. Equations that look something like 2x plus 3y is equal to 5, 7x plus 8y is equal to 15. Something similar? And you applied some known techniques. What were the known techniques? Before you learned about matrices, you are still solving these equations if I am not mistaken, right. You knew how to solve them. What was it you wanted to eliminate one variable? You wanted to cook up, if I dare say, another equation out of these two equations in a special manner by subtracting a certain scaled factor of this equation from the first equation, yeah. As we shall see, we will be doing the same thing throughout this one part of this course, yeah. There are mainly two problems. To put it very simply, there are mainly two problems we shall be focusing on in this course. And that sounds like, oh, that's really simple, right. There's just two problems in a graduate level course. But we shall see there are a lot more things than meet the eye at first glance, even in those two problems, right. So, this of course, you know that you multiply this whole equation by seven and probably this whole equation by minus two and then you add them up together and you get a resultant equation, which depends only on y and then you can solve for the variable y, yeah. But what if I told you that I want you to give me the solutions of an equation that cannot be any arbitrary number, but some special kind of numbers, right. So, let's say I give you an equation like this. X cubed plus y cubed is equal to x plus y. It's just one equation, right. Not even a linear equation now, right. I'm saying that I want solutions x, y that belong to, by the way, get used to these sort of notations. This is belongs to, yeah. So, both of them have to be rational numbers. What do you do? You get stuck, not as simple as that high school example that we had just given a while back, right. But it is in these cases that algebra comes to our aid. Of course, not linear algebra anymore, yeah. What you can do is you can treat them, you know, this particular sort of equation as a trick to solve this. You just take y upon x as some small q. I mean, I can leave it to you as an exercise. Since this is not directly pertaining to linear equation, not to our course, but just an interesting example, maybe try this and substitute this in the first equation here. There's only one equation, though. So, what do you have? Can you guess what's the technique? So, you can just, you know, substitute this. So, you have x cubed plus y cubed. What will that turn out to be? Can you replace all of the symptoms of q by eliminating one of those variables? y is qx, right. So, instead of writing y, I can write this as q cubed x cubed. And on the right-hand side too, what can I do? I can do the same thing here. x plus qx, yeah. How does that help? Any idea? I'll not take you too far down this line. One solution you can readily see is x is equal to zero. But that, of course, is a trivial case. That doesn't come under this ambit because then you wouldn't have been able to do this substitution in the first place, right? For the other solutions, what do you do? You, of course, take x common out of this. So, what you have is x squared plus q squared x squared is equal to one plus q. And then you can just do this. x squared is equal to one plus q upon one plus q squared. Right? Q? Q cubed. Q cubed, yeah, sorry. Right? Is that very satisfactory? Not yet, right? Because you want to find x. But once you have x, what is y in terms of x? y is nothing but qx. So, I am saying you can go ahead and substitute all possible rational numbers q and that will give you a solution. So, you have one equation and you can have an infinity of solutions, right? So, this is something we are uncomfortable with. Of course, we might encounter such problems in daily life for some research problem maybe. But this doesn't have too good a structure in this, right? We are used to this thought process where we have n equations. We know there has got to be this one solution. So, we love those unique solution and those cases, right? But what if the solutions are not unique? We want to characterize all possible solutions. As it turns out in this course, even though we shall not be dealing with such equations as these, we shall be dealing with simple linear equations. It will turn out to be the case that when the number of equations does not equal the number of unknown variables, you might often land up with systems of equations where there is no unique solution. There may be either no solution, there may be unique solution or there may be a multitude of solutions. In which case, we want to characterize all those solutions. We are not happy with any one particular solution and it's a degree of freedom that is granted to you because sometimes a particular engineering problem or any application problem might demand out of you that you give the best possible solution in some sense. So, when you have a multitude of solutions to pick from, you want to somehow choose the best possible solution among them. Even when you have no solution for that matter, you want to still get as close as possible. Think about it like this. There's a balloon that's floating about in the sky and there's a small child that's running after the balloon but the child doesn't have wings. Child wants to get as close to the balloon as possible, maybe never with any hope of grasping it with his hands. What does the child do? The closest the child can get to the balloon is stand right below it. We know this from standard geometry, assuming flat earth. Not propounding flat earth beliefs but assuming a flat earth, that's the closest you get. And it is exactly these sort of things that we shall study in a much more abstract sense in this course. Best possible solution. Earlier you've been taught to, I mean, resign yourself to the fact that when no solution exists for a system of linear equations, we just give up and that's it, you get full marks for that and you're happy with it. But when you're doing research, you don't stop at that juncture. You must explore all possibilities. Even when no solution exists, you must still try to get as close as possible. What do you mean by as close as possible? This requires the development of certain theory, certain structure behind whatever objects we'll be dealing with. That is exactly the goal of this course, right? I'll give you a couple more examples, which again will appear like they're sort of not linear equations. So let's take this, okay? Not linear equations, three unknowns, three variables nonetheless, right? Three unknowns, three equations, sorry, not variables, three unknowns, three equations, okay? So where does linear equations come into the picture in all of this, you might wonder, right? So let's try some trick here, shall we? Let's say I multiply this first equation by, sorry, x, is it? No, I don't think it should be x, I should multiply it by y. I should multiply the second equation by z and the third equation by x and then add them up, what results? What results on the left-hand side, right? So I have ay plus bz plus cx is equal to zero, right? Let's try another thing, shall we? Let's now multiply this, of course I'm using a different color, it's a different operation, not doing it simultaneously. So let's multiply this by z now. Let's multiply this equation by x and let's multiply this equation by y. Again, what happens to the left-hand side? Zero, right? What happens to the right-hand side? What equation do I land up with? A z bx plus bx plus cy. Plus bx plus cy. And what do we have? A couple of linear equations. What is the point of this exercise? I can leave it to you at this point to work this out. One way to work this out is, you know, what does this mean? What is this an equation of? Think a little geometrically. It's an equation of a plane. What is this an equation of, also a plane? What do you mean when you say that two planes are intersecting? It's a line. So of course, the solution is not unique. Any point on that line of intersection of those two planes is a solution. But is it really true that the solution of this equation is not unique? Do you think so? Because it has to meet some very specific conditions here. So first, you find out the line of intersection of these two planes, which you can do by any method you like. I might suggest you can take a cross product. Right? How does that work? It's a cross product, no? Is it not? So you can find out the solution in terms of x, y and z in terms of some parameter. Parameterize it in some terms of some parameter. Say alpha. And then you can choose that alpha by plugging back in any one of these equations. Right? Please ask if it's not very clear. Not that it's very important for this course as such. What I'm trying to show you is that even if you start with something that looks nothing like a linear equation or a system of linear equations, there might be ways lurking within which allow you to massage it into something that is very much a system of linear equations which underscores the importance of the subject applied linear algebra, which is why you should probably be studying this course. Right? So apart from this, there are also other examples of equations that might look very elegant and though not linear, but can be transformed into linear equations. So for instance, let's say now I give you another system of equations, but this is going to be a rather special case because in the sense that not the techniques that you've learned so far will be handy. Yeah, those won't be handy anymore. So let's say you have x cubed plus y cubed plus z cubed is equal to 3 and x plus y plus z is also equal to 3. Right? And I want you to find integer solutions. It's what we call Diophantine equations. Okay? So this is not the conventional method, right? Where you just plug in any real number. This is integer solutions. As it is, this is not a linear equation. This is a linear equation, but this isn't. So you might wonder what is this doing here? Why are we devoting so much time to this equation? Again, I'll not complete the entire process, but I'll just outline what you can do in this case. What can you do? Well, you can just say x plus y plus z, the whole cubed. Anyone remember the expression? x cubed plus y cubed plus z cubed plus. What's a pretty convenient way of writing this? You agree? That's right. Is it not? Now, what can we say about this? It's three cubed. That's 27. What can we say about this? It's given to be three. So what you have is 24 is equal to three times this, right? In other words, if I divide it out, excuse my jumping steps here, I can just write eight is equal to x plus y into y plus z into z plus x. And this still looks nothing like a linear equation, but I tell you there is a linear system of equations lurking here. Why? Because we are looking for solutions x, y, z that belong to the set of integers. So if they have to be integers, each of them individually have to be integers. So we split this up into its prime factors, all possible splitting up, positive, negative, everything that you've got. Plug it back in and solve for them simultaneously. See which of them give you integer solutions, right? And there's something beautiful else also here, which is the symmetry. So if something works out for x, y, z, you can just flip it around a bit and it'll work for a different choice of y, z, x and so on and so forth, right? So you have to consider all possible prime, all possible prime factorizations of this number and then combine them in three tuples. The product of three of those numbers will have to be this. So prime factors means there could be more than, in this case it just happens to be two cubed, but you can split them up at any junction. Again, the problem here is that there are so many systems of equations. Every one choice is going to land up with one system of three linear equations, which you then have to solve and check if the solution is indeed an integer. If it is, then you admit it. If not, then you discard it. So you can try and complete this again as an exercise, just to get a feel for the sort of things. Again, up until this point, if you had seen this equation, you might not have figured that this is a stand amount of solving a system of linear equations, right? But linear equations as it turns out are everywhere, okay? And in order to understand linear equations, sometimes it is very important to understand the geometry behind them. And when we want to understand the geometry behind them, we might need a clear idea of certain things that we understand in two-dimensional or three-dimensional spaces. Again, I use the term dimension very loosely. We have not yet defined it. We will define it, in fact, maybe in fifth or sixth lecture or thereabout, right? You might think this is surprising. Dimension, we all understand what it is. We live in the three-dimensional space. This board is two-dimensional. Why is it two-dimensional? Why is the space we are living in three-dimensional? What is dimension? We will question these fundamental notions and understand them through the filter of linear algebra. That is the idea of this course, okay? Let me give you a kind of a puzzle, okay? Again, which turns out to be predicated, the solution of which is predicated on an understanding of basic geometry, okay? So, I've been a fan of the Beatles. I don't know if these days you hear them, but suppose there are four Beatles. You can call them Beatles or you can replace this with E so they become insects, which is how we are going to use them. So, you have four Beatles on a plane. Let's call them John, Paul, George and Ringo, okay? They all start on this plane and go in different directions so that no two of them are moving along some parallel lines. So, obviously, it stands to reason that their paths will crisscross at some point, yeah? You agree? Okay, this looks very parallel. Let's just tweak it a bit. Their paths definitely cross, but I haven't told you yet that they meet each other. So, remember, they're moving with constant speeds. That's the nugget of info that I'm giving you. So, the four Beatles are moving at constant speeds. The speeds can be different, okay? There's no obligation for them to have the same speed. The speeds are different, but constant speed nonetheless, yeah? Their paths definitely cross each other. Another piece of information because they're not, no two of them are parallel, okay? Third piece of information is now, now see there are these pairs, right? Except for one pair, any one pair, you take your pick, maybe Paul and Ringo, the two who are alive, okay? So Paul and Ringo, we don't know if they have met, but we certainly know that John has met Paul, George and Ringo, okay? George has met Paul, Ringo and John, yeah? But we don't know about this one pair, except for them, every other pair not only has their path crossed, yeah? But they've also met each other. So they have actually collided. They've been there simultaneously at the same junction where their paths have crisscrossed. The question is, is it true that somehow, you know the two who are alive, that is Ringo and Paul, they must also meet each other? What do you think? Is it necessary that given that all the other pairs have met except for this one pair, is it necessary that this pair of Paul and Ringo must also meet? Is it? What do you say that? Yes, I heard a yes, okay? No, I'm not saying they have met at the same time. They have met at different, each pair has rather met at a different instant of time. It's not true that they have all met at the same point. I've never said that. So all these intersections are happening at different instance of time. The question is, is this intersection that I've drawn here also going to imply that they're going to meet at the same time? The paths obviously cross. Any two lines that are not parallel on a plane, they'll cross each other. So the crossing of the paths is not a question. That point is resolved. The point is, do they also meet? Which means, do they arrive at that point of intersection of their paths at the same instant? Yes? Why? Yeah, but the relative velocity is a vector, right? Yeah. Yeah, it's a vector on this plane, okay? Yeah. So you're going to write all those bunch of equations and then try and solve this. These two equations. Two equations, which two? One with the... Ringo and Paul is the one that we are debating. The other ones? And say what about it? John, let's say. Yeah? Paul and John. Paul and John, okay? And we supplied the two equations. Okay. But they have met at different instance. I mean, Ringo and John have met at, say, at time instant T1. And Paul and Ringo have met at time instance, let's say T2. At different time instance. And the velocities are v1, v2, v3, v4. I mean, look at the number of variables that you have already, right? And then if you resolve them along the x direction and y direction, you have v1, x, v2, x, v2, you know? v1, x, v1, y, v2, x, v2, y, you have already eight variables in the plane. And then you have to equate them and say that, oh, hang on, this is an initial condition. So this is x1, 0, y1, 0, x2, 0, y2, 0. I mean, look at the number of variables in the construction that you end up with already. I mean, there must be some trick somewhere, isn't it? Is it something that we are missing? Yes? Mm-hmm. No, no, different speeds, but constant speeds. V1, v2, v3, v4, not the same speed, but constant speeds. So let me give you a hint and maybe not, maybe just give you the solution itself because the hint is the solution. So that's one thing we are missing here. And I'll tell you what, it's not a shortcoming or anything. We often think problems in lower dimensions are simpler to solve. Lesser variables, easier to visualize. I mean, who would go to five dimensions, 10 dimensions, things that you cannot even visualize if you can actually solve the problem in three dimensions? Right? Wrong. That's what this problem illustrates. You're looking at it in two dimensions. If you're talking about meeting of people, not just their paths, what about the third axis which is the axis called time? Remember, the speeds are constant. So the time axis multiplied by the velocity or the speed is just a distance, is it not? It's a constant speed, so it's not curved or anything. It's just a straight line axis, is it not? So when you're saying that two people, not just their paths crossing, but they're also meeting, it means their x and y coordinates are meeting and their v1, t and v2, t's, those are also meeting. So the third coordinate, which is the time which we can draw like the z axis here, is also the same. If you're saying just the paths meet, it's just that they agree in x and y. But if they also meet each other, they agree not just in x and y, but also along this vt. Now think of this problem in 3D. Let me erase this and remove the clutter a bit. So things, hopefully, I'm not a good artist, but hopefully it will be clearer. So let's say you have something like this. Now x and y is the plane and t is the third axis. So what happens? I've told you that there is this one pair of lines that is one pair of beetles who've met. What do a pair of lines imply? They define a plane, is it not? So this is a plane. I'm not even naming the beetles anymore separately. I'm just taking any three of them who have definitely met each other. The fourth one is the one under question. The fourth one has met two of them. Whether the fourth one meets the third also is the question, right? So this is two of them. Now when I say that the third fellow has met both of them, that means the line corresponding to the third fellow must also lie in the same plane, is it not? Right? But we've not said anything about them meeting at the same point at the same time, right? They may not, okay. But you'll see that soon enough when you include the fourth fellow to also meet at least two of them, then it means that it must also be in the same plane. So if the fourth fellow has met two of them, it's also lying on the same plane as those defined by the other three and then it must also meet the other line because they are not parallel. No solution of any equations, no variables, nothing. Just a simple understanding of 3D, geometry, right? So sometimes abstraction is good. Even if you might argue that, well, you've bloated a problem in two dimensions in a planar problem, two, three dimensions. So that's the reason why we need to understand where a smart choice of a variable can yield or lend an otherwise, render an otherwise intractable problem, intractable, right? So of course, this course is not just going to be about these fun puzzles and tricks and all. This is just motivation today being the introductory lecture. Is it thought maybe, you know, just have a bit of fun delving into some of these questions and puzzles and at least give some motivation behind why we engage ourselves in this question of solving linear systems of equations.