 So, maybe I need to mention this here again I am going to assume specifically that my beta t the sequence they are increasing with beta 1 first element is going to be greater than or equals to 1. So, I am going to construct a confidence set right using CT to define the CT I need bts. I am just saying that that bts are all increasing with beta 1 being greater than or equals to 1 ok fine. So, let us now do regret analysis for this algorithm. So, what is the regret it is going to be in round t the regret in a particular round t is going to be this much right. So, let me write it. So, this is nothing but max. So, regret in round t. So, I am just going to use this notation r small t to denote this quantity r small t and r capital t is nothing but summation of all r t is that fine. So, r t is nothing but max over d theta star minus whatever you played d t into theta star. So, this is nothing but d t star theta star minus d t theta star. So, I am just denoting theta star to be what the one maximizes this. Notice that this d t star is different from this d t that you have played in that round t and that is this d t here ok. So, now, this is nothing but d t by d t it is just this. So, now, let us focus this. What is this quantity? This is nothing but I am just going to put it as this is nothing but I am just going to replace this u c b value again by this quantity here what I have written here. So, this is nothing but max of max of theta belongs to C t times d theta t right. I have just replaced this u c b quantity by this quantity. What it is basically doing? It is maximizing this product treating both of them as variables over theta as well as d. What such program is called? This is an optimization problem right. Is it a linear optimization problem? No right. What is such optimization problem? It is a bilinear optimization problem because both are variables for you. We are trying to optimize both over theta as well as d. It is a bilinear optimization problem ok. So, now, let us denote by theta d t theta. So, for I have to make this t minus 1 right ok. I am going to use what is that the way we have defined C t should be. I think C t only you are going to use all the quantities this is fine. I am now going to denote it as t and tilde. So, whatever the pair d and theta that maximizes this quantity I am going to denote it as d t and theta t tilde is that fine. So, this d t or which maximizes this I have already written is d t and the theta which maximizes that quantity this quantity I am going to write it as theta t tilde ok. So, this is the maximizing pair. This theta t tilde does it belong to C t? It belongs to C t right because that is where it is chosen from ok. So, now, let us try to understand this. So, before I write this is it true that if I write it as d t times theta tilde minus d t theta star is this correct? What I have done is this is a inner product between d t star and theta star right. I am saying that this is dominated by d t theta t star theta theta t tilde is this true? Why is this? I have already said that this pair d t theta t is the maximization of all possible ones ok and notice that I have already already used the fact that this guy C t contains theta star this guy contains theta star. So, because of this this pair should be dominated by this pair convinced ok fine. So, now, this is nothing but I can write it as theta tilde minus theta star that I have assumed right with high probability right as of now I am assumed. . Yeah that I have assumed. So, we have to later show that it is indeed possible I should be able to consider such a set where theta star belong to C t with high probability ok. So, I have assumed that that is why I am for I am safe here ok. We are going to do one more manipulation here what we will do is do this we will just add and subtract. So, I simply I have added and subtracted theta hat t minus 1. What is this theta hat t minus 1? This is the estimate I am going to get about theta star in round t minus 1 ok. So, now, let us simplify this this is nothing but. So, you will see that why I am doing all these manipulations here to get it in some certain required format. So, what I will now do is. So, I am going to club this and this and write it as theta t and theta t tilde minus theta t minus 1 hat plus theta t times theta t minus 1 hat minus theta star. Is this correct? It is the same thing right this is I have just taken this minus this and then this minus this and plugged it here both of them are getting multiplied with the term ok. So, now, let us focus on this term here ok. So, the first term here can I write it as this is nothing but by definition this is theta transpose and what I will do is I will deliberately get this term here and similarly the other term this is fine right I have just been nothing, I think I have just inserted a matrix and multiplied by its inverse. So, nothing changes. So, now what I will do is. So, now, this is nothing but inner product of these two vectors. Detected transpose V t minus 1 is an another row vector this matrix let us say assume its inverse exist and then by this column matrix. So, this matrix into this column matrix is another column matrix. So, this is nothing but product of these two these two vectors ok. I could also do the same thing for the other, but let me only just continue with this guy. So, by the way what is V t minus 1? So, it is nothing but we have say lambda i plus summation ds transpose ds ok. So, now, I am going to apply Cauchy shorts now we are going to apply Cauchy shorts. So, apply Cauchy shorts what you are going to get d t transpose V t minus 1 norm of this times norm of. So, just let me see this we need a square root or norm. This norm is enough right and this is the L 2 norm. So, is this similarly I can do it for the other also V t minus 1 this is fine right what is the issue that we will come that is the next step. As of now it is just like this is one vector this is another vector and this is the norm of that vector this is the norm of the second vector yeah. So, if you want to do it the other way just if you get make it a column vector if you want, but it does not matter it is a row vector or a column vector the norm is still going to be the same this norm is going to be. So, now what is this? This is nothing but d t in a product. So, this is nothing but in a product of d t with V t minus 1 with itself right I mean. So, right now I am messing up with columns and row vector, but this is one vector this is another vector and similarly this is also in a product between V t inverse theta t minus theta t hat times V t inverse theta t theta t is this correct? So, norm of a x is nothing but in a product of x with itself right. So, now, I need to have a square ok do I need it? No, I do not need it right like this is already squared some why you need it? Why you need a square root here? Yeah, but say here ok. So, I think I should have squared here also right I should have squared here ok that is fine let us see just let us see. So, let us understand this if we have in a product between x transpose and y. So, this is nothing but what summation x i y i and what now we are saying is this is upper bounded by sigma x i square and sigma y i square right and what is this giving us? This is giving us if I just say like this. The square. This is just the square sum yeah then I need to have a square of whole of this right. So, what is this quantity this is now at nothing but x transpose x right. So, this is this is a square root of this right. Yeah, this is like norm of x yeah square and norm of y squared right. Yes. So, let us now let us go with this we need to have a square root here for this consistent upper bounding let us take this. So, now, I am going to write this as this quantity here can I write it as d t with again it need to have a square ok and then what is this other quantity this is going to be theta tilde theta hat with respect to v t minus 1. So, this is v t minus 1 and inverse is this correct? Square. Square yes till this point it is correct we have just like by definition norm of d t with respect to v t minus 1 is exactly this definition. So, now we know that this is nothing but simply d t v t minus 1 and this quantity is again nothing but norm of theta t theta hat v t minus 1 minus 1. So, you notice that this one I got it for the first term ok. So, I can do a same thing everything remains same right except that my theta t tilde is replaced by theta t hat. So, if I do the same business here I am going to get this extra term. Transpose v t minus 1. Yeah. So, that is going to be like v t transpose v t minus 1 times v t minus 1 transpose times v t. Yeah. So, that is now this norm would be with respect to v t minus 1 times v t minus 1 transpose right. So, what you are saying this are not correct same. So, I know. Ok, let us see this. So, what is this? So, this is what by definition is this is nothing but inner product between these two right. Yes, yes. So, then. If you open this. Ok, let us open this v t transpose. v t minus 1. And then v t minus 1 transpose. v t minus 1 ok, you want to you are just taking a transpose of this right first or this is already a row vector ok. And then I want to take the transpose of this. Yes. So, I am going to get v t minus 1 transpose d t and what is this by definition? This is the same quantity right. v t minus 1 transpose when you define this norm with respect to this. Ok. So, ok fine let me write this. So, this is nothing but d t times v t minus 1 and. v t transpose v t minus 1 v t. So, this is what we have defined it right. Yes. Ok fine. So, there is this there is a difference between these two. So, let me see. v t minus 1 is symmetric Yeah, yeah. So, let me see. Suppose if I do this nothing changes right, I mean I will do that also similarly later there. So, this is also half and this is also half then this is half half and then this is I just keep it v t minus 1 only here. So, now everything is fine. Yeah fine. So, that is why this is going to be going to be the same. So, because of that if you just going to. So, what is this? This is half and this is another half bit transpose, but that will give you back v t. So, is this fine? Because this is a semi definite. So, therefore, we could define that. We could define that. Because Eigen values. Yeah, this is p s d right. So, we all the Eigen values are going to posit you. So, I am which line you are talking about where otherwise what we cannot have done. So, this is fine right. So, usually this kind of we are going to define this norm with respect to matrix if this matrix happens to be a positive semi definite or positive definite. So, positive definite. So, in this case I already know that this my matrix v t is already positive definite. So, is this now all fine? So, now let us focus on this part. Now, let us come to the big assumption I have made where I assume that sorry this one I said theta minus 1 right these are all theta minus t minus 1 why I have not written t minus 1 ok. So, this is all t minus 1. What I know? So, I should be a bit careful here what I want to do is I want to take the minus on its side and I want to take the plus on this side ok. So, this is minus minus this is plus this is going to be minus this is going to be plus and this is going to be plus here is that fine I have just and I have just made I have made this minus and this is plus because of that this is going to be everything is fine I have just like I have just replaced v t by v t inverse and this by v t just v t with without inverting it ok. So, now let us go back and see what our assumption says we have assumed that what is my c t? The c t kind of I assume that it is going to be of this form right theta such that theta minus theta minus 1 hat is going to be upper bounded by what beta t. Now, can I replace upper bound this quantity by beta t? Because I know that this quantity is upper bounded by beta t right. Can I also upper bound this quantity by beta t? See notice that the difference between this and this is this is theta t tilde, but it is still in c t right theta t is still in c t. So, I should be able to still bound that quantity this quantity is also by both this and this quantity by beta t right. So, now I that is why I am going to write it as 2 times norm of theta t v t minus 1 inverse times beta t. So, we had I think earlier we had this square right the way we have defined then I am going to upper bounded by what square root by beta t ok. So, now what is this? Now finally, what you have arrived is r t is upper bounded by this quantity right after all this manipulation manipulations and applying our exploiting our assumptions we are able to show r t is upper bounded by 2 times this not d t norm with respect to v t inverse and square root of beta. So, now let us see I want to now make another manipulation I know that r t is upper bounded by 2 also. Why is that? I know that by my first assumption I know that this mean quantity is the utmost one right absolute value of this is for each d is upper bounded by 1. So, the claim is then this quantity can be utmost 2 right if this is 1 and this is minus 1 then I am going to get that upper bound right. So, in any round that is why the regret can be either utmost 2 or upper bounded by this quantity is that correct? So, r t is upper bounded by min of and further I assume that this quantity beta t are increasing and they are at least 1. So, that is what I will also pull that out. So, this should be beta t square root of min of 1 times I mean just assume that if I replace this 1 by upper bounded by beta t then I will just pull out that square root of beta t then I am going to get it. So, finally, I have been able to reduce my r t to this format ok. So, now, now it all boils down to I know that I have already are known assume that there is some sequence. So, now the quantity is how to bound this quantity? This depends all the observations I have made so far right. So, how to bound this quantity? So, now let us focus on this ok. Now, the claim is this quantity here. So, what is now r t? r t is nothing, but summation of this quantities right. Now, what is this r t is nothing, but summation of r t t going from 1 t. So, this should be nothing, but upper bounded by 2 times summation square root of b t and what I will do is further I will also make it square root b t d capital t because I know that b t sequence is increasing right. I will take the largest value of that and then I am going to make it as minimum then it is going to be some i equals to 1 to capital minimum of 1 norm of d t b t minus 1 which is correct. Now, claim we are now going to argue that summation i equals to 1 to t min of 1 comma d t b t minus 1 inverse it is going to depend on my i matrices like this. This is going to be log of determinant of v t divided by determinant of v 0. So, now, what we are basically connecting is basically how this norm of this depends on the determinant of my matrix I have observed so far. So, let us quickly try to get this how why this is true. So, then we are actually now going to use some linear algebra and matrix algebra here. So, first I am going to use this property that is always upper bounded by 2 log, this is log to the base e here actually is 1 plus 2. Is this inequality correct? And this is this equality holds. So, if you are going to take u between 0, 1 then minimum of this 2 is going to satisfy this. So, now, let us try to apply this inequality to this quantity. So, I am going to take one term here. So, this is sorry this is t here. So, then min of 1 d t d t minus 1 minus 1 minus 1 minus 1 minus 1 minus 1 minus 1 minus 1. What is this quantity is going to look like? So, first before I could apply this I should argue that this norm is going to be between 0, 1. If I can use that, then I can use this quantity and bound it as 2 times log 1 plus norm of d t d t minus 1. This question is suppose if u is already between 0, 1, right. So, then why not this mean of u my, u comma 1 is just u, right. So, why is this? So, let me just check this what I am doing is correct. Yeah, so let us say I think maybe let us let us try to ignore this part. So, let us say u is positive. Let us try to verify that this holds always. So, let us take the simplest case when u equals to 1, this is 1, this is anyway this holds. Now, yeah. So, then this and in that case it is going to be always greater than 1, right. So, that is fine. So, log just check what is the log 2 to the base e after 2 multiplied I mean this quantity, right, 2 times this for u equals to 1. So, that is greater than 1. So, that is fine. So, I think maybe I do not need this actually, okay. Now, then I really do not need to worry about what is this quantity, right. This is some positive quantity I know, okay. This positive quantity with this holds, okay. Now, what I am interested in this going to t 1 to t. So, this is going to be t 1 to t. So, now, if I can just write it as 2 times log of e product 1 plus norm of dt vt minus 1 this and where this is now t equals to 1 to capital T, okay. So, just some of log I use this log of product, okay. So, fine. So, let us take this inequality. Let us keep it like this. Now, I am going to write my vt. I can write my vt to be iteratively like this, right plus dt transpose d dt dt transpose transpose. If I do this, so just see whether this is correct. So, I have taken vt to the power half on the left side and then again I have taken vt to the power half again on the right side. So, if you just multiply this, it is just the numerator, right. It is just this, okay. So, if I have a matrix C which is nothing but product of two matrices A and B, what is determinant of C? It is determinant of A and B, right. So, let us use this determinant of vt is equals to determinant of vt minus 1 half times determinant of this entire quantity and then determinant of vt minus 1 half, okay. So, this you can pull out and now this is nothing but determinant of vt. Now, let us focus on this part. This matrix, if you just look into this, this has first identity matrix i plus. Now, look into this. So, this part it is a vector. This part is also a vector. So, this part is also a vector. So, this is a vector. So, now it is just nothing but product of two vectors. So, okay. So, let us focus on this. This is of the form determinant of i plus y, y transpose, where y is the things I have put it in circle, okay. So, do you know what should be the determinant of such a matrix? Okay. So, what should be the eigenvalues of such a matrix? So, okay. So, you can just go and compute. So, the eigenvalues of such matrix are 1 and so, eigenvalues here are what is the rank of this matrix? 1, 2. So, eigenvalues are going to be 1 and other eigenvalue is simply going to be 1 plus norm of y 2. So, it will have only these two eigenvalues and these are the two values, okay. So, if I know these are the eigenvalues, what is the determinant of this quantity? It is just going to be the product of these two values. That is why I am saying how what is the rank of this? 1 eigenvalues with this and how many ones will be there? D minus 1. D minus 1. Rest are all 1, 1, 1, 1. So, only this is the only, these are only two distinct ones. This one will be repeated, okay. So, you can check this. Now, determinant of V t is nothing but determinant of V t minus 1 times what is this now? This is nothing but 1 plus this quantity here. Norm of V t minus 1 half times V t minus 1 half times D t square. But we know that this is nothing but what? By definition, this is nothing but 1 plus norm of D t, D t minus 1 inverse, right. This quantity here is nothing but this, okay. So, now, yeah, let us let us try to quickly wind this up. I just, so I have this. Now, what we have done? We have iteratively written determinants of V t in terms of determinants of V t minus 1 and into this quantity. So, go back and check that this indeed correct, like these are all eigenvalues of this metric, right. Now, we are just writing. So, if you just compute this, this is going to give you determinants of V t is equals to determinant of V 0 times product of these guys, right. So, this right. Agreed? I have just written iteratively. I have now replaced determinant of V t minus 1 by the same equation. So, it will give me finally, determinant of V t equals to determinant of V 0 plus product of these two terms. Product of these two terms, all this term, okay, should be S here that is going from S 1 to T, okay, fine. So, now, let us go back, okay, okay, maybe we will just stop this. So, now, if you look into this, what I have? The product of this is nothing but determinant of V t divided by determinant of V 0, right. So, if I am going to have a product from t equals to 1 to capital T, this is going to give me what? What was this? This was like min of 1 comma norm of D t times V t minus 1 inverse. So, this is nothing but 2 times log of E determinant of V capital T divided by what is going to be determinant of V 0 lambda multiplied by how many times? D, right, whatever it is, okay, fine. So, I have this quantity now, okay. So, finally, now let us try to find out what is the determinant of V t is going to be. If I can bound this last term, then we are more or less done, okay. What is going to be determinant of V t? Determinant of V t. So, what is going to be the rank of this matrix V t? It is going to a full rank, right. This is a positive definite matrix, okay. And in that case, what is going to be positive? It is eigenvalues, all of them are going to be positive. And if I am going to multiply all of their eigenvalues, what I am going to get? The determinant, right. So, this is nothing but, this is going to be a determinant. So, what are lambda is here are the eigenvalues of this matrix V t. So, how many of you know GP AP inequality, geometry progression, arithmetic progression? Can you apply GP AP and tell me what should be the bound for this? So, what is the AP, arithmetic progression of this? This is nothing but D or, okay. So, now let us do another round of simplification i equals to 1 to D. What is summation of lambda i? What is the sum of eigenvalues? Can you connect it to the trace of a matrix? It is going to be what? Trace, just going to be trace, right. So, this is going to be. So, now what is V t? V t is nothing but V 0 plus other quantities, right. So, I know that V t is nothing but V 0 plus. What is this? Each one of them is a matrix. How they look like? So, if you are going to look at their diagonal values, they are squares, right, like each component squared. The diagonals where each component is squared. Is it true that trace of this is nothing but the trace of each one of them? So, this is nothing but trace of V 0 plus summation of trace of D s, D s transpose for s equals to 1 to t. This whole thing divided by D. What is trace of D, D, S transpose norm of norm of D s, right, squared norm. So, this is going to be upper bound, right. So, sorry, this is nothing but trace of V 0. What is D lambda? D lambda, right, because there are D elements getting added lambda times. Now, this is nothing but norm of D s squared. This is sum, right, 1 to D. This is divided by D whole to the power D. S equals to 1 to D. Yeah, right. This is S equals to 1 to capital T, followed. So, now, now let us appeal to our second assumption. What was our second assumption? Norm of s is less than or equals to L, right. So, let us apply here. What this gave me? This gave me D lambda t times L whole divided by D to the power D. Yeah, first we get cancelled. So, now finally, what we have shown is this quantity determinant of V t is upper bounded by this quantity here, okay. This is square here, okay, because that is norm square. So, fine. I do not know which portion I need to erase. So, let us continue to erase here. So, now what is this? I have finally able to bond as this is 2 times log V t is this quantity, right. If I take a logarithmic of this, this is going to be 2 D times lambda plus T L by D, okay. So, did I miss? Now, what is this? Denominator is going to give me. So, what was the determinant of V 0? Lambda to the power D, right. So, how does that come now? So, that D will come out of this and there is a lambda here, inside the log only, right. It is already inside the log. So, this is going to be 2 D log 1 plus T L by lambda D. So, this is what we have got. So, T L square. Now, the assumption is the square norm is L, okay. We just said, just check whether we said on the square norm or just a norm, right. If it is, let me see what I wanted actually. So, this is L square, okay. So, fine. So, let us finish this now. Can somebody tell me where was the, what was the bound I had? So, if I remember correctly, I had R t is upper bounded by summation 2 beta t summation t equals to capital T min of 1 comma dt, dt minus 1 inverse, right, root b t, okay. Now, what I have shown finally, this quantity here is bounded by this quantity, right. So, this is now 2 D square root of b t times log of 1 plus T L square by lambda t. So, that is, so before I think I have made a small mistake. So, anyway, so let me see if what is that we had said this dt is equals to R t, right. What I wanted to basically use is, so the bound we had on R t, what is the bound we gave on R t? Okay, there is some issue. So, I want to actually do this 2 summation 1 1 1 and then R t square. I wanted to bound R t square. I do not know what is that R t square. This is dt square here. Okay, just a minute, hold on. So, fine. As of now, let us take for time being, we have done with this bound, okay. So, okay, let us conclude this. Let us say till this point we are correct, right. If I am going to bound this minimum of these 2 quantities, this is the bound I am going to get, okay. So, let us take another 2 minutes and finish this, otherwise we will have a discontinuity. So, what is this R t? R t is my R t over t equals to 1 2 capital T is this, right. And what I showed earlier, this R t is what this is we showed this is upper bounded by 2 times square root of B t times, what is that, min of 1 comma dt B t minus 1, this whole quantity, okay. So, now let us find. So, I know that this quantity here, no, no, no, R n s is correct. I think this point was just like I did not put the things correctly. So, this part here, I am going to treat them as like this. If I am going to treat this, is it true that now I am going to get the summation of 1 and summation of R t square, okay. So, then this is nothing but square root of T times summation of R t square, okay. And now whatever we got here from this to this point, so what we had this as just for dt or dt square, I think that is one mistake, we had it as dt square. Can you check this stuff? No, no, no, when we actually proved this is product is equals to this determinant, right? Yes. Was that norm of dt or dt square? Can I just verify that? I think that is supposed to be dt square. It should be square, just check that, that is why it is messing up. It should be dt square. What is that? Yeah, here also, yes, yes. So, say that is why this whole of this works, if this is dt square, I think this is the mistake we have made. This is actually this. Now, is that fine? So, just verify that. Let us quickly wind it up. So, now we have R t is upper bounded by what? So, this is why? Because we have only bound on this dt square, I cannot directly apply whatever the values I have. That is why I am trying to get this quantity in terms of R t square, okay. So, now if I have this in R t square, now what is this? This is now square root t square and then still it is going to be 2 times square root beta. I am now going to, I have made it the largest possible, right? I have made it beta capital T and now this is going to be square root of summation t equals to 1 to t minimum of 1 comma norm of dt square dt minus 1 inverse, okay. So, is this correct now? So, now this is nothing but square root t, t square root t, this is beta t. Now on this I have a bound norm which is this quantity here. This quantity is what? 2 d log 1 plus t lambda square by lambda d. So, what finally I am able to show is my regret bound R t is upper bounded like this. So, if you see this, right now I have not told anything about how this square root beta t looks like. We will talk about it next time. But at least if you assume this beta t is fixed, you will see that this regret bound goes like square root t and also there is one more t here, but that is inside the log term. So, the dominant term here is going to be square root t. This beta t term here, they themselves grow like in d, square root d. This beta term, they also grow in d. So, later when you plug in that value, this becomes square root d, but there is another square root d term here because of that it becomes like d square root t. So, right now let us just only worry about now in terms of, so now we have a bound which goes like square root t. So, now later when we plug in the value of beta t, then we will worry how it depends on the other parameters, ok. You see that already like this kind level that this analysis is different from how we did the analysis for our standard multiamp band data, right. So, now it is using lot of linear algebra and matrix manipulations. It is because our assumption that my reward function is linear. So, because of all this linear algebra is kicked in, ok. So, let us stop here.