Lecture 2 | Machine Learning (Stanford)

[hylandsjgcn]

15:10



[rahulsanal1]

kindly check the link.. i can't view the video.. it says some error has occured for the past two days..



[wpeng001]

should add trace to x'x to make it equal to \sum X_{ii}

[ByThe1Way]

I know matrices But i was lost after he was talking about proving it. 

[harrycook111]

the calculus part is easy enough but he skipped most of the linear algebra details which is a big part of his math. So... you can go to MIT web site for another FREE education from Prof. Strang and the final equation will become very clear.

[shrit110]

watch after 9:00

[robertxxx74]

(see prev comment 1st) I often find these leaps in the math used by engineers to "prove" things work. I don't think it's just my lack of background either - he shows the operations he's using, then expects you to take it as read that the math really does work, and expects you to be able to follow right away. You don't usually need to understand the proofs to pass the course though, or to use the results. It's just not as satisfying.

[robertxxx74]

That proof at the end (from about 1hr - 1hr 15min) is hard to follow. If you download the lecture notes from the site, it's written down in very few lines. I had to really work to put it all together. It's taken me about an hour to understand it, and even then I'm not fully satisfied that I have proven it to myself. He makes many leaps that are not obvious to me.

[astroboomboy]

This lecture is interesting, but I don't get the math at all. What kind of math is this, and what should I read in order to understand this (my background is linguistics, so I have no math training). Is it enough to learn calculus? And is it possible to really learn math at an older age, or is it like playing violin, you have to start at a very young age in order to become professional?

[tessb]

@astroboomboy on the course website (google it) it says you need linear algebra and probability theory, but it said you need basic linear algebra and probability and a little programming experience.

[turkiym2]

@astroboomboy Mainly calculus and linear algebra, you may pick up the two in 2-3 months if you're intent on learning as they're usually freshman level courses and have no pre-requisites themselves (you may also learn them concurrently as they are independent on the basic level and will intertwine easily as necessary). For Calculus, I recommend the James Stewart textbook, as for linear algebra, I recommend the text by Otto Bretscher. Both are illustrative, thorough and easy to follow.

[a1a2a3skurr1l]

The question at 42:32 made him sad...

[turkiym2]

@a1a2a3skurr1l The asker was clearly sleeping off his calculus classes.

[chvan2335]

A couple of lectures in, it's surprisingly easy to get your head around this shit. Guess it all gets very tricky and intricate soon after, though.

[Jacob011]

In my course of linear systems we used the same normal equation for estimating parameters of a discrete model of continuous system.

The thing is, it can be derived in much simpler way than the one shown in the lecture. (without the use of traces, let alone the traces algebra) :)

So besides that, great lecture and certainly motivating.

[kiriappeee]

hmm.. so alvin looks at the road ahead and records the steering direction. So what if the road ahead is a curve but since I'm on a straight patch for the moment my steering direction is still straight? Seeing where the cam was placed and that there was no bonnet in the pictures it must have been calculated for a few metres ahead. Does that affect anything? In the video it seems like Alvins response is about 0.5 seconds behind a typical human response. Specially in the live tests

[jazzrockr]

This is a very good intro course to AI! It's unfortunately easier to say from the outside that this is "something awesome" than if you had to take that class at Stanford in that classroom, and your work was graded in THAT class.

[axeld93]

This is soo much faster than anything we have here at Auckland University. You can definitely tell this is an Ivy League course.

[muhammadshafei]

Please take it easy on the UM's!

[icommand]

This lecture is a great one..I would like to apply to Stanford next year..It is really a great place of learning..

[jcf139er]

I was completely ecstatic to learn that Stanford offered an entire introductory machine learning course online for free. And by completely ecstatic, I mean that, literally, I was jumping up and down in my chair. I can't wait to work through the exercises over the course of this summer!! Thank you so much Professor Ng, and thank you so much Stanford for affording me and others such an amazing opportunity!

[abramswee]

agree with caesiume. this type of lecture is great. both free and good.

[emok31]

I love how useless the questions are in the grand scheme of things :P

[pmsutube]

thank you very muuucccchhhhh :-)

[gorilah1]

nice informative lecture ... thanks for uploading

[Darvon]

"Umm"

[armanrainy]

Lecture 2 is done Sir (1:13 am).

See u 2morrow on lecture 3.

Thank you Professor. Thank you Stanford.

[ouoh1]

The lecturer should be from Beijing, China based on my brain pattern recognition. :D

[hvutrong]

@ouoh1 nope, he was born in England

[ouoh1]

@hvutrong Oh, my brain pattern recognition failed.

[ozkansafak]

Teaching abilities a bit on the weak side..

[ozkansafak]

Teaching abilities a bit on the weak side..

[jsymons1985]

this lecture makes more sense when you hit the 1911 button

[rakeshprab1]

learning a whole new concept easily in one hour is fantabulous.......thanx...

[AnanyaVilas]

My popcorn and pepsi are ready! Bring it on!

[amanteo]

@AnanyaVilas

My coffee and cigarets too :D

[MrQuincle]

See 3.24 for a nice movie on one of the first "DARPA" race-like challenges.

[Cryto]

Really Helpful! Andrew makes everything looks soo easy :)

Those who have trouble understanding, should really blame their lame colleges.

[vinkhe]

I think Andrew missed the point of the last question, namely that (X^T X)^(-1) X^T indeed is a formula for the pseudoinverse of X (when m > n and X^T X is invertible).

[phamnamlong]

1:04:05 Should the y by the y (i) instead with (i) be the superscript ?

[phamnamlong]

1:04:05 Should the y by the y (i) instead with (i) be the superscript ?

[MikaelUmaN]

Uhm. It's really not necessary to spend that much time on arriving at the normal equations... Basic linear algebra will get you there very quickly. Just reason about the subspaces of the matrix X (e.g. the error vector should be perpendicular to the column space of A => the error is in the left nullspace => done).

Otherwise great lecture.

[qiuxiaoqiu]

@MikaelUmaN

I also think that the conventional way to do OLS regression is simpler compared to doing a gradient descendant method. But perhaps the point here is the gradient method, not regression (so that they can use this method later for more complex models)

[daftrhetoric]

The assumption that someone with an analytic understanding of learning would make a good teacher has yielded some disappointment.

[killingenola]

Can someone plz explain the algebra fact: [ABA^C=CAB+C^AB^] where ^ denotes the transpose operator, thx in advance

[ceokevin]

Why does it need to be so drab and boring? Bad lecturer. Make it interesting at least in the first lecture!!!

[taketaxisky]

For batch and stochastic gradient descent, is alpha (learning rate) usually the same size?

[matharoofmaths]

he is young for a professor

[joshuaburkholder]

@matharoofmaths; Yes ... and that's why he makes so many mistakes in this lectures and has a hard time answering his student's questions (and occassionally evades student questions) in later lectures ... but if his research papers are any indication, he will definitely be an outstanding teacher in the future.

All criticism aside, this is much better than what we had before - nothing. Thank you Dr. Ng and Stanford for letting us in. This is making Machine Learning that much more accessible.

[liuedison]

Wonderful lectures. I was a Stanford undergrad in 1969-1973 and this is so much better than what we had to deal with before. Not so much that the teachers are better (though Andrew Ng is great), but that the pedagogical tools are so advanced.

[Juefawn]

The best thing about video learning is the ability to rewind the course.

[wizztjh]

it's just like study in Stanford! Although it is not physically , but i really let me gain more knowledge of machine learning that only from my university. And he is really a good lecturer!

thank you for you guys that propose it to the Standford University and upload it!

[juludd]

Could someone explain how to get Gradient tr ABAC^T=CAB+C^TAB^T? I can't see how you can get an addition on the right hand side. At least not from within the rules he described in the lecture. Could one use the chain rule for derivation?

[Fusionicon]

I am so interested in machine learning,I can program in java and I have MatLab student version. What previous Math would I need to better understand the tutorial?

Cheer.

[Gaiacarra]

@Fusionicon Basic Calculus. Other than the weird stats stuff he brings into play when formulating the error function ("J"), you don't need anything else, so long as you really pay close attention.

[caesiume]

I'm getting a Stanford education for free B) awesome

[kapildalwani]

@caesiume how?

[abuhajara]

@caesiume the true Stanford Education happens on campus, espacially for PhD research. I mean you have three to five visitors every week to present some of best research work. ever made. I regret that I did not get my PhD from Stanford. If you are a PhD student just move to MIT or Stanford, the rest is junk.

[praneeta133]

These videos are brilliant!!Andrew is super cool at teaching, thanks Stanford!!

[kanobi14]

Awesome lectures.. I realise how good the Stanford and its professors are. Thanks a lot for believing in open education. May the force be with you.

[munmuwang]

이거 도대체...머~꼬!

[harkes12]

Its safe to skip first 9.30 min ...

[Mootsterdotcom]

@harkes12

Actually, I found the intelligent vehicle quite interesting...

[vinhbt123456]

thank you very much

[hnomier]

Thank you stanford ...really great work ...The lectures are great

[Foolean]

He's really good at teaching.

[outfile]

he not just good...he is excellent.

[realmadridvideos]

Thank you Stanford

[arran5498]

Stanford. Thanks for posting these lectures! Big thank you!

[mayaahmed]

Really nice. Well taught. I am really enjoying listening to these lectures. A true service to public.

[stanza2200]

Where are the lecture notes posted?

[ninjakannon]

stanza2200, look in the video description.

[bidexue]

good

[tcyue]

gradient of f(x.vector)= max increase rate. However the negative of it is called gradient decent

[joshuaburkholder]

Around time = 43:00, Dr. Ng again gave the wrong description of the gradient.

Example: Let f(x,y) = x^2 + y^2. Hence, the gradient is ( 2x, 2y ). At the point (1,1), the gradient is (2,2). Since the only local minimum of f(x,y) is at (0,0) and since (1,1)+(2,2)=(3,3), then the gradient at (1,1) points away from the only local minimum of f(x,y); therefore, the gradient does not point toward the direction of steepest descent. The gradient points in the direction of steepest ASCENT.

[siddharthbatra]

what you are saying is right but it depends on your update rule as well. The update rule for descent has a negative sign in itself,

theta_i+1 = theta_i - alpha * gradient

so your above example would be

x_i+1 = x_i - alpha * 2

y_i+1 = y_i - alpha * 2

where x_i = y_i = 1

with an alpha of 0.5 your parameters become

x_i+1 = 0 , y_i+1 = 0

but yeah you are correct. He maybe had the update rule in his mind when he said that.

[joshuaburkholder]

Around time = 28:00, Dr. Ng noted that to go in the direction of steepest descent from a point, ( theta1, theta2, J(theta1, theta2) ), we should go in the direction of the gradient of J at that point; however, this is incorrect. The gradient always points in the direction of steepest ascent, not descent; therefore, the direction of steepest descent from ( theta1, theta2, J(theta1, theta2) ) is opposite of the gradient: -Del( J( theta1, theta2 ) ).

[joshuaburkholder]

Around time = 28:00, Dr. Ng that if we want to go in the direction of steepest descent from a point J( theta1, theta2 ), then we should go in the direction of the gradient of J( theta1, theta2 ); however, this is incorrect.  The gradient always points toward the direction of steepest ascent, not descent; therefore, if we want to go in the direction of steepest descent from a point J( theta1, theta12 ), then we should go in the direction that is opposite of the gradient ... -J( theta1, theta2 ).

[joshuaburkholder]

Some of this comment didn't come through properly. Please disregard this post. I will repost this comment with corrections.

[Scutchris]

Yeah, that's why there's a negative sign.

[tbouloutas]

This course is outstanding. Standford should make more courses like this available online