Lecture 15 | Machine Learning (Stanford)

Lecture by Professor Andrew Ng for Machine Learning (CS 229) in the Stanford Computer Science department. Professor Ng lectures on principal component analysis (PCA) and independent component analysis (ICA) in relation to unsupervised machine learning.

This course provides a broad introduction to machine learning and statistical pattern recognition. Topics include supervised learning, unsupervised learning, learning theory, reinforcement learning and adaptive control. Recent applications of machine learning, such as to robotic control, data mining, autonomous navigation, bioinformatics, speech recognition, and text and web data processing are also discussed.

Complete Playlist for the Course:
http://www.youtube.com/view_play_list?p=A89DCFA6ADACE599

CS 229 Course Website:
http://www.stanford.edu/class/cs229/

Stanford University:
http://www.stanford.edu/

Stanford University Channel on YouTube:
http://www.youtube.com/stanford

Category:

Education

Tags:

• science
• math
• engineering
• computer
• technology
• robotics
• learning
• algorithm
• principal
• independent
• component
• analysis
• ICA
• PCA
• latent
• semantic
• indexing
• LSI
• singular
• value
• decomposition
• SVD

[ymwdalex]

@MaoMaoTM I mail the lecture mail address 1 month ago, but reply yet :(

[MaoMaoTM]

@ymwdalex I have the same doubt..does anyone know why the solution is svd(X*X') instead of svd(X)?

[peerajak]

Sigma = (1/m)XX'

so it should be U, not V

isn't it?

I think the Lecture at 0:27:32 is not correct. I think Sigma = (1/m)XX'

but in lecture Sigma = X'X.

Anyone agree or disagree with me? please justify your opinion.

[ymwdalex]

Dose anybody think the solution is wrong? For the problem 4.3: give the PCA for natural image.

The solution is [U,S,V] = svd(X*X’); Here, X*X' is the covariance matrix. But U is not the eigenvector of the covariance matrix X.

If [U,S,V] = svd(A); U is the eigenvector of the covariance matrix of A.

[dck1760]

@cooldood83 correct

[ceyceycey21]

I thought, "OMG, too many people know SVD, he is gonna skip it..". Then he didn't :D Thanks.

[cooldood83]

As some student pointed out the error at 29:39, the reason why it should be first k columns of V is the following: if X = UDV' then X'X = VDDV'=VD^2V' .... hence u shud pick the first k columns of the matrix V rather than the matrix U

[xvgonzalo]

@thanhcbn should be orthonormal square matrix = unitary matrix

the svd mentioned here is the "reduced svd"

[thanhcbn]

the svd mentioned here is wrong. both U and V should be orthogonal square matrix

[kapildalwani]

PCA: LSI + SVD and ICA