Computing the Singular Value Decomposition | MIT 18.06SC Linear Algebra, Fall 2011



hey we're back today we're going to do a singular value decomposition question the problem is really simple to state find the singular value decomposition of this matrix C equals 5 5 negative 1 7 hit pause try it yourself I'll be back in a minute and we can do it together all right we're back now let's do it together now I know professor Strang has done a couple of these in lecture but um as he pointed out there it's really easy to make a mistake so you can never do enough examples of finding the SVD so what does the SVD look like what do we want to end up with well we want a decomposition C equals u Sigma V transpose U and V are going to be orthogonal matrices that is their columns are orthonormal sets Sigma is going to be a diagonal matrix with non-negative entries ok good so now how do we find this decomposition well we need two equations okay one is C transpose C is equal to V Sigma transpose Sigma V transpose and you get this just by plugging in C transpose C here and noticing that u transpose U is 1 since u is an orthogonal matrix ok and the second equation is just noticing that V transpose this V inverse and moving it to the other side of the equation which is C V equals u Sigma ok so these are the two equations we need to use to find V Sigma and you okay so let's start with the first one let's let's compute C transpose C so C transpose C what is that well if you compute will get a 26 and 18 and 18 and a 74 great now what you notice about this equation is this is just a diagonalization of C transpose C so we need to find the eigenvalues those will be the entries of Sigma transpose Sigma and the eigenvectors which will be the columns of V okay good so how do we find those well we look at the determinant of C transpose C minus lambda times the identity which is the determinant of 26 minus lambda 18 18 and 74 okay 74 minus lambda thank you good okay and what is that polynomial well we get a lambda squared now is that 26 plus 74 is 100 so minus 100 lambda and I'll let you do 26 times 74 minus 18 squared on your own but you'll see you get 1600 okay and this off this easily factors as lambda minus 20 times the lambda minus 80 good so the eigenvalues are 20 and 80 okay now what are the eigenvectors well you take C transpose C minus 20 times the identity and you get um 6 18 18 and 54 to find the eigenvector with eigenvalue 20 we need to find a vector in the null space of this matrix note that the second are the second column is 3 times the first column so our first vector V 1 we can just take that to be well we could take it to be a negative 3 1 but we like it to be a unit vector right because the remember the columns of V should be unit vectors because they're orthonormal so 3 squared plus 1 squared is 10 we have to divide by the square root of 10 okay similarly we do C transpose C minus 80 times the identity will get um what do we get we get negative 54 1818 and negative 6 and similarly we can find that V 2 will be 1 over square root of 10 3 over the square root of 10 great okay so what information do we have now we have our V matrix which is just made up of these two columns and we actually have our Sigma matrix too right because the squares of the of the diagonal entries of Sigma are 20 and 80 good so let's write those down write down what we have so we have V it's just equal to I just add these vectors and make them the columns of my matrix square root of 10 1 over square root of 10 1 over square root of 10 3 over square root of 10 and Sigma this is just other square roots of 20 and 80 which is just a 2 root 5 and 45 along the diagonal ok squeezing it in here I hope you all can see these two good so so these are two of my other three parts of my singular value decomposition I the last thing I need to find is U and for that I need to use this second equation right here so I need to multiply C times V okay so C is 5 5 negative 1 7 let's multiply it by V 1 over root 10 3 over square to 10 what do we get well I'll let you work out the details but it's some it's it's not hard here you get negative 10 over root 10 which is just negative square root of 10 here then I just get a a I just get a to square root of 10 okay and then I get um one is um to square root of 10 and um wait so I uh and I think I made it an error here let me give me a second to uh look through my computation again to one entry should be uh yes thank you ah the to one entries should be the square root of ten good yes that would thats what i was hoping yes because we get um right plus 15 is 20 over squared oh yes sorry I I did it in the wrong order right so um your your recitation instructor should know how to multiply matrices great yes thank you you multiply this first then this and this and then this and if you do it correctly um you will get this this matrix you good great so now um this I'd like this to be my U matrix but it it's actually u times Sigma so I need to make these entries unit length okay so I get a negative 1 over root 2 1 over root 2 um 1 over root 2 1 over root 2 times my Sigma matrix here which is remember to square root of 5 for square root of 5 and those it these constants are just what I needed to divide these columns by in order to make them unit vectors good so now here's here's my you you matrix 1 over square root of 2 negative 1 over square root of 2 1 over square root of 2 1 over square root of 2 good so now I have all three matrices u v and sigma and despite some little errors here and there i think this is actually right you should go check it yourself because if you're at all like me you screwed up several times by now but anyway this is this is a good illustration of how to find the singular value decomposition recall that you're looking for u Sigma V transpose where u and V are orthogonal matrices and Sigma is diagonal with non-negative entries and you find it using these two equations you compute C transpose C that's V Sigma transpose Sigma times V transpose and you you also have C V is use Sigma ok I hope this was a helpful illustration

32 Comments

  1. Ajay Sharma said:

    Nice both video and the way you up the eyebrow in last second

    June 26, 2019
    Reply
  2. Jacob Smith said:

    Great video!

    June 26, 2019
    Reply
  3. star hopper said:

    HOMIE U A REAL G

    June 26, 2019
    Reply
  4. Rivaldi Julviar said:

    Great way of teching!! you've teach me SVD in 11 minutes. a few error on the end but that's understandable

    June 26, 2019
    Reply
  5. JasonT760 said:

    Thank you, most examples I found for this were simple examples, this helped me figure out the more complex problems.

    June 26, 2019
    Reply
  6. PRIYANSHU VERMA said:

    At 5:08, can you explain how did you calculated the value of V1?

    June 26, 2019
    Reply
  7. ved prakash dubey said:

    In final step there is a mistake in first column of u reverses sign of element.

    June 26, 2019
    Reply
  8. Ajay Jadhav said:

    I think he skipped a step where product CV is multiplied by inverse of sigma matrix

    June 26, 2019
    Reply
  9. Andrei Platonov said:

    I would be scared af with those numbers

    June 26, 2019
    Reply
  10. Leandro Souza said:

    God, I love MIT.

    June 26, 2019
    Reply
  11. ZeroDay Fracture said:

    But, what do the numbers mean mason???

    June 26, 2019
    Reply
  12. Al En said:

    Do you habe more explaination videos ?

    June 26, 2019
    Reply
  13. Tttt Y said:

    for rectangular matrix, is sigma still symmetric?

    June 26, 2019
    Reply
  14. Saddam Hussain said:

    Thank you, It was very helpful.

    June 26, 2019
    Reply
  15. Lemaure44 said:

    SVD? its a russian sniper idiots!

    June 26, 2019
    Reply
  16. Roberto Augusto Gómez Loenzo said:

    Excellent video. Quite clear.

    June 26, 2019
    Reply
  17. 冯毅强 said:

    nice video, thanks

    June 26, 2019
    Reply
  18. gloriousholy said:

    Eigenvector signs are wrong, that's what's caused the confusion. The eigenvectors should be this: [[-3/sqrt(10), -1/sqrt(10)],[1/sqrt(10), -3/sqrt(10)]].

    June 26, 2019
    Reply
  19. Andrew Hutchings said:

    Explained a complicated problem in a simple way. Amazing work.

    June 26, 2019
    Reply
  20. Smruti Charkha said:

    It was a really good illustration.

    June 26, 2019
    Reply
  21. Mehmet Kazakli said:

    Singular values need to be ordered decreasingly. When you write sigma, should not 1st and 4th values switched ?

    June 26, 2019
    Reply
  22. Abdulbaki Aybakan said:

    Awesome 🙂

    June 26, 2019
    Reply
  23. Shubhi Sinha said:

    Chutiya hai tu saala budbak

    June 26, 2019
    Reply
  24. Free said:

    7:24 No need to look for me man, I'm right here.

    June 26, 2019
    Reply
  25. Locke Jet said:

    Thank you for your another method to get U.

    June 26, 2019
    Reply
  26. Trang Nguyen said:

    Perhaps there's a faster way. let's note C' the transpose of C, S for sigma matrix
    By using two equations : CC' = V S'S V'
    C'C = US'SU'
    You can compute S'S by finding eigenvalues of CC' but it happens to be the same eigen values of C'C. So after finding V, instead of using your second equation. You can just find the eigen vectors of C'C by doing :
    C'C – 20I
    C'C – 80I
    You'll find the vectors of U faster and without inversing S.
    Also by factorizing 1/sqrt(10), it's easier to compute

    June 26, 2019
    Reply
  27. Abdul Fatah Rajri said:

    Excellent effort of this young man ! But I have a little confusion that if the evectors V1 and V2 do not happen to be orthonormals to each other then what should be there because we always require our V to be orthogonal
    ?

    June 26, 2019
    Reply
  28. cool guy said:

    mini gilbert

    June 26, 2019
    Reply
  29. Marwan Sabri said:

    Thank you very much

    June 26, 2019
    Reply
  30. Aisha Anwar Malik said:

    How are the values of E filled in? 7:00

    June 26, 2019
    Reply
  31. Joujau9 said:

    Can anyone explain me precisely, what is happening @10:00, how does he get this first matrix? What is he diving it with? I'm a bit confused.

    June 26, 2019
    Reply
  32. Eliot McLellan said:

    Thanks, noob!lol

    June 26, 2019
    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *