SVD vs PCA vs NMF

Singular value decomposition and principal component analysis are two eigenvalue methods used to reduce a high-dimensional dataset into fewer dimensions while retaining important information

SVD

• SVD performs low-rank matrix approximation
• SVD procedure finds the optimum k vectors

PCA on the other hand is:

• 1) subtract the mean sample from each row of the data matrix.
• 2) preform SVD on the resulting matrix.
• The core idea behind PCA uses result obtained through SVD as its backbone

NMF: Non-negative matrix factorization. PCA and NMF optimize for a different result. PCA finds a subspace that conserves the data's variance, while NMF finds nonnegative features.

PCA is highly recommended when you have to transform high dimensions into low dimensions and you are okay to loose original features in process as new one are introduced.

The output of NMF can be visualized as a compressed version of the original dataset

Recommendations systems, Topic modeling, Image compression everything uses the same concepts PCA, SVD, NMF...

Keep Thinking!!!