SVD vs PCA vs NMF - Singular value decomposition vs principal component analysis vs Non-negative matrix factorization

NMF

• The objective of NMF is dimensionality reduction and feature extraction
• NMF Nonnegativity induces sparsity. Nonnegativity leads to part-based decompositions
• NMF splits a face into a number of features that one could interpret as "nose", "eyes" etc, that you can combine to recreate the original image. 
• NMF: Non-negative matrix factorization, which is a group of algorithms in multivariate analysis and linear algebra that can be used to analyze dimensional data.
• Non-negative matrix factorization, or NMF, is a dimension reduction technique often used in unsupervised learning that combines the product of non-negative features into a single one.

PCA

• PCA instead gives you "generic" faces ordered by how well they capture the original one.
• Generate a low-dimensional encoding of a high-dimensional space
• Reduce dimensionality while maintaining maximal variance
• PCA transforms data linearly into new properties that are not correlated with each other.
• PCA is a special case of SVD on the centered covariance matrix

PCA on the other hand is:

• Subtract the mean sample from each row of the data matrix.
• perform SVD on the resulting matrix.

The core idea behind PCA uses results obtained through SVD as its backbone

SVD

• Singular value decomposition and principal component analysis are two eigenvalue methods used to reduce a high-dimensional dataset into fewer dimensions while retaining important information.
• SVD performs low-rank matrix approximation
• SVD procedure finds the optimum k vectors

More Reads

Link1, Link2, Link3, Link4

Keep Exploring!!!

SVD vs PCA vs NMF

Singular value decomposition and principal component analysis are two eigenvalue methods used to reduce a high-dimensional dataset into fewer dimensions while retaining important information

SVD

• SVD performs low-rank matrix approximation
• SVD procedure finds the optimum k vectors

PCA on the other hand is:

• 1) subtract the mean sample from each row of the data matrix.
• 2) preform SVD on the resulting matrix.
• The core idea behind PCA uses result obtained through SVD as its backbone

NMF: Non-negative matrix factorization. PCA and NMF optimize for a different result. PCA finds a subspace that conserves the data's variance, while NMF finds nonnegative features.

PCA is highly recommended when you have to transform high dimensions into low dimensions and you are okay to loose original features in process as new one are introduced.

The output of NMF can be visualized as a compressed version of the original dataset

Recommendations systems, Topic modeling, Image compression everything uses the same concepts PCA, SVD, NMF...

Keep Thinking!!!