SVD vs PCA vs NMF - Singular value decomposition vs principal component analysis vs Non-negative matrix factorization

NMF

• The objective of NMF is dimensionality reduction and feature extraction
• NMF Nonnegativity induces sparsity. Nonnegativity leads to part-based decompositions
• NMF splits a face into a number of features that one could interpret as "nose", "eyes" etc, that you can combine to recreate the original image. 
• NMF: Non-negative matrix factorization, which is a group of algorithms in multivariate analysis and linear algebra that can be used to analyze dimensional data.
• Non-negative matrix factorization, or NMF, is a dimension reduction technique often used in unsupervised learning that combines the product of non-negative features into a single one.

PCA

• PCA instead gives you "generic" faces ordered by how well they capture the original one.
• Generate a low-dimensional encoding of a high-dimensional space
• Reduce dimensionality while maintaining maximal variance
• PCA transforms data linearly into new properties that are not correlated with each other.
• PCA is a special case of SVD on the centered covariance matrix

PCA on the other hand is:

• Subtract the mean sample from each row of the data matrix.
• perform SVD on the resulting matrix.

The core idea behind PCA uses results obtained through SVD as its backbone

SVD

• Singular value decomposition and principal component analysis are two eigenvalue methods used to reduce a high-dimensional dataset into fewer dimensions while retaining important information.
• SVD performs low-rank matrix approximation
• SVD procedure finds the optimum k vectors

More Reads

Link1, Link2, Link3, Link4

Keep Exploring!!!