Non–negative matrix factorisation (NMF) which is a technique for reducing the dimensionality of the data matrix considered, into two lower rank matrices. It is a rather popular algorithm for matrix decomposition and it implicitly imposes non-negative constraint on its data sets. This constrains proves to be enhancing the interpretability by obtaining parts based representation. The paper explores NMF definition and its variations, modifications and extensions pursued over the years. A deep recurrent variation of the sparse NMF for source separation is also discussed extensively. The problems of optimisation in sparse NMF are tackled by sequentially iterating a thresholding algorithm. This exhibits more interpretability and a better convergence rate when compared with basic sparse NMF. When small amount of data is available, the deep variation of the NMF gives better performance compared to sparse NMF. Various furthering of the NMF algorithm avenue is explored and addressed.