Despite the availability of a plethora of imaginative tools in Applied Mathematics, the preeminent instrument that persists for mathematicians is the linear model. This model exhibits straightforward and seemingly restrictive attributes, including linearity, constancy of variance, normality, and independence. Linear models, along with their associated methodologies, are distinguished for their remarkable flexibility and potency. Given that nearly all advanced statistical techniques stem from generalizations of linear models, competence in this domain becomes a prerequisite for the comprehensive study of advanced statistical tools. The central focus of this research article centers around the specific formulations of the Simple Linear Regression Model and the Multiple Linear Regression Model. It delves into the Least Squares Estimation (LSE) of their parameters and elucidates the properties inherent to LSE. Moreover, an innovative proof of the Gauss-Markov theorem is introduced, employing the Principles of Matrix Calculus as a foundational basis. Additionally, the article portrays the concept of Best Linear Unbiased Estimators (BLUE), underscoring its significance within the context of this study.