## Gauss Markov Theorem: OLS is BLUE!

The Gauss-Markov theorem states that if your linear regression model satisfies the classical assumptions, then ordinary least squares (OLS) regression produces best linear unbiased estimates (BLUE) that have the smallest variance of all possible linear estimators. There are five Gauss Markov assumptions: Linearity: the parameters we are estimating using the OLS method must be themselves linear. Random:…

## Scaling- Normalization vs Standardization

Feature scaling is an important technique in Machine Learning and it is one of the most important steps during the preprocessing of data before creating a machine learning model. The reason to perform features scaling is to ensure one feature doesn’t dominate others. The two most important scaling techniques are…

## Q-Q plot – Importance in Linear Regression.

Quantile-Quantile (Q-Q) plot, is a graphical tool for determining if two data sets come from populations with a common distribution such as a Normal, Exponential, or Uniform distribution. This helps in a scenario of linear regression when we have the training and test data set received separately and then we…

## Hypothesis Testing in Linear Regression

Hypothesis testing can be carried out in linear regression for the following purposes: To check whether a predictor is significant for the prediction of the target variable. Two common methods for this are — By the use of p-values:If the p-value of a variable is greater than a certain limit…

Gradient Descent is an optimization algorithm used to find the values of the parameters of any function that minimizes the cost function. The average difference of the squares of all the predicted values of y and the actual values of y is called a Cost Function. It is also called…

## Heteroscedasticity

A random variable is said to be heteroscedastic when different subpopulations have different variabilities (standard deviation). One of the basic assumptions of linear regression is that the data should be homoscedastic, i.e., heteroscedasticity is not present in the data. Due to the violation of assumptions, the Ordinary Least Squares (OLS) estimators…

## Assumptions of Linear Regression Model

There are the five major assumptions: 1. Linear relationship: There should be a linear and additive relationship between the dependent (Y) variable and the independent (X –> x1,x2,x3,…) variable(s). A linear relationship suggests that a change in response Y due to one unit change in x1 is constant, regardless of…

## Linear Regression

Linear regression is a supervised learning algorithm in machine learning (ML). I would like to say it is the starting point of anyone’s ML journey! Regression is a method of modelling a target value based on independent predictors. This method is mostly used for forecasting and finding out the cause…