Linear Regression

Linear regression is a linear approach to model the relationship between a scalar variable (also called criterion) and more predictor variables (known as dependent and independent variables). It is applied to solve mathematical problems and used in machine learning algorithms based on supervised learning techniques to perform regression operations. Linear regression is broadly classified into two categories: simple linear regression and multiple linear regression. There is multivariate linear regression, which is much different than multiple linear regression. In multivariate linear regression, multiple correlated dependent variables are predicted and not the single scalar variable.

Simple Linear Regression: The explanatory variable case is called simple linear regression. It is a regression model to estimate the relationship between an independent and one or more dependent variables using a straight line. Both independent and dependent variables should be quantitative. The relationship between temperature and mercury expansion in thermometers can be a modeled example using a straight line. The mercury expands as the temperature increases. This is the reason why mercury is used to measure temperatures in thermometers. 

Multiple Linear Regression: Multiple linear regression, abbreviated as MLR, is also known as multiple regression. Multiple regression is the extension of linear regression called OLS. It is a statistical method that uses various explanatory variables to predict the results of a response variable. It makes use of only one explanatory variable.

The relationship models are designed based on linear predictor functions where unknown model parameters are identified from the data. These kinds of models are called linear models. The conditional mean of the response is most commonly the values given of the explanatory variables, also called predictor values. It is assumed to be an affine function of the shared values. The conditional median or other quantile are used rarely. Linear regression centers on the conditional probability distribution, like all other forms of regression analysis, using the response value of the predictors given. It does not use joint probability distribution of all the predictor variables since it is a multivariate analysis.

Linear regression was the first rigorously studied type of regression analysis. It was the first analysis model used extensively in practical applications since the models depend linearly on their unknown parameters. These models easily fit better than those models which are non-linearly related to their parameters. These parameters are also chosen because the estimators’ outcome’s statistical properties are easier to find.

Linear regression is applied to solve many real-time problems. These applications mostly fall under one of the two broad following categories:

• Linear regression is used to work with predictive models if the goal is: prediction, forecasting, or error reduction. This is used to fit an observed data set of values of the predictor values and explanatory variables. The fitted model can predict the response variables if the additional values of the explanatory variables are gathered without the responsive values in developing these models. 
• The other category sets goals to explain variations in the predictor variables. These are attributed to variations in the explanatory variables. In this case, linear regression analysis is applied to quantify the relationship strength between the predictor and the explanatory variables. This approach is specifically used to find if some explanatory variables might not have a linear relationship with the response to all or even to identify the subsets of explanatory variables which might have redundant data about the reaction.

Linear regression models are fit using the least square approach. They can be equipped using other techniques, such as minimizing lack of fit like with least absolute deviation regression, or by reducing a penalized version of the least-squares cost functions like in ridge regression and lasso. On the contrary, the least-squares methods are used to fit the models that are not linear. This is why least squares and linear models are associated closely, but they are not synonymous. 

Generalized Linear Models

Generalized linear models, abbreviated as GLMs, are environments for modeling response variables that are either bounded or discrete. This is used for scenarios such as:

• When designing positive quantities models, such as prices or populations, that vary over a large scale in which they are described better with the help of skewed distributions like the log-normal distribution or Poisson distribution even though generalized linear models are not applied for log-normal data, rather than applying on the response variable that is simply transformed using the logarithm function. 

• When designing a model for categorical data, like the choices between the list of candidates given in a particular election which is described better with the help of Bernoulli distribution or the binomial distribution for binary options. This choice is given along with a categorical distribution or the multinomial distribution for multiway choices. This is applied where there are a certain number of choices that are not systematically ordered. 

• When designing the model for ordinal data, such as rating on a scale of 10, the different outcomes are systematically ordered, but the quantity may not have absolute meaning like rating 8 cannot be considered twice as good in any objective situation rating 4. It simply indicated that the rating is better than its previous ratings of 5, 6, and 7, but not as good as 9 and 10. 

Generalized linear models (GLMs) provide an arbitrary link function, g, which relates the predictors to the mean of one or more response variables. These arbitrary link functions are commonly associated with the distribution of the response, and in specific, it directly affects transformation. The typical examples of generalized linear regression models include:

• Poisson regression for count data

• Logistic regression and probit regression for binary data

• Multinomial logistic regression and multinomial probit regression is applied on categorical data. 

• Ordered logit and ordered probit regression are applied to ordinal data. 

The degree of nonlinear regression in the relationship between variables x and y is structured by single-index models. This ensures the role of the linear predictor centered as in the classical linear regression model. Simply applying OLS to the data obtained from a single index model can estimate the value up to a proportionality constant consistently in certain conditions. 

Great Learning gives you an opportunity to learn a free Linear Regression certification course to have a thorough understanding of regression types and have a good hold on problem-solving ability. Linear regression is applied to solve a wide range of mathematical and computer science in machine learning. This way linear regression course will benefit you to improve your mathematical skill set but also helps you to work with coding skills where linear approaches are required. Enroll in Great Learning Academy today to avail the course for free. We wish you a Happy Learning!