A Complete List of Metrics for Evaluation of Regression Models

Performance metrics are vital for supervised machine learning models – including regression models – to evaluate and monitor the performance and accuracy of their predictions. Therefore, such metrics add substantial and necessary value in the model selection and model assessment and can be used to evaluate different models.

But before we focus on various performance metrics for the regression model, let’s first focus on why we want to choose the right evaluation metrics for regression models. And what role do they play in machine learning performance and model monitoring?

First, our goal is to identify how well the model performs on new data. This can be measured using only evaluation metrics. In regression models, we cannot use a single value to evaluate the model. We can only assess the prediction errors. So, here are a few reasons why you need better regression model performance metrics:

A few reasons why you need better regression model performance metrics:

• If your use case is more concerned with large errors, you are likely to choose metrics like Mean Square Error (MSE), which will harshly penalize the large errors.

• If you want to know how the proportion of the variance of the model’s outcome and its predictor variables are accounting for, you can choose a metric like R-Squared (R2) to discover how well the model fits the dependent variables. The linearity of the data will also play a role in whether the R2 metric can be used to measure model fit.

• Some metrics are robust to outliers and do not penalize errors, making it not suitable for the use cases where you concentrate more on outliers.

• As we will have a rather large N-number of metrics for evaluating prediction errors, only a few metrics will be relevant to evaluate the model depending on the business use cases and the nature of the data.

Next, let’s discuss how we generally calculate accuracy for a regression model. Unlike classification models, it is harder to illustrate the accuracy of a regression model. It is also impossible to predict a particular value for accuracy; instead, we can see how far model predictions are from the actual values using the following main metrics:

• R-Squared (R2) and Adjusted R-Squared shows how well the model fits the dependent variables.

• Mean Square Error (MSE) and Root Mean Square Error (RMSE) illustrates the model fitness.

• Mean Absolute Error (MAE) measures the prediction error of a model.

Since there are a vast number of regression metrics that are commonly used, the following attempts to provide a full list of regression metrics used to achieve continuous outcomes and proper classification.

Regression Metrics for Continuous Outcomes:

• R-Squared (R2) refers to the proportion of variation in the outcome explained by the predictor variables.

• Adjusted R-Squared compares the descriptive power of regression models.

• Mean Squared Error (MSE) is a popular error metric for regression problems.

• Root Mean Squared Error (RMSE) is an extension of the mean squared error, measuring the average error performed by the model in its predictions.

• Absolute Error is the difference between measured (or inferred) value and the actual value of a quantity.

• Mean Absolute Error (MAE) measures the prediction error, i.e., the average absolute difference between observed and predicted outcomes.

• Residual Standard Error (RSE) is a variant of the RMSE adjusted for the number of predictors in the model.

• Mean Absolute Deviation (MAD) provides information on the variability of a dataset.

• Maximum Residual Error (MRE)

• Root Relative Squared Error (RRSE) is the root of the squared error of the predictions relative to a naive model predicting the mean.

• Bayesian Information Criteria (BIC) is a criterion for model selection among a finite set of models.

• Mallows’s C_p assesses the fit of a regression model that has been estimated using ordinary least squares.

• Correlation Coefficient measures how strong a relationship between two variables is.

Regression Metrics for Proper Classification:

• Accuracy Score

• Precession

• Recall

• F1-Score

• Confusion Matrix

• ROC Curve

• AUC Curve

Despite having access to these numerous metrics to evaluate prediction errors, data engineers often use only three or four of them because of the following reasons:

• The metric can be easily explained to the reader.

• Based on the business use-cases. Are sensitive to outliers and costlier to predict values with huge variation.

• The metric is computationally simple and easily differentiable.

• The metric is easy to interpret and easy to understand.

Before we wrap up this list, let’s ask one final question: While these metrics are computationally simple, can they be misleading? For example, the metric R-Squared (R2) can be used for explanatory purposes of model accuracy. It explains how well the selected independent variables explain the variance of the model outcome. They both show how well the independent variables fit the curve or the line. But in some cases, there are definite drawbacks with these metrics. For instance, when the number of independent variables increases, the value of this metric will automatically increase, even though some of the independent variables may not be very impactful. This can mislead the reader to think that the model is performing better if they add extra predictors if they are solely looking at this metric for tracking accuracy.

To know more about Qualdo, sign-up here for a free trial.