University of Michigan - Ann Arbor

A regression problem is a type of supervised machine learning task where the objective is to predict an unknown, continuous numerical label based on known input features. It is used to answer 'how much?' or 'how many?' questions. The defining characteristic is that the target labels take on arbitrary numerical values, and the goal of the learning algorithm is to produce a model whose predictions closely approximate these actual observed values.

Regression Problem

Classification problems involve predicting a categorical label for a given input. The model's output for a classification task is typically a probability distribution over the set of possible labels, indicating the likelihood of the input belonging to each category.

Classification problem in statistical learning

Predicting the height of a pediatric patient for a given year based on family trends and input data about the patient's current height/weight data.

Example of A Regression Problem

- Linear regression
- Nonlinear regression

Types of regression

https://towardsdatascience.com/regression-an-explanation-of-regression-metrics-and-what-can-go-wrong-a39a9793d914

Regression Metrics

https://beginningwithml.wordpress.com/2018/07/02/5-locally-weighted-linear-regression/
Locally weighted linear regression is a general linear regression method, it's also a local regression method. We can know more about local regression by checking its related wiki: https://en.wikipedia.org/wiki/Local_regression

Locally Weighted Linear Regression

A regression model predicts a value $$Y$$ given a set of $$X$$ variables. In the regression model $$Y \sim X$$, we measure the effect of $$X$$ on $$Y$$, i.e., we measure the association between changes in values of $$X$$ and changes in values of $$Y$$. Analyzing a regression model is a method used to solve a regression problem.

Reference: James, G., Witten, D., Hastie, T., & Tibshirani, R. (2017). An introduction to statistical learning: with applications in R. New York: Springer.

Regression Model

There is a paper on predicting a user's personality from 16 personality types (based on the Myers-Briggs Type Index). The input values were the user's written text they provide on an Internet forum. This is a classification because the output is a qualitative prediction. For example, the user could be classified as an Extrovert.


Example of a classification problem in statistical learning

Classifiers are techniques used to predict qualitative variables.
- Methods that are not very computer intensive (widely-used)
- Methods that are much more computer intensive
- Decision Tree that can be used on regression and classification problems

Methods of Classification

1. Confusion Matrix
2. Accuracy, Recall, Precision
3. F1 Score
4. Receiving Operating Characteristics(ROC),Area under the Curve(AUC)
5. Log Loss

Evaluation Metrics of Classification Models

https://towardsdatascience.com/top-5-metrics-for-evaluating-classification-model-83ede24c7584

Classification Evaluation Metrics

In machine learning, classification refers to a predictive modeling problem where a class label is predicted for a given example of input data. There are three main types of classification tasks that you may encounter; they are:
- Binary Classification,
- Multi-Class Classification,
- Multi-Label Classification.


3 Types of Classification Tasks in Machine Learning

Imbalanced Classification refers to classification tasks where the number of examples in each class is unequally distributed. Typically, imbalanced classification tasks are binary classification tasks where the majority of examples in the training dataset belong to the normal class and a minority of examples belong to the abnormal class.

Balanced Classification is the opposite. It is primarily equally distributed and commonly occurs in multi-label or multi-class classification. 

Imbalanced Classification vs. Balanced Classification

- Confusion Matrix
- False-Positive rate (Type-I error)
- False-Negative rate (Type-II error)
- Accuracy
- Recall
- F1 Score
- Optimizing vs. Satisficing
- Specificity (True Negative rate Specificity)
- Negative Predictive Value
- False Discovery Rate
- True-Positive rate (Recall, Sensitivity)
- Positive-Predictive Value (Precision)
- ROC curve and ROC AUC

Common Performance Metrics for Classification

Classification is essentially a function mapping input data to output categories. If there are missing data in the input, the function can perform poorly in the prediction task. Therefore, having multiple functions that do the predicting job collectively is crucial.

Classification with Missing Data

A data science team is building a predictive model for an e-commerce company. Which of the following tasks represents a classification problem?

Given the following scenario, explain why this task is a classification problem. In your explanation, describe the nature of the model's predicted output.

Learn Before

Related