深度学习-回归问题的评估指标:MAE, MAPE, MSE, RMSE, R2_Score

回归问题常用的评估指标

回归问题常用的评估指标包括:MAE, MAPE, MSE, RMSE, R2_Score等。

这些评价指标基本都在 sklearn 包中都封装好了,可直接调用。

安装 sklearn, 完整的名字是 scikit-learn

1
2
pip install -U scikit-learn
# 现在最新版是 V0.22.2.post1
metric formula method
MAE $\frac{1}{n} \sum_{i=1}^n \vert y_i - \hat{y}_i \vert$ sklearn.metrics.mean_absolute_error
MAPE $\frac{1}{n} \sum_{i=1}^n \frac{\vert y_i - \hat{y}_i \vert }{\vert y_i \vert}$ sklearn.metrics.mean_absolute_percentage_error
MSE $\frac{1}{n} \sum_{i=1}^n (y_i - \hat{y}_i)^2$ sklearn.metrics.mean_squared_error
RMSE $\sqrt{\frac{1}{n} \sum_{i=1}^n (y_i - \hat{y}_i)^2}$ sklearn.metrics.mean_squared_error
R2 Score $1 - \dfrac{\sum_{i=1}^n (y_i - \hat{y}_i)^2} {\sum_{i=1}^n (y_i - \bar{y})^2}$ sklearn.metrics.r2_score

注:

  • MAPE 在V0.22.2中还不能直接调用,预计会在V0.23中发布;
  • RMSE 可以调用 mean_squared_error 方法实现, 设置 squared=False 即可;
    • rmse_score = mean_squared_error(y_test, y_pred, squared=False)

示例

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
from sklearn.metrics import mean_absolute_error
from sklearn.metrics import mean_absolute_percentage_error
from sklearn.metrics import mean_squared_error
from sklearn.metrics import r2_score

x_true = xxx
y_true = yyy
y_pred = model.predict(x_true)

# 评估标准: mae, mape, mse, rmse, r2_score
mae_score = mean_absolute_error(y_true, y_pred)
mape_score = mean_absolute_percentage_error(y_true, y_pred)

mse_score = mean_squared_error(y_true, y_pred)
rmse_score = mean_squared_error(y_true, y_pred, squared=False)

r_2_score = r2_score(y_true, y_pred)

PS: 修改 sklearn(0.22.2) 源码, 直接调用 mean_absolute_percentage_error

需要修改两个文件: sklearn/metrics/__init__.py, sklearn/metrics/_regression.py

  • sklearn/metrics/__init__.py, 修改两处。

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    ## 第 63 行左右
    from ._regression import explained_variance_score
    from ._regression import max_error
    from ._regression import mean_absolute_error
    from ._regression import mean_absolute_percentage_error # +++ add MAPE +++
    from ._regression import mean_squared_error
    from ._regression import mean_squared_log_error
    from ._regression import median_absolute_error

    # 第 126 行左右
    'matthews_corrcoef',
    'max_error',
    'mean_absolute_error',
    'mean_absolute_percentage_error', # +++ add MAPE +++
    'mean_squared_error',
    'mean_squared_log_error',
    'mean_poisson_deviance',
  • sklearn/metrics/_regression.py, 修改两处

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
# 第 36 行左右
"max_error",
"mean_absolute_error",
"mean_absolute_percentage_error", # +++ add MAPE +++
"mean_squared_error",
"mean_squared_log_error",
"median_absolute_error",

# 第 190 行左右
return np.average(output_errors, weights=multioutput)

# +++ add MAPE +++
def mean_absolute_percentage_error(y_true, y_pred,
sample_weight=None,
multioutput='uniform_average'):
"""Mean absolute percentage error regression loss
Note here that we do not represent the output as a percentage in range
[0, 100]. Instead, we represent it in range [0, 1/eps]. Read more in the
:ref:`User Guide <mean_absolute_percentage_error>`.
Parameters
----------
y_true : array-like of shape (n_samples,) or (n_samples, n_outputs)
Ground truth (correct) target values.
y_pred : array-like of shape (n_samples,) or (n_samples, n_outputs)
Estimated target values.
sample_weight : array-like of shape (n_samples,), default=None
Sample weights.
multioutput : {'raw_values', 'uniform_average'} or array-like
Defines aggregating of multiple output values.
Array-like value defines weights used to average errors.
If input is list then the shape must be (n_outputs,).
'raw_values' :
Returns a full set of errors in case of multioutput input.
'uniform_average' :
Errors of all outputs are averaged with uniform weight.
Returns
-------
loss : float or ndarray of floats in the range [0, 1/eps]
If multioutput is 'raw_values', then mean absolute percentage error
is returned for each output separately.
If multioutput is 'uniform_average' or an ndarray of weights, then the
weighted average of all output errors is returned.
MAPE output is non-negative floating point. The best value is 0.0.
But note the fact that bad predictions can lead to arbitarily large
MAPE values, especially if some y_true values are very close to zero.
Examples
--------
>>> from sklearn.metrics import mean_absolute_percentage_error
>>> y_true = [3, -0.5, 2, 7]
>>> y_pred = [2.5, 0.0, 2, 8]
>>> mean_absolute_percentage_error(y_true, y_pred)
0.3273...
>>> y_true = [[0.5, 1], [-1, 1], [7, -6]]
>>> y_pred = [[0, 2], [-1, 2], [8, -5]]
>>> mean_absolute_percentage_error(y_true, y_pred)
0.5515...
>>> mean_absolute_percentage_error(y_true, y_pred, multioutput=[0.3, 0.7])
0.6198...
"""
y_type, y_true, y_pred, multioutput = _check_reg_targets(
y_true, y_pred, multioutput)
check_consistent_length(y_true, y_pred, sample_weight)
epsilon = np.finfo(np.float64).eps
mape = np.abs(y_pred - y_true) / np.maximum(np.abs(y_true), epsilon)
output_errors = np.average(mape,
weights=sample_weight, axis=0)
if isinstance(multioutput, str):
if multioutput == 'raw_values':
return output_errors
elif multioutput == 'uniform_average':
# pass None as weights to np.average: uniform mean
multioutput = None

return np.average(output_errors, weights=multioutput)
# +++ add MAPE +++


def mean_squared_error(y_true, y_pred,
sample_weight=None,

参考 github:#15007