如何通过预测概率获得每个样本的所有类别的独立概率？

Question

在我的工作中，有一个功能集完全由布尔数据组成，并且有属于这些功能的类。类是字符串

     feature set              class (String)
[True False True   ...]        "A"
[True True  True   ...]        "B"
[True True  False   ...]       "C"

当我用随机森林算法训练这些数据时

factor = pd.factorize(classes)
classes = factor[0]

classifier = RandomForestClassifier(n_estimators=100, criterion="entropy", random_state=0)
classifier.fit(x_train, classes)

分类器可以正确检测97%的类。当我这样做的时候

classifier.predict_proba(sample1_feature_set)

它给出了样本1中每个类别的相对概率。例如像

 [0.80    0.05    0.15]
   ↓        ↓        ↓
  Prob.    Prob.    Prob.
   of       of       of
  "A"      "B"      "C" 
  for      for      for
sample1   sample1  sample1

因此，当我添加list的值（0.80+0.05+0.15）时，结果总是1。这表明它实际上进行了相对评估，即一类的概率影响另一类的概率

我想得到样本1中所有类的独立概率，比如

 [0.95    0.69    0.87]
   ↓        ↓        ↓
  Prob.    Prob.    Prob.
   of       of       of
  "A"      "B"      "C" 
  for      for      for
sample1   sample1  sample1

样本1是“A”类的%95、“B”类的%69和“C”类的%87。你知道我怎么做吗

Answer 1

随机林是一个ensemble method。基本上，它使用不同的数据子集（称为bagging）构建单独的决策树，并对所有树的预测进行平均，以给出概率。“帮助”页实际上是一个很好的起点：

In averaging methods, the driving principle is to build several estimators independently and then to average their predictions. On average, the combined estimator is usually better than any of the single base estimator because its variance is reduced.

Examples: Bagging methods, Forests of randomized trees, …

因此，概率总和总是为一。下面是如何访问每个树的单个预测的示例：

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
iris = load_iris()

X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=0.33, random_state=42)

from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(n_estimators=10)
model.fit(X_train, y_train)

pred = model.predict_proba(X_test)
pred[:5,:]

array([[0. , 1. , 0. ],
       [1. , 0. , 0. ],
       [0. , 0. , 1. ],
       [0. , 0.9, 0.1],
       [0. , 0.9, 0.1]])

这是对第一棵树的预测：

model.estimators_[0].predict(X_test)
Out[42]: 
array([1., 0., 2., 2., 1., 0., 1., 2., 2., 1., 2., 0., 0., 0., 0., 2., 2.,
       1., 1., 2., 0., 2., 0., 2., 2., 2., 2., 2., 0., 0., 0., 0., 1., 0.,
       0., 2., 1., 0., 0., 0., 2., 2., 1., 0., 0., 1., 1., 2., 1., 2.])

我们记录了所有树木：

result = np.zeros((len(X_test),3))
for i in range(len(model.estimators_)):
    p = model.estimators_[i].predict(X_test).astype(int)
    result[range(len(X_test)),p] += 1

result[:5,:]
Out[63]: 
array([[ 0., 10.,  0.],
       [10.,  0.,  0.],
       [ 0.,  0., 10.],
       [ 0.,  9.,  1.],
       [ 0.,  9.,  1.]])

将其除以树的数量，即可得出您之前获得的概率：

result/10
Out[65]: 
array([[0. , 1. , 0. ],
       [1. , 0. , 0. ],
       [0. , 0. , 1. ],
       [0. , 0.9, 0.1],
       [0. , 0.9, 0.1],