Python milk库：对象权重问题

Question

我正在尝试使用“一对一”的决策树组合来进行多分类。问题是，当我给分类器传递不同的对象权重时，结果却没有变化。

我是不是对权重的理解有误，还是说它们根本就不工作？

谢谢大家的回复！

这是我的代码：

class AdaLearner(object):
    def __init__(self, in_base_type, in_multi_type):
        self.base_type = in_base_type
        self.multi_type = in_multi_type

    def train(self, in_features, in_labels):
        model = AdaBoost(self.base_type, self.multi_type)
        model.learn(in_features, in_labels)

        return model

class AdaBoost(object):
    CLASSIFIERS_NUM = 100
    def __init__(self, in_base_type, in_multi_type):
        self.base_type = in_base_type
        self.multi_type = in_multi_type
        self.classifiers = []
        self.weights = []

    def learn(self, in_features, in_labels):
        labels_number = len(set(in_labels))
        self.weights = self.get_initial_weights(in_labels)

        for iteration in xrange(AdaBoost.CLASSIFIERS_NUM):
            classifier = self.multi_type(self.base_type())
            self.classifiers.append(classifier.train(in_features,
                                                     in_labels,
                                                     weights=self.weights))
            answers = []
            for obj in in_features:
                answers.append(self.classifiers[-1].apply(obj))
            err = self.compute_weighted_error(in_labels, answers)
            print err
            if abs(err - 0.) < 1e-6:
            break

            alpha = 0.5 * log((1 - err)/err)

            self.update_weights(in_labels, answers, alpha)
            self.normalize_weights()

    def apply(self, in_features):
        answers = {}
        for classifier in self.classifiers:
            answer = classifier.apply(in_features)
            if answer in answers:
                answers[answer] += 1
            else:
                answers[answer] = 1
        ranked_answers = sorted(answers.iteritems(),
                                key=lambda (k,v): (v,k),
                                reverse=True)
        return ranked_answers[0][0]

    def compute_weighted_error(self, in_labels, in_answers):
        error = 0.
        w_sum = sum(self.weights)
        for ind in xrange(len(in_labels)):
            error += (in_answers[ind] != in_labels[ind]) * self.weights[ind] / w_sum
        return error

    def update_weights(self, in_labels, in_answers, in_alpha):
        for ind in xrange(len(in_labels)):
            self.weights[ind] *= exp(in_alpha * (in_answers[ind] != in_labels[ind]))

    def normalize_weights(self):
        w_sum = sum(self.weights)
        for ind in xrange(len(self.weights)):
            self.weights[ind] /= w_sum

    def get_initial_weights(self, in_labels):
        weight = 1 / float(len(in_labels))
        result = []
        for i in xrange(len(in_labels)):
            result.append(weight)
        return result

正如你所看到的，这只是一个简单的AdaBoost（我用tree_learner作为基础类型，one_against_one作为多分类类型），无论参与了多少个基础分类器，它的表现都是一样的。它就像一个多分类的决策树。

然后我做了一个小改动。在每次迭代中，我根据对象的权重随机选择了一些样本，并用这些随机选择的对象训练分类器，而没有使用任何权重。这样做就如我所期望的那样有效。

随机选择机器学习决策树分类器样本选择多分类对象权重 AdaBoost

Python milk库：对象权重问题

1 个回答

撰写回答