Keras回调函数如ReduceLROnPlateau, ModelCheckpoint, EarlyStopping抛出NoneType属性错误

Question

我正在使用Keras（后端是TensorFlow）来训练一个模型，使用train_on_batch()、evaluate()，并在每个训练周期结束时使用回调函数。这段代码是来自以下链接的原始代码：https://www.kaggle.com/code/kmader/pretrained-inceptionv3-for-bone-age

我不打算使用给定的fit_generator()，因为它出现了错误，所以我想实现它的简单版本。具体错误可以在这个链接查看：https://github.com/keras-team/keras/issues/8595

下面是回调函数列表的代码：

from keras.callbacks import ModelCheckpoint, LearningRateScheduler, EarlyStopping, ReduceLROnPlateau
weight_path="/kaggle/working/{}_weights.best.hdf5".format('bone_age')

checkpoint = ModelCheckpoint(weight_path, monitor='val_loss', verbose=1, 
                             save_best_only=True, mode='min', save_weights_only = True)


reduceLROnPlat = ReduceLROnPlateau(monitor='val_loss', factor=0.8, patience=10, verbose=1, mode='auto', epsilon=0.0001, cooldown=5, min_lr=0.0001)
early = EarlyStopping(monitor="val_loss", 
                      mode="min", 
                      patience=5) # probably needs to be more patient, but kaggle time is limited
callbacks_list = [reduceLROnPlat, checkpoint, early]

还有主要的训练代码：

epochs = 2
steps_per_epoch = 10

for epoch in range(epochs):
    print("Epoch {}/{}".format(epoch + 1, epochs))

    # Training phase
    for step in range(steps_per_epoch):
        # Get the next batch of training data and labels
        batch_X, batch_Y = next(train_gen)

        # Train on batch
        loss = bone_age_model.train_on_batch(batch_X, batch_Y)

        print("Step {" + str(step + 1) + "}/{" + str(steps_per_epoch) + "}

    # Validation phase
    val_loss, accuracy = bone_age_model.evaluate(test_X, test_Y)
    print("Validation Loss: " + str(round(val_loss, 4)))

#     Apply callbacks
    for callback in callbacks_list:
        callback.on_epoch_end(epoch, logs={'val_loss': val_loss})

（注意：我是在Kaggle上运行代码，而不是在Google Colab或我的本地电脑上）

我注意到出现了以下错误：

AttributeError                            Traceback (most recent call last)
<ipython-input-56-28ff6d63cd15> in <module>()
     27 #     Apply callbacks
     28     for callback in callbacks_list:
---> 29         callback.on_epoch_end(epoch, logs={'val_loss': val_loss})  # Call the callback at the end of each epoch
     30 
     31 

/opt/conda/lib/python3.6/site-packages/Keras-2.1.3-py3.6.egg/keras/callbacks.py in on_epoch_end(self, epoch, logs)
    914     def on_epoch_end(self, epoch, logs=None):
    915         logs = logs or {}
--> 916         logs['lr'] = K.get_value(self.model.optimizer.lr)
    917         current = logs.get(self.monitor)
    918         if current is None:

AttributeError: 'NoneType' object has no attribute 'optimizer'

我尝试逐个使用回调函数，也就是说，在callbacks_list中只放一个回调函数，但所有的回调函数都会导致错误。

1. callbacks_list = [reduceLROnPlat] 会出现上面提到的错误。
2. callbacks_list = [early] 会出现

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-19-28ff6d63cd15> in <module>()
     27 #     Apply callbacks
     28     for callback in callbacks_list:
---> 29         callback.on_epoch_end(epoch, logs={'val_loss': val_loss})  # Call the callback at the end of each epoch
     30 
     31 

/opt/conda/lib/python3.6/site-packages/Keras-2.1.3-py3.6.egg/keras/callbacks.py in on_epoch_end(self, epoch, logs)
    498             )
    499             return
--> 500         if self.monitor_op(current - self.min_delta, self.best):
    501             self.best = current
    502             self.wait = 0

AttributeError: 'EarlyStopping' object has no attribute 'best'

3. callbacks_list = [checkpoint] 会出现

AttributeError                            Traceback (most recent call last)
<ipython-input-38-28ff6d63cd15> in <module>()
     27 #     Apply callbacks
     28     for callback in callbacks_list:
---> 29         callback.on_epoch_end(epoch, logs={'val_loss': val_loss})  # Call the callback at the end of each epoch
     30 
     31 

/opt/conda/lib/python3.6/site-packages/Keras-2.1.3-py3.6.egg/keras/callbacks.py in on_epoch_end(self, epoch, logs)
    414                         self.best = current
    415                         if self.save_weights_only:
--> 416                             self.model.save_weights(filepath, overwrite=True)
    417                         else:
    418                             self.model.save(filepath, overwrite=True)

AttributeError: 'NoneType' object has no attribute 'save_weights'

我对这些内容有点陌生，理论上学过一些。有人能告诉我我是不是用错了这些函数，还是Keras模块本身有问题（因为我看到很多帖子说旧版本有模块错误）？由于所有模块都出现了某种错误，我觉得我的实现可能有问题。

附注：我不知道这是否相关，但我正在使用预训练的InceptionV3和Resnet模型。

Answer 1

确实，回调函数无法保存模型的权重，因为它没有访问模型权重的权限 : )

当你使用keras的fit函数时，它会创建一个回调列表，并将模型传递给这个列表，下面是相关的代码：

if not isinstance(callbacks, callbacks_module.CallbackList):
    callbacks = callbacks_module.CallbackList(
        callbacks,
        add_history=True,
        add_progbar=verbose != 0,
        verbose=verbose,
        epochs=epochs,
        steps=epoch_iterator.num_batches,
        model=self,
    )

这个列表会为每个传入的回调设置模型，相关的代码如下：

def set_model(self, model):
    super().set_model(model)
    if self._history:
        model.history = self._history
    for callback in self.callbacks:
        callback.set_model(model)

所以，如果你想手动使用这些回调，你需要在调用它们之前先设置模型。在你的例子中，你可以在训练循环之前添加以下几行代码来解决这个问题：

for callback in callback_list:
    callback.set_model(bone_age_model)

这样一来，你的检查点就能访问到模型了，比如你的模型检查点就能访问到权重，从而保存它们。

备注：你可以使用CallbackList类来统一管理你的回调，这样就不需要每次都写循环。具体用法可以参考keras文档。