验证精度/损失随每个连续历元线性上升和下降

2024-03-29 04:52:49 发布

您现在位置:Python中文网/ 问答频道 /正文

我正在用tensorflow后端培训keras中的CNN,该后端使用以下模型体系结构来解决二进制分类问题。我已经将大约41k个图像按70:25:5的比例分为训练集、验证集和测试集,其中训练集29k个图像,验证集10k个图像,测试集2k个图像。你知道吗

没有类不平衡,在posneg类中各有大约20k个样本。你知道吗

model = Sequential()
model.add(Conv2D(32, (7, 7), padding = 'same', input_shape=input_shape))
model.add(Conv2D(32, (7, 7), padding = 'same'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.5))

model.add(Conv2D(32, (7, 7), padding = 'same'))
model.add(Conv2D(32, (7, 7), padding = 'same'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.6))

model.add(Conv2D(32, (7, 7), padding = 'same'))
model.add(Conv2D(32, (7, 7), padding = 'same'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.6))

model.add(Conv2D(64, (7, 7), padding = 'same'))
model.add(Conv2D(64, (7, 7), padding = 'same'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.6))

model.add(Conv2D(64, (7, 7), padding = 'same'))
model.add(Conv2D(64, (7, 7), padding = 'same'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.6))

model.add(Conv2D(64, (7, 7), padding = 'same'))
model.add(Conv2D(64, (7, 7), padding = 'same'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.6))

model.add(Conv2D(128, (7, 7), padding = 'same'))
model.add(Conv2D(128, (7, 7), padding = 'same'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.6))

model.add(Flatten())
model.add(Dense(512))
model.add(Activation('relu'))

model.add(Dense(512))
model.add(Activation('relu'))

model.add(Dense(1))
model.add(Activation('sigmoid'))

model.compile(loss='binary_crossentropy',
              optimizer=optimizers.Adam(lr=3e-5),
              metrics=['accuracy'])

checkpoint = ModelCheckpoint(filepath='checkpointORCA_adam-{epoch:02d}-{val_loss:.2f}.h5', monitor='val_loss', verbose=0, save_best_only=True)

reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.5,
                              patience=20, min_lr=1e-8)

train_datagen = ImageDataGenerator(rescale=1. / 255,
        shear_range=0.2,
        zoom_range=0.2)

# this is the augmentation configuration we will use for testing:
# only rescaling
test_datagen = ImageDataGenerator(rescale=1. / 255)

# Change the batchsize according to your system RAM
train_batchsize = 32 # changed them to 64 and 128 respectively, but same 
                      results
val_batchsize = 32

train_generator = train_datagen.flow_from_directory(
    train_data_path,
    target_size=(img_width, img_height),
    batch_size=train_batchsize,
    class_mode='binary',
shuffle=True)

# train_generator.reset()
# validation_generator.reset()
validation_generator = test_datagen.flow_from_directory(
    validation_data_path,
    target_size=(img_width, img_height),
    batch_size=val_batchsize,
    class_mode='binary',
shuffle=False)

# validation_generator.reset()

history = model.fit_generator(
    train_generator,
    steps_per_epoch=nb_train_samples // batch_size,
    epochs=epochs,
    validation_data=validation_generator,
    validation_steps=nb_validation_samples // batch_size,
    callbacks=[checkpoint, reduce_lr])

这些都是训练进度的时期,验证精度以线性方式波动。它先高后低的幅度几乎相同。为什么会这样?你知道吗

我已经检查了几乎所有其他的答案,我的数据是标准化的,在训练集中适当地洗牌,lr很小,并且在类似问题领域的其他研究人员发现的成功中得到了很好的检查。你知道吗

Found 29124 images belonging to 2 classes.
Found 10401 images belonging to 2 classes.
Epoch 1/60
910/910 [==============================] - 530s 582ms/step - loss: 0.6105 - acc: 0.6161 - val_loss: 0.2298 - val_acc: 0.9548
Epoch 2/60
910/910 [==============================] - 520s 571ms/step - loss: 0.3590 - acc: 0.8480 - val_loss: 0.8340 - val_acc: 0.6604
Epoch 3/60
910/910 [==============================] - 520s 571ms/step - loss: 0.3160 - acc: 0.8695 - val_loss: 0.0983 - val_acc: 0.9558
Epoch 4/60
910/910 [==============================] - 528s 580ms/step - loss: 0.2925 - acc: 0.8830 - val_loss: 0.5063 - val_acc: 0.8385
Epoch 5/60
910/910 [==============================] - 529s 581ms/step - loss: 0.2718 - acc: 0.8895 - val_loss: 0.0541 - val_acc: 0.9745
Epoch 6/60
910/910 [==============================] - 530s 583ms/step - loss: 0.2523 - acc: 0.8982 - val_loss: 0.5849 - val_acc: 0.8060
Epoch 7/60
910/910 [==============================] - 528s 580ms/step - loss: 0.2368 - acc: 0.9076 - val_loss: 0.0682 - val_acc: 0.9695
Epoch 8/60
910/910 [==============================] - 529s 582ms/step - loss: 0.2168 - acc: 0.9160 - val_loss: 0.6503 - val_acc: 0.7660
Epoch 9/60
910/910 [==============================] - 527s 579ms/step - loss: 0.1996 - acc: 0.9213 - val_loss: 0.0339 - val_acc: 0.9850
Epoch 10/60
910/910 [==============================] - 529s 581ms/step - loss: 0.1896 - acc: 0.9258 - val_loss: 0.5710 - val_acc: 0.8033
Epoch 11/60
910/910 [==============================] - 529s 581ms/step - loss: 0.1814 - acc: 0.9285 - val_loss: 0.0391 - val_acc: 0.9834
Epoch 12/60
910/910 [==============================] - 529s 581ms/step - loss: 0.1715 - acc: 0.9342 - val_loss: 0.6787 - val_acc: 0.7792
Epoch 13/60
910/910 [==============================] - 527s 579ms/step - loss: 0.1678 - acc: 0.9361 - val_loss: 0.0451 - val_acc: 0.9796
Epoch 14/60
910/910 [==============================] - 529s 581ms/step - loss: 0.1683 - acc: 0.9356 - val_loss: 0.7874 - val_acc: 0.7306
Epoch 15/60
910/910 [==============================] - 528s 580ms/step - loss: 0.1618 - acc: 0.9387 - val_loss: 0.0483 - val_acc: 0.9761
Epoch 16/60
910/910 [==============================] - 528s 581ms/step - loss: 0.1569 - acc: 0.9398 - val_loss: 0.9105 - val_acc: 0.7060
Epoch 17/60
910/910 [==============================] - 527s 579ms/step - loss: 0.1566 - acc: 0.9397 - val_loss: 0.0380 - val_acc: 0.9853
Epoch 18/60
910/910 [==============================] - 529s 581ms/step - loss: 0.1506 - acc: 0.9416 - val_loss: 0.7649 - val_acc: 0.7435
Epoch 19/60
910/910 [==============================] - 527s 580ms/step - loss: 0.1497 - acc: 0.9429 - val_loss: 0.0507 - val_acc: 0.9778
Epoch 20/60
910/910 [==============================] - 529s 581ms/step - loss: 0.1476 - acc: 0.9439 - val_loss: 0.7189 - val_acc: 0.7665
Epoch 21/60
910/910 [==============================] - 527s 579ms/step - loss: 0.1426 - acc: 0.9447 - val_loss: 0.0377 - val_acc: 0.9873
Epoch 22/60
910/910 [==============================] - 528s 580ms/step - loss: 0.1407 - acc: 0.9463 - val_loss: 0.7066 - val_acc: 0.7817
Epoch 23/60
910/910 [==============================] - 526s 578ms/step - loss: 0.1427 - acc: 0.9444 - val_loss: 0.0376 - val_acc: 0.9877
Epoch 24/60
910/910 [==============================] - 528s 580ms/step - loss: 0.1373 - acc: 0.9467 - val_loss: 0.6619 - val_acc: 0.8023
Epoch 25/60
910/910 [==============================] - 528s 580ms/step - loss: 0.1362 - acc: 0.9466 - val_loss: 0.0457 - val_acc: 0.9844
Epoch 26/60
910/910 [==============================] - 529s 582ms/step - loss: 0.1350 - acc: 0.9474 - val_loss: 0.8683 - val_acc: 0.7046
Epoch 27/60
910/910 [==============================] - 527s 579ms/step - loss: 0.1339 - acc: 0.9492 - val_loss: 0.0411 - val_acc: 0.9855
Epoch 28/60
910/910 [==============================] - 529s 581ms/step - loss: 0.1339 - acc: 0.9499 - val_loss: 0.9552 - val_acc: 0.6762
Epoch 29/60
910/910 [==============================] - 527s 579ms/step - loss: 0.1343 - acc: 0.9488 - val_loss: 0.0446 - val_acc: 0.9859
Epoch 30/60
910/910 [==============================] - 528s 580ms/step - loss: 0.1282 - acc: 0.9513 - val_loss: 0.8127 - val_acc: 0.7298
Epoch 31/60
910/910 [==============================] - 527s 579ms/step - loss: 0.1286 - acc: 0.9504 - val_loss: 0.0484 - val_acc: 0.9857
Epoch 32/60
910/910 [==============================] - 529s 581ms/step - loss: 0.1258 - acc: 0.9506 - val_loss: 0.5007 - val_acc: 0.8479
Epoch 33/60
910/910 [==============================] - 527s 579ms/step - loss: 0.1301 - acc: 0.9495 - val_loss: 0.0467 - val_acc: 0.9859
Epoch 34/60
910/910 [==============================] - 529s 581ms/step - loss: 0.1253 - acc: 0.9516 - val_loss: 0.6061 - val_acc: 0.8056
Epoch 35/60
910/910 [==============================] - 527s 579ms/step - loss: 0.1259 - acc: 0.9521 - val_loss: 0.0469 - val_acc: 0.9873
Epoch 36/60
910/910 [==============================] - 528s 580ms/step - loss: 0.1249 - acc: 0.9511 - val_loss: 0.8658 - val_acc: 0.7121
Epoch 37/60
910/910 [==============================] - 527s 579ms/step - loss: 0.1206 - acc: 0.9548 - val_loss: 0.0459 - val_acc: 0.9869
Epoch 38/60
910/910 [==============================] - 527s 580ms/step - loss: 0.1229 - acc: 0.9512 - val_loss: 0.4516 - val_acc: 0.8646
Epoch 39/60
910/910 [==============================] - 527s 579ms/step - loss: 0.1206 - acc: 0.9528 - val_loss: 0.0469 - val_acc: 0.9861
Epoch 40/60

下图不是针对这个问题,而是针对我所问的类似情况:

loss

acc


Tags: addsizemodelsteptrainvalgeneratoractivation
1条回答
网友
1楼 · 发布于 2024-03-29 04:52:49

我试着得到如下曲线:

enter image description here

  • 将模型的容量减少到只有几个神经元的几层。你知道吗
  • 降低了学习速率,并将批处理大小从32增加到128。你知道吗

这可能不是唯一可能的解决办法,但我提出了一些简单的提示,使事情正确。有用的链接:12

我还计划为更多的时代而训练。你知道吗

相关问题 更多 >