我有一个不平衡的小数据集,包含4116 224x224x3(RGB)航空图像。我很可能会遇到过度拟合的问题,因为数据集不够大。图像预处理和数据增强有助于解决这个问题,如下所述。在
"Overfitting is caused by having too few samples to learn from, rendering you unable to train a model that can generalize to new data. Given infinite data, your model would be exposed to every possible aspect of the data distribution at hand: you would never overfit. Data augmentation takes the approach of generating more training data from existing training samples, by augmenting the samples via a number of random transformations that yield believable-looking images."
Deep Learning with Python by François Chollet, page 138-139, 5.2.5 Using data augmentation.
我读了Medium - Image Data Preprocessing for Neural Networks并检查了斯坦福大学的CS230 - Data Preprocessing和 CS231 - Data Preprocessing课程。它在SO question中再次突出显示,我理解不存在“一刀切”的解决方案。是什么迫使我问这个问题:
"No translation augmentation was used since we want to achieve high spatial resolution."
我知道我将使用Keras - ImageDataGenerator Class,但不知道在小对象的语义分割任务中使用哪些技术和参数。有人能开导我吗?提前谢谢。:)
from keras.preprocessing.image import ImageDataGenerator
datagen = ImageDataGenerator(
rotation_range=20, # is a value in degrees (0–180)
width_shift_range=0.2, # is a range within which to randomly translate pictures horizontally.
height_shift_range=0.2, # is a range within which to randomly translate pictures vertically.
shear_range=0.2, # is for randomly applying shearing transformations.
zoom_range=0.2, # is for randomly zooming inside pictures.
horizontal_flip=True, # is for randomly flipping half the images horizontally
fill_mode='nearest', # is the strategy used for filling in newly created pixels, which can appear after a rotation or a width/height shift
featurewise_center=True,
featurewise_std_normalization=True)
datagen.fit(X_train)
扩充和预处理阶段总是取决于您遇到的问题。你必须考虑所有可能的扩充,以扩大你的数据集。但最重要的是,你不应该进行极端的扩充,这使得新的训练样本无法在真实的例子中发生。如果您不希望实际的示例将水平翻转,请不要执行水平翻转,因为这会给您的模型提供错误的信息。考虑所有可能发生在输入图像中的更改,并尝试从现有图像中人为地生成新图像。您可以使用Keras的许多内置函数。但是你应该意识到每一个都不会产生新的例子,这些例子不太可能出现在你的模型的输入中。在
正如你所说,没有“一刀切”的解决方案,因为一切都取决于数据。分析数据并建立与之相关的一切。在
关于小物体-你应该检查的一个方向是损失函数,它强调目标体积相对于背景的影响。看看骰子损失或一般骰子损失。在
相关问题 更多 >
编程相关推荐