Tensorflow分配内存：38535168的分配超过系统内存的10%

Question

使用ResNet50预先训练的权重，我试图建立一个分类器。代码库完全在Keras高级Tensorflow API中实现。完整的代码发布在下面的GitHub链接中。

源代码：Classification Using RestNet50 Architecture

预训练模型的文件大小为94.7mb。

我加载了预先训练的文件

new_model = Sequential()

new_model.add(ResNet50(include_top=False,
                pooling='avg',
                weights=resnet_weight_paths))

并符合模型

train_generator = data_generator.flow_from_directory(
    'path_to_the_training_set',
    target_size = (IMG_SIZE,IMG_SIZE),
    batch_size = 12,
    class_mode = 'categorical'
    )

validation_generator = data_generator.flow_from_directory(
    'path_to_the_validation_set',
    target_size = (IMG_SIZE,IMG_SIZE),
    class_mode = 'categorical'
    )

#compile the model

new_model.fit_generator(
    train_generator,
    steps_per_epoch = 3,
    validation_data = validation_generator,
    validation_steps = 1
)

在训练数据集中，我有两个文件夹dog和cat，每个文件夹包含将近10000个图像。当我编译脚本时，得到以下错误

Epoch 1/1 2018-05-12 13:04:45.847298: W tensorflow/core/framework/allocator.cc:101] Allocation of 38535168 exceeds 10% of system memory. 2018-05-12 13:04:46.845021: W tensorflow/core/framework/allocator.cc:101] Allocation of 37171200 exceeds 10% of system memory. 2018-05-12 13:04:47.552176: W tensorflow/core/framework/allocator.cc:101] Allocation of 37171200 exceeds 10% of system memory. 2018-05-12 13:04:48.199240: W tensorflow/core/framework/allocator.cc:101] Allocation of 37171200 exceeds 10% of system memory. 2018-05-12 13:04:48.918930: W tensorflow/core/framework/allocator.cc:101] Allocation of 37171200 exceeds 10% of system memory. 2018-05-12 13:04:49.274137: W tensorflow/core/framework/allocator.cc:101] Allocation of 19267584 exceeds 10% of system memory. 2018-05-12 13:04:49.647061: W tensorflow/core/framework/allocator.cc:101] Allocation of 19267584 exceeds 10% of system memory. 2018-05-12 13:04:50.028839: W tensorflow/core/framework/allocator.cc:101] Allocation of 19267584 exceeds 10% of system memory. 2018-05-12 13:04:50.413735: W tensorflow/core/framework/allocator.cc:101] Allocation of 19267584 exceeds 10% of system memory.

有什么想法可以优化加载预先训练的模型（或）摆脱此警告消息的方式吗？

谢谢！

Answer 1

我在用Docker和Jupyter笔记本运行Tensorflow容器时遇到了同样的问题。我可以通过增加容器内存来解决这个问题。

在Mac OS上，您可以从以下位置轻松完成此操作：

       Docker Icon > Preferences >  Advanced > Memory

将滚动条拖动到最大值（例如4GB）。应用后将重新启动Docker引擎。

现在再次运行张力流容器。

在单独的终端中使用docker stats命令很方便 它实时显示容器内存使用情况，您可以看到内存消耗量正在增长：

CONTAINER ID   NAME   CPU %   MEM USAGE / LIMIT     MEM %    NET I/O             BLOCK I/O           PIDS
3170c0b402cc   mytf   0.04%   588.6MiB / 3.855GiB   14.91%   13.1MB / 3.06MB     214MB / 3.13MB      21