tfnightlygpu在整个培训过程中会降低性能

2024-04-16 06:57:44 发布

您现在位置:Python中文网/ 问答频道 /正文

  • 视窗10
  • RTX 3070
  • CUDA 11.1
  • cuDNN 8.0.5(适用于CUDA 11.1)
  • python 3.8.5
  • tf夜间gpu 2.5.0.dev20201113
  • 使用Python环境

我终于设法让tf夜间gpu与我的3070工作,然而,在训练时,性能似乎呈指数级下降。从50年代开始,到700年代结束。在TaskManager中,在性能下降一段时间后,3070似乎几乎没有被使用。我还没有真正尝试过什么,因为我不知道该做什么。有什么建议吗

以下是我的完整代码和输出:

import numpy as np
import os
import tensorflow as tf
from keras.callbacks import EarlyStopping
from keras.layers.core import Dense, Dropout, Activation, Flatten
from keras.layers import Conv2D, MaxPooling2D, BatchNormalization, LeakyReLU
from keras.models import Sequential
from keras.preprocessing.image import ImageDataGenerator
from keras.regularizers import l2

config = tf.compat.v1.ConfigProto(gpu_options = tf.compat.v1.GPUOptions(per_process_gpu_memory_fraction=0.70))
for device in tf.config.experimental.list_physical_devices("GPU"):
    tf.config.experimental.set_memory_growth(device, True)
session = tf.compat.v1.Session(config=config)
tf.compat.v1.keras.backend.set_session(session)

IMG_SIZE = 350
Version = 1
batch_size = 8

val_aug = ImageDataGenerator(rescale=1/255)
aug = ImageDataGenerator(
        rescale=1/255, 
        rotation_range=30, 
        width_shift_range=0.1, 
        height_shift_range=0.1, 
        shear_range=0.2, 
        zoom_range=0.2, 
        channel_shift_range=25, 
        horizontal_flip=True, 
        fill_mode='constant')

train_gen = aug.flow_from_directory('F:/Storage/DataSet_Bal/Train', 
        target_size=(IMG_SIZE, IMG_SIZE), 
        batch_size=batch_size,
        class_mode='binary',
        shuffle=True)
val_gen = val_aug.flow_from_directory('F:/Storage/DataSet_Bal/Val', 
        target_size=(IMG_SIZE, IMG_SIZE), 
        batch_size=batch_size,
        class_mode='binary',
        shuffle=True)

model = Sequential()
model.add(Conv2D(64, 3, strides=(1,1), padding = 'same', activation = 'relu', input_shape = (IMG_SIZE, IMG_SIZE, 3)))
model.add(BatchNormalization())
model.add(Conv2D(64, 3, strides=(1,1), activation = 'relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), strides=2))
model.add(BatchNormalization())

model.add(Conv2D(128, 3, strides=(1,1), activation = 'relu'))
model.add(BatchNormalization())
model.add(Conv2D(128, 3, strides=(1,1), activation = 'relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), strides=2))
model.add(BatchNormalization())

model.add(Conv2D(256, 3, strides=(1,1), activation = 'relu'))
model.add(BatchNormalization())
model.add(Conv2D(256, 3, strides=(1,1), activation = 'relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), strides=2))
model.add(BatchNormalization())

model.add(Conv2D(512, 3, strides=(1,1), activation = 'relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(BatchNormalization())

model.add(Flatten())
model.add(Dense(128, activation = 'relu'))
model.add(BatchNormalization())
model.add(Dropout(0.5))
model.add(Dense(32, activation = 'relu'))
model.add(BatchNormalization())
model.add(Dropout(0.5))

model.add(Dense(1, activation = 'sigmoid'))


model.compile(loss = "binary_crossentropy", optimizer = 'adam', metrics = ['accuracy'])
#model.summary()

earlyStop = EarlyStopping(monitor = 'val_accuracy', min_delta = 0.0001, patience = 50, restore_best_weights = True)
model.fit(
    train_gen,
    workers=8,
    epochs= 250,
    validation_data=val_gen,
    callbacks=earlyStop,
    verbose=2)

model.save(f'F:/Storage/TrainedVersions/YiffModel{Version}')

输出:

2020-11-13 10:36:05.885624: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2020-11-13 10:36:07.923109: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2020-11-13 10:36:07.925060: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library nvcuda.dll
2020-11-13 10:36:07.966126: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1724] Found device 0 with properties: 
pciBusID: 0000:01:00.0 name: GeForce RTX 3070 computeCapability: 8.6
coreClock: 1.725GHz coreCount: 46 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 417.29GiB/s
2020-11-13 10:36:07.966451: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2020-11-13 10:36:07.973267: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2020-11-13 10:36:07.973467: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2020-11-13 10:36:07.977131: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2020-11-13 10:36:07.978353: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2020-11-13 10:36:07.986944: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2020-11-13 10:36:07.990210: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2020-11-13 10:36:07.991036: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2020-11-13 10:36:07.991349: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1866] Adding visible gpu devices: 0
2020-11-13 10:36:08.537694: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1265] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-11-13 10:36:08.537864: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1271]      0 
2020-11-13 10:36:08.537967: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1284] 0:   N 
2020-11-13 10:36:08.538347: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1410] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 5734 MB memory) -> physical GPU (device: 0, name: GeForce RTX 3070, pci bus id: 0000:01:00.0, compute capability: 8.6)
2020-11-13 10:36:08.539462: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
WARNING:tensorflow:From e:\PYTHON\YiffMiner\TrainYIFF.py:15: The name tf.keras.backend.set_session is deprecated. Please use tf.compat.v1.keras.backend.set_session instead.

2020-11-13 10:36:08.659266: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2020-11-13 10:36:08.659494: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1724] Found device 0 with properties: 
pciBusID: 0000:01:00.0 name: GeForce RTX 3070 computeCapability: 8.6
coreClock: 1.725GHz coreCount: 46 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 417.29GiB/s
2020-11-13 10:36:08.659873: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2020-11-13 10:36:08.660078: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2020-11-13 10:36:08.660248: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2020-11-13 10:36:08.660466: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2020-11-13 10:36:08.660683: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2020-11-13 10:36:08.660912: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2020-11-13 10:36:08.661140: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2020-11-13 10:36:08.661317: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2020-11-13 10:36:08.661570: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1866] Adding visible gpu devices: 0
2020-11-13 10:36:08.662123: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1724] Found device 0 with properties: 
pciBusID: 0000:01:00.0 name: GeForce RTX 3070 computeCapability: 8.6
coreClock: 1.725GHz coreCount: 46 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 417.29GiB/s
2020-11-13 10:36:08.662477: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
2020-11-13 10:36:08.662692: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2020-11-13 10:36:08.662877: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2020-11-13 10:36:08.663105: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll
2020-11-13 10:36:08.663290: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll
2020-11-13 10:36:08.663528: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll
2020-11-13 10:36:08.663727: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll
2020-11-13 10:36:08.663931: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2020-11-13 10:36:08.664231: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1866] Adding visible gpu devices: 0
2020-11-13 10:36:08.664427: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1265] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-11-13 10:36:08.664608: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1271]      0 
2020-11-13 10:36:08.664718: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1284] 0:   N 
2020-11-13 10:36:08.664948: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1410] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 5734 MB memory) -> physical GPU (device: 0, name: GeForce RTX 3070, pci bus id: 0000:01:00.0, compute capability: 8.6)
2020-11-13 10:36:08.665299: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2020-11-13 10:36:09.429191: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:126] None of the MLIR optimization passes are enabled (registered 2)
2020-11-13 10:36:11.556454: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dll
2020-11-13 10:36:12.526029: I tensorflow/stream_executor/cuda/cuda_dnn.cc:344] Loaded cuDNN version 8005
2020-11-13 10:36:13.224088: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code: 0

2020-11-13 10:36:13.264015: I tensorflow/core/platform/windows/subprocess.cc:308] SubProcess ended with return code: 0

2020-11-13 10:36:13.345749: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll
2020-11-13 10:36:14.220640: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll
2020-11-13 10:36:15.740837: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
C:\Users\circu\anaconda3\envs\tf2\python.exe
Found 3266 images belonging to 2 classes.
Found 86 images belonging to 2 classes.
Epoch 1/250
409/409 - 63s - loss: 0.8360 - accuracy: 0.5113 - val_loss: 0.7363 - val_accuracy: 0.5000
Epoch 2/250
409/409 - 51s - loss: 0.7559 - accuracy: 0.4951 - val_loss: 0.6892 - val_accuracy: 0.4651
Epoch 3/250
409/409 - 51s - loss: 0.7106 - accuracy: 0.5211 - val_loss: 0.6842 - val_accuracy: 0.4884
Epoch 4/250
409/409 - 50s - loss: 0.6959 - accuracy: 0.5380 - val_loss: 0.6874 - val_accuracy: 0.5465
Epoch 5/250
409/409 - 50s - loss: 0.6918 - accuracy: 0.5508 - val_loss: 0.7049 - val_accuracy: 0.5465
Epoch 6/250
409/409 - 50s - loss: 0.6875 - accuracy: 0.5554 - val_loss: 0.6623 - val_accuracy: 0.5814
Epoch 7/250
409/409 - 50s - loss: 0.6813 - accuracy: 0.5652 - val_loss: 0.6630 - val_accuracy: 0.6047
Epoch 8/250
409/409 - 50s - loss: 0.6770 - accuracy: 0.5720 - val_loss: 0.6815 - val_accuracy: 0.5698
Epoch 9/250
409/409 - 50s - loss: 0.6843 - accuracy: 0.5517 - val_loss: 0.7317 - val_accuracy: 0.6047
Epoch 10/250
409/409 - 50s - loss: 0.6791 - accuracy: 0.5765 - val_loss: 0.6666 - val_accuracy: 0.6047
Epoch 11/250
409/409 - 50s - loss: 0.6751 - accuracy: 0.5870 - val_loss: 0.6628 - val_accuracy: 0.6163
Epoch 12/250
409/409 - 50s - loss: 0.6654 - accuracy: 0.5989 - val_loss: 0.6964 - val_accuracy: 0.5465
Epoch 13/250
409/409 - 50s - loss: 0.6703 - accuracy: 0.5928 - val_loss: 0.6455 - val_accuracy: 0.6395
Epoch 14/250
409/409 - 50s - loss: 0.6722 - accuracy: 0.5814 - val_loss: 0.6470 - val_accuracy: 0.6395
Epoch 15/250
409/409 - 51s - loss: 0.6675 - accuracy: 0.5946 - val_loss: 0.6208 - val_accuracy: 0.6628
Epoch 16/250
409/409 - 52s - loss: 0.6710 - accuracy: 0.5882 - val_loss: 0.6413 - val_accuracy: 0.6860
Epoch 17/250
409/409 - 55s - loss: 0.6696 - accuracy: 0.5937 - val_loss: 0.9023 - val_accuracy: 0.5465
Epoch 18/250
409/409 - 58s - loss: 0.6763 - accuracy: 0.5860 - val_loss: 0.6736 - val_accuracy: 0.5814
Epoch 19/250
409/409 - 62s - loss: 0.6790 - accuracy: 0.5848 - val_loss: 0.6366 - val_accuracy: 0.6163
Epoch 20/250
409/409 - 66s - loss: 0.6716 - accuracy: 0.5943 - val_loss: 0.5956 - val_accuracy: 0.6860
Epoch 21/250
409/409 - 65s - loss: 0.6745 - accuracy: 0.5824 - val_loss: 0.6232 - val_accuracy: 0.6279
Epoch 22/250
409/409 - 69s - loss: 0.6675 - accuracy: 0.6038 - val_loss: 0.6290 - val_accuracy: 0.6512
Epoch 23/250
409/409 - 73s - loss: 0.6670 - accuracy: 0.6007 - val_loss: 0.6173 - val_accuracy: 0.6744
Epoch 24/250
409/409 - 77s - loss: 0.6583 - accuracy: 0.6139 - val_loss: 0.7856 - val_accuracy: 0.6163
Epoch 25/250
409/409 - 82s - loss: 0.6656 - accuracy: 0.6035 - val_loss: 0.5991 - val_accuracy: 0.7093
Epoch 26/250
409/409 - 86s - loss: 0.6561 - accuracy: 0.6234 - val_loss: 0.6191 - val_accuracy: 0.6047
Epoch 27/250
409/409 - 90s - loss: 0.6593 - accuracy: 0.6081 - val_loss: 0.7744 - val_accuracy: 0.5581
Epoch 28/250
409/409 - 95s - loss: 0.6630 - accuracy: 0.6035 - val_loss: 1.0304 - val_accuracy: 0.5698
Epoch 29/250
409/409 - 100s - loss: 0.6620 - accuracy: 0.5977 - val_loss: 0.6371 - val_accuracy: 0.6512
Epoch 30/250
409/409 - 105s - loss: 0.6499 - accuracy: 0.6197 - val_loss: 0.5892 - val_accuracy: 0.6512
Epoch 31/250
409/409 - 110s - loss: 0.6589 - accuracy: 0.6179 - val_loss: 0.6426 - val_accuracy: 0.6163
Epoch 32/250
409/409 - 116s - loss: 0.6542 - accuracy: 0.6139 - val_loss: 0.6295 - val_accuracy: 0.6279
Epoch 33/250
409/409 - 122s - loss: 0.6557 - accuracy: 0.6240 - val_loss: 0.5679 - val_accuracy: 0.7442
Epoch 34/250
409/409 - 127s - loss: 0.6515 - accuracy: 0.6219 - val_loss: 0.6137 - val_accuracy: 0.6512
Epoch 35/250
409/409 - 133s - loss: 0.6510 - accuracy: 0.6283 - val_loss: 0.6011 - val_accuracy: 0.6860
Epoch 36/250
409/409 - 140s - loss: 0.6495 - accuracy: 0.6332 - val_loss: 0.6108 - val_accuracy: 0.6163
Epoch 37/250
409/409 - 146s - loss: 0.6450 - accuracy: 0.6219 - val_loss: 0.6034 - val_accuracy: 0.6860
Epoch 38/250
409/409 - 153s - loss: 0.6444 - accuracy: 0.6320 - val_loss: 0.6000 - val_accuracy: 0.6860
Epoch 39/250
409/409 - 161s - loss: 0.6599 - accuracy: 0.6056 - val_loss: 0.6403 - val_accuracy: 0.5930
Epoch 40/250
409/409 - 168s - loss: 0.6466 - accuracy: 0.6332 - val_loss: 0.6008 - val_accuracy: 0.6047
Epoch 41/250
409/409 - 177s - loss: 0.6494 - accuracy: 0.6277 - val_loss: 0.6248 - val_accuracy: 0.6047
Epoch 42/250
409/409 - 184s - loss: 0.6458 - accuracy: 0.6286 - val_loss: 0.6352 - val_accuracy: 0.5581
Epoch 43/250
409/409 - 193s - loss: 0.6320 - accuracy: 0.6445 - val_loss: 0.6383 - val_accuracy: 0.6279
Epoch 44/250
409/409 - 203s - loss: 0.6468 - accuracy: 0.6231 - val_loss: 0.7030 - val_accuracy: 0.5465
Epoch 45/250
409/409 - 212s - loss: 0.6365 - accuracy: 0.6467 - val_loss: 0.6000 - val_accuracy: 0.6977
Epoch 46/250
409/409 - 221s - loss: 0.6351 - accuracy: 0.6494 - val_loss: 0.5629 - val_accuracy: 0.7326
Epoch 47/250
409/409 - 231s - loss: 0.6378 - accuracy: 0.6482 - val_loss: 0.6051 - val_accuracy: 0.6628
Epoch 48/250
409/409 - 241s - loss: 0.6342 - accuracy: 0.6473 - val_loss: 0.5751 - val_accuracy: 0.6860
Epoch 49/250
409/409 - 293s - loss: 0.6365 - accuracy: 0.6424 - val_loss: 0.5925 - val_accuracy: 0.6395
Epoch 50/250
409/409 - 262s - loss: 0.6351 - accuracy: 0.6399 - val_loss: 0.6040 - val_accuracy: 0.6744
Epoch 51/250
409/409 - 278s - loss: 0.6380 - accuracy: 0.6356 - val_loss: 0.6200 - val_accuracy: 0.6512
Epoch 52/250
409/409 - 286s - loss: 0.6351 - accuracy: 0.6436 - val_loss: 0.6334 - val_accuracy: 0.6395
Epoch 53/250
409/409 - 300s - loss: 0.6369 - accuracy: 0.6454 - val_loss: 0.6013 - val_accuracy: 0.7093
Epoch 54/250
409/409 - 312s - loss: 0.6352 - accuracy: 0.6430 - val_loss: 0.5895 - val_accuracy: 0.6628
Epoch 55/250
409/409 - 318s - loss: 0.6385 - accuracy: 0.6445 - val_loss: 0.6480 - val_accuracy: 0.5814
Epoch 56/250
409/409 - 336s - loss: 0.6340 - accuracy: 0.6448 - val_loss: 0.5963 - val_accuracy: 0.6395
Epoch 57/250
409/409 - 350s - loss: 0.6288 - accuracy: 0.6513 - val_loss: 0.6665 - val_accuracy: 0.5465
Epoch 58/250
409/409 - 360s - loss: 0.6267 - accuracy: 0.6528 - val_loss: 0.5441 - val_accuracy: 0.6860
Epoch 59/250
409/409 - 374s - loss: 0.6221 - accuracy: 0.6540 - val_loss: 0.5957 - val_accuracy: 0.6395
Epoch 60/250
409/409 - 383s - loss: 0.6181 - accuracy: 0.6604 - val_loss: 0.6006 - val_accuracy: 0.6744
Epoch 61/250
409/409 - 397s - loss: 0.6289 - accuracy: 0.6494 - val_loss: 0.6211 - val_accuracy: 0.6628
Epoch 62/250
409/409 - 416s - loss: 0.6250 - accuracy: 0.6500 - val_loss: 0.5410 - val_accuracy: 0.6977
Epoch 63/250
409/409 - 429s - loss: 0.6222 - accuracy: 0.6647 - val_loss: 0.7609 - val_accuracy: 0.4767
Epoch 64/250
409/409 - 443s - loss: 0.6295 - accuracy: 0.6479 - val_loss: 0.6089 - val_accuracy: 0.6628
Epoch 65/250
409/409 - 456s - loss: 0.6452 - accuracy: 0.6326 - val_loss: 0.6687 - val_accuracy: 0.5814
Epoch 66/250
409/409 - 562s - loss: 0.6400 - accuracy: 0.6363 - val_loss: 0.5692 - val_accuracy: 0.7209
Epoch 67/250
409/409 - 658s - loss: 0.6267 - accuracy: 0.6445 - val_loss: 0.5671 - val_accuracy: 0.7209
Epoch 68/250
409/409 - 523s - loss: 0.6283 - accuracy: 0.6549 - val_loss: 0.5553 - val_accuracy: 0.6977
Epoch 69/250
409/409 - 524s - loss: 0.6513 - accuracy: 0.6258 - val_loss: 0.6875 - val_accuracy: 0.5465
Epoch 70/250
409/409 - 543s - loss: 0.6652 - accuracy: 0.5980 - val_loss: 0.6886 - val_accuracy: 0.6047
Epoch 71/250
409/409 - 558s - loss: 0.6457 - accuracy: 0.6310 - val_loss: 0.6871 - val_accuracy: 0.5930
Epoch 72/250
409/409 - 576s - loss: 0.6360 - accuracy: 0.6310 - val_loss: 0.6281 - val_accuracy: 0.6395
Epoch 73/250
409/409 - 597s - loss: 0.6598 - accuracy: 0.6096 - val_loss: 0.6684 - val_accuracy: 0.5349
Epoch 74/250
409/409 - 613s - loss: 0.6526 - accuracy: 0.6139 - val_loss: 0.6379 - val_accuracy: 0.5698
Epoch 75/250
409/409 - 627s - loss: 0.6405 - accuracy: 0.6381 - val_loss: 0.6337 - val_accuracy: 0.5581
Epoch 76/250
409/409 - 650s - loss: 0.6306 - accuracy: 0.6555 - val_loss: 0.6146 - val_accuracy: 0.5814
Epoch 77/250
409/409 - 669s - loss: 0.6244 - accuracy: 0.6479 - val_loss: 0.6165 - val_accuracy: 0.6279
Epoch 78/250
409/409 - 678s - loss: 0.6313 - accuracy: 0.6513 - val_loss: 0.5995 - val_accuracy: 0.6512
Epoch 79/250
409/409 - 702s - loss: 0.6297 - accuracy: 0.6473 - val_loss: 0.6190 - val_accuracy: 0.6163
Epoch 80/250
409/409 - 732s - loss: 0.6212 - accuracy: 0.6540 - val_loss: 0.6515 - val_accuracy: 0.5581

[Done] exited with code=1 in 20909.362 seconds

Tags: adddefaultstreammodelgputensorflowvalloader