使用load_模型加载tensorflow.keras训练模型会返回JSON解码错误,而未训练的模型正常加载

2024-04-20 01:03:08 发布

您现在位置:Python中文网/ 问答频道 /正文

我使用tensorflow.Keras API构建和训练了一个经过训练的Keras模型,并使用tf.keras.save_model()方法保存该模型,没有可选参数。Tensorflow是最新的,我的Python版本是3.8。根据我的理解,这个方法应该使用默认的“tf”格式保存模型,这在tf2.X中是推荐的,然后使用load_model()就可以了

但是,再次加载模型会产生以下结果:

model = tf.keras.models.load_model("/Volumes/thesis_drive/thesis_project_local_new/trained_model_640x64/")

---------------------------------------------------------------------------
JSONDecodeError                           Traceback (most recent call last)
/var/folders/c1/8tq0t8y90195qxyt5hppjjtr0000gq/T/ipykernel_933/1131710361.py in <module>
----> 1 model = tf.keras.models.load_model("/Volumes/thesis_drive/thesis_project_local_new/trained_model_640x64/")

~/miniforge3/lib/python3.9/site-packages/tensorflow/python/keras/saving/save.py in load_model(filepath, custom_objects, compile, options)
    204         filepath = path_to_string(filepath)
    205         if isinstance(filepath, str):
--> 206           return saved_model_load.load(filepath, compile, options)
    207 
    208   raise IOError(

~/miniforge3/lib/python3.9/site-packages/tensorflow/python/keras/saving/saved_model/load.py in load(path, compile, options)
    153 
    154   # Finalize the loaded layers and remove the extra tracked dependencies.
--> 155   keras_loader.finalize_objects()
    156   keras_loader.del_tracking()
    157 

~/miniforge3/lib/python3.9/site-packages/tensorflow/python/keras/saving/saved_model/load.py in finalize_objects(self)
    624 
    625     # Initialize graph networks, now that layer dependencies have been resolved.
--> 626     self._reconstruct_all_models()
    627 
    628   def _unblock_model_reconstruction(self, layer_id, layer):

~/miniforge3/lib/python3.9/site-packages/tensorflow/python/keras/saving/saved_model/load.py in _reconstruct_all_models(self)
    643       all_initialized_models.add(model_id)
    644       model, layers = self.model_layer_dependencies[model_id]
--> 645       self._reconstruct_model(model_id, model, layers)
    646       _finalize_config_layers([model])
    647 

~/miniforge3/lib/python3.9/site-packages/tensorflow/python/keras/saving/saved_model/load.py in _reconstruct_model(self, model_id, model, layers)
    659   def _reconstruct_model(self, model_id, model, layers):
    660     """Reconstructs the network structure."""
--> 661     config = json_utils.decode(
    662         self._proto.nodes[model_id].user_object.metadata)['config']
    663 

~/miniforge3/lib/python3.9/site-packages/tensorflow/python/keras/saving/saved_model/json_utils.py in decode(json_string)
     60 
     61 def decode(json_string):
---> 62   return json.loads(json_string, object_hook=_decode_helper)
     63 
     64 

~/miniforge3/lib/python3.9/json/__init__.py in loads(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    357     if parse_constant is not None:
    358         kw['parse_constant'] = parse_constant
--> 359     return cls(**kw).decode(s)

~/miniforge3/lib/python3.9/json/decoder.py in decode(self, s, _w)
    335 
    336         """
--> 337         obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    338         end = _w(s, end).end()
    339         if end != len(s):

~/miniforge3/lib/python3.9/json/decoder.py in raw_decode(self, s, idx)
    353             obj, end = self.scan_once(s, idx)
    354         except StopIteration as err:
--> 355             raise JSONDecodeError("Expecting value", s, err.value) from None
    356         return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

为了测试这是否是save_model()load_model()的错误,我在Jupyter笔记本中再次构建了相同的模型,保存了它,然后重新加载,没有错误:

import tensorflow as tf
from tensorflow.keras.layers import Dense, Activation, Flatten, Dropout, Conv2D, MaxPooling2D, Input
from tensorflow.keras.losses import CategoricalCrossentropy
from tensorflow.keras import optimizers
from tensorflow.keras.models import Model

def build_model():
    _input = Input(shape=(640,64,3))
    x = Conv2D(filters=64, kernel_size=4, input_shape=(640, 64, 3))(_input)
    x = Activation('relu')(x)
    x = MaxPooling2D(pool_size=(4, 4))(x)
    x = Dropout(0.5)(x)
    x = Conv2D(filters=128, kernel_size=4, input_shape=(640, 64, 3))(_input)
    x = Activation('relu')(x)
    x = MaxPooling2D(pool_size=(4, 4))(x)
    x = Dropout(0.5)(x)
    x = Conv2D(filters=256, kernel_size=4, input_shape=(640, 64, 3))(_input)
    x = Activation('relu')(x)
    x = MaxPooling2D(pool_size=(2, 2))(x)
    x = Dropout(0.5)(x)
    x = Flatten()(x)
    output = Dense(161, activation = 'softmax')(x)
    model = Model(_input,output)
    model.compile(optimizer=optimizers.Adam(), loss="categorical_crossentropy")
    tf.keras.models.save_model(model,"model_test")

model = build_model()

Metal device set to: Apple M1
2021-09-23 13:40:46.234438: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.
2021-09-23 13:40:46.234631: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)
2021-09-23 13:40:47.112730: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
INFO:tensorflow:Assets written to: model_test/assets

del model
model = tf.keras.models.load_model("model_test")

进一步的细节:该模型是在另一台运行Linux的机器(我可以通过我的大学访问的超级计算机)上训练的,并通过SCP传输到我的Apple M1机器上,在那里它现在显示了这个加载错误

我不知道为什么要调用JSON模块——目录中似乎没有JSON文件。但是,考虑到在不进行训练和加载的情况下重建模型不会产生错误,我怀疑save没有正确执行


Tags: inpyselfidjsoninputmodelmodels