将两个预训练张量流模型的预测串联起来

Question

我有两个独立的训练模型，分别从同一张图片预测不相关的回归值。两种型号均使用预训练VGG16型号作为基础，并添加了特定的顶层

单独测试时，两种模型都表现良好。当我连接两个预先训练的模型时，我得到的预测与单独运行不同

我宣布各型号如下：

# VGG
vggModel = tf.keras.applications.VGG16(weights='imagenet', include_top=False, input_shape=(310, 765, 3))
vggModel.trainable = True
trainableFlag = False

for layer in vggModel.layers:
    
    if layer.name == 'block5_conv1':
        trainableFlag = True
  
    layer.trainable = trainableFlag

# Model A
model_a = tf.keras.Sequential(name='model_a')
model_a.add(vggModel)
model_a.add(tf.keras.layers.Flatten())
model_a.add(tf.keras.layers.Dropout(0.1))
model_a.add(tf.keras.layers.Dense(128, activation='relu', kernel_regularizer=tf.keras.regularizers.l1(0.01)))
model_a.add(tf.keras.layers.Dense(128, activation='relu', kernel_regularizer=tf.keras.regularizers.l1(0.01)))
model_a.add(tf.keras.layers.Dense(1, activation='linear'))
model_a.load_weights(model_a_wts)

# Model B
model_b = tf.keras.Sequential(name='model_b')
model_b.add(vggModel)
model_b.add(tf.keras.layers.Flatten())
model_b.add(tf.keras.layers.Dropout(0.1))
model_b.add(tf.keras.layers.Dense(256, activation='relu', kernel_regularizer=tf.keras.regularizers.l1(0.01)))
model_b.add(tf.keras.layers.Dense(256, activation='relu', kernel_regularizer=tf.keras.regularizers.l1(0.01)))
model_b.add(tf.keras.layers.Dense(1, activation='linear'))
model_b.load_weights(model_b_wts)

然后我通过以下方式连接模型：

common_input = tf.keras.Input(shape=(310, 765, 3))

a_out = model_a(common_input)
b_out = model_b(common_input)

concatOut = tf.keras.layers.Concatenate()([a_out, b_out])

branched_model = tf.keras.Model(common_input, concatOut, name='Branched')

在这种情况下，如何可能得到不同的预测

Answer 1

根据@hkyi的评论，答案是：

这两个模型并非完全独立，因为它们共享VGG模型（可培训）。因此，您需要在将vggModel添加到model_b之前克隆它。否则，加载到模型_b的权重将覆盖模型_a中使用的VGG模型权重

改用tf.keras.models.clone_model（vggModel）：

# VGG
vggModel = tf.keras.applications.VGG16(weights='imagenet', include_top=False, input_shape=(310, 765, 3))
vggModel.trainable = True
trainableFlag = False

for layer in vggModel.layers:
    
    if layer.name == 'block5_conv1':
        trainableFlag = True
  
    layer.trainable = trainableFlag

# Model A
model_a = tf.keras.Sequential(name='model_a')
model_a.add(vggModel)
model_a.add(tf.keras.layers.Flatten())
model_a.add(tf.keras.layers.Dropout(0.1))
model_a.add(tf.keras.layers.Dense(128, activation='relu', kernel_regularizer=tf.keras.regularizers.l1(0.01)))
model_a.add(tf.keras.layers.Dense(128, activation='relu', kernel_regularizer=tf.keras.regularizers.l1(0.01)))
model_a.add(tf.keras.layers.Dense(1, activation='linear'))
model_a.load_weights(model_a_wts)

# Model B
model_b = tf.keras.Sequential(name='model_b')
model_b.add(tf.keras.models.clone_model(vggModel))   < - HERE IS THE CHANGE REQUIRED
model_b.add(tf.keras.layers.Flatten())
model_b.add(tf.keras.layers.Dropout(0.1))
model_b.add(tf.keras.layers.Dense(256, activation='relu', kernel_regularizer=tf.keras.regularizers.l1(0.01)))
model_b.add(tf.keras.layers.Dense(256, activation='relu', kernel_regularizer=tf.keras.regularizers.l1(0.01)))
model_b.add(tf.keras.layers.Dense(1, activation='linear'))
model_b.load_weights(model_b_wts)