Keras和Pytorch的权重相同，但结果不同

Question

我有一个编码器和一个解码器模型（monodepth2）。我尝试使用Onnx2Keras将它们从Pytorch转换为Keras，但是：

• 编码器（ResNet-18）成功
• 我自己在Keras中构建解码器（使用TF2.3），并将Pytorch到Keras的每一层的权重（numpy数组，包括权重和偏差），无需任何修改

但是Onnx2Keras转换的编码器和自建的解码器都无法再现相同的结果。下面是交叉比较图片，但我首先介绍解码器的代码

首先是核心层，所有conv2d层（Conv3x3，ConvBlock）都基于此，但不同的DIM或添加激活：

# Conv3x3 (normal conv2d without BN nor activation)
# There's also a ConvBlock, which is just "Conv3x3 + ELU activation", so I don't list it here.
def TF_Conv3x3(input_channel, filter_num, pad_mode='reflect', activate_type=None):

    # Actually it's 'reflect, but I implement it with tf.pad() outside this
    padding = 'valid'  

    # if TF_ConvBlock, then activate_type=='elu
    conv = tf.keras.layers.Conv2D(filters=filter_num, kernel_size=3, activation=activate_type,
                                  strides=1, padding=padding)
    return conv

然后是结构。请注意，该定义与原始code完全相同。我想一定是实施的一些细节

def DepthDecoder_keras(num_ch_enc=np.array([64, 64, 128, 256, 512]), channel_first=False,
                       scales=range(4), num_output_channels=1):
    num_ch_dec = np.array([16, 32, 64, 128, 256])
    convs = OrderedDict()
    for i in range(4, -1, -1):
        # upconv_0
        num_ch_in = num_ch_enc[-1] if i == 4 else num_ch_dec[i + 1]
        num_ch_out = num_ch_dec[i]

        # convs[("upconv", i, 0)] = ConvBlock(num_ch_in, num_ch_out)
        convs[("upconv", i, 0)] = TF_ConvBlock(num_ch_in, num_ch_out, pad_mode='reflect')


        # upconv_1
        num_ch_in = num_ch_dec[i]
        if i > 0:
            num_ch_in += num_ch_enc[i - 1]
        num_ch_out = num_ch_dec[i]
        convs[("upconv", i, 1)] = TF_ConvBlock(num_ch_in, num_ch_out, pad_mode='reflect')  # Just Conv3x3 with ELU-activation

    for s in scales:
        convs[("dispconv", s)] = TF_Conv3x3(num_ch_dec[s], num_output_channels, pad_mode='reflect')

    """
    Input_layer dims: (64, 96, 320), (64, 48, 160),  (128, 24, 80), (256, 12, 40), (512, 6, 20)
    """
    x0 = tf.keras.layers.Input(shape=(96, 320, 64))
    # then define the the rest input layers
    input_features = [x0, x1, x2, x3, x4]

    """
    # connect layers
    """
    outputs = []
    ch = 1 if channel_first else 3
    x = input_features[-1]
    for i in range(4, -1, -1):
        x = tf.pad(x, paddings=[[0, 0], [1, 1], [1, 1], [0, 0]], mode='REFLECT')
        x = convs[("upconv", i, 0)](x)
        x = [tf.keras.layers.UpSampling2D()(x)]
        if i > 0:
            x += [input_features[i - 1]]
        x = tf.concat(x, ch)
        x = tf.pad(x, paddings=[[0, 0], [1, 1], [1, 1], [0, 0]], mode='REFLECT')
        x = convs[("upconv", i, 1)](x)
    x = TF_ReflectPad2D_1()(x)
    x = convs[("dispconv", 0)](x)
    disp0 = tf.math.sigmoid(x)

    """
    build keras Model ([input0, ...], [output0, ...])
    """
    # decoder = tf.keras.Model(input_features, outputs)
    decoder = tf.keras.Model(input_features, disp0)

    return decoder

交叉比较如下。。。如果有人能提供一些见解，我将不胜感激。谢谢

原始结果：

原始编码器+自建解码器：

ONNX转换Enc+原始Dec（纹理不错，但对比度不够，车应该很近，即颜色很亮）：

ONNX转换Enc+自建Dec：

Answer 1

解决了

事实证明，实现确实没有问题（至少没有重大问题）。这是weights复制的问题

原始权重有（H，W，3，3），但TF模型要求dim为（3，3，W，H），因此我用[3,2,1,0]对其进行了置换，忽略了（3，3）也有自己的序列

所以应该是weights.permute([2,3,1,0])，一切都很好