凯拉斯习俗

Question

我正在尝试添加一个OpenCV作为自定义层，以使另一个功能添加到resnet功能上，我已经成功地完成了。我正试图将两个特征合并为一个，但它一直给我所有的层必须是张量错误。你知道吗

Layer concatenate_1 was called with an input that isn't a symbolic tensor. Received type: <class 'numpy.ndarray'>. Full input: [<tf.Tensor 'resnet50/activation_49/Relu:0' shape=(?, 7, 7, 2048) dtype=float32>, array([<tf.Tensor 'Custom/StopGradient:0' shape=(?, 7, 7, 2048) dtype=float32>],
      dtype=object)]. All inputs to the layer should be tensors.

这是我到目前为止的情况。你知道吗

自定义层

def create_lbp_img(img):
    gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    lbp = feature.local_binary_pattern(gray_img, 10, 10, method="uniform")
    lbp = np.array(lbp, dtype = 'float32')
    #lbp = tf.convert_to_tensor(lbp, dtype='float32')
    return lbp

def image_tensor_translater(img4d):
    results = []
    for img3d in img4d:
        lbp3img = create_lbp_img(img3d)
        results.append(np.expand_dims(lbp3img, axis=0))
    return np.concatenate(results, axis=0)    

class LBPLayer(Layer):

    # Call function holds the logic of the layer
    # This should create the local binary pattern for texture feature
    def call(self, x):
        output = tf.py_func(image_tensor_translater,
                       [x],
                        'float32',
                       stateful=False,
                       name="openCVOutput")
        output = K.stop_gradient(output) # Explicitly set no gradient effect
        # Set the output shape to match the resnet output
        output.set_shape([x.shape[0], 7, 7, 2048])
        return output

    # Calculate the output shape
    def compute_output_shape(self, input_shape):
        return(input_shape[0], 7,7,2048)

模型和输入创建

# Create Model
input_dim = (224, 224, 3)
out_dim = 2048
# Create custom layer
f = LBPLayer(name="Custom")

my_image = Input(shape=input_dim)
# get features
resnet_out = resnet(my_image) # Model defined earlier
feature_lbp = f(my_image)

# Merge the layer
merge = Concatenate(axis=-1)([resnet_out , feature_lbp ])

model = Model(inputs=[anchor_in, pos_in, neg_in], outputs=merged_final)
adam = Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=None, decay=0.0, amsgrad=False)
model.compile(optimizer=adam, loss=triplet_loss)

现在，它已经通过了两层的处理，但仍然无法将它们合并在一起。你知道吗