Opencv DNN在YoloV3上设置不同的输入

Question

我正在使用来自YoloV3的通用实现来进行一些推断。使用常规的输入和输出可以很好地工作

modelWeightPath = r"./yolov3.weights"
modelPath = r"./yolov3.cfg"
network = cv2.dnn.readNetFromDarknet(modelPath,modelWeightPath)

由于我们正在使用一些边缘设备，这些设备通常无法“转换”最后几层，因此我将使用原始实现仅对最后几层进行推断

我知道这些层是如何命名的（network.getLayerNames()），并且我知道上一层的de数据是什么样子的，因为我保存了它们来进行测试。（请参见conv_81层的输入数据->；inputScale1）

inputLayers = ['permute_82','permute_94','permute_106']
inputData = [cv2.UMat(inputScale1),cv2.UMat(inputScale2),cv2.UMat(inputScale3)]

现在我不确定我应该如何使用这些知识来进行推理，因为我所有的尝试都只得到例外

network.setInput(blob=inputData[0],name=inputLayers[0]) - throws
outs = network.forward(outputlayers[0])

引发以下异常：OpenCV(4.0.1) C:\ci\opencv-suite_1573470242804\work\modules\dnn\src\dnn.cpp:2929: error: (-204:Requested object was not found) Requested blob "permute_82" not found in function 'cv::dnn::dnn4_v20181221::Net::setInput'

network.setInputsNames(inputLayers)
network.setInput(inputData[0],name=inputLayers[0])
network.setInput(inputData[1],name=inputLayers[1])
network.setInput(inputData[2],name=inputLayers[2])
outs = network.forward() -> throws

将抛出：cv2.error: OpenCV(4.0.1) C:\ci\opencv-suite_1573470242804\work\modules\dnn\src\dnn.cpp:686: error: (-215:Assertion failed) inputs.size() == requiredOutputs in function 'cv::dnn::dnn4_v20181221::DataLayer::getMemoryShapes'

编辑： 但问题是，这个例子是有效的：

imgPath = r'./frame_93.png'
image = cv2.imread(imgPath);
blobInputimage = cv2.dnn.blobFromImage(image,1.0 / 255.0,(416,416),(0, 0, 0))
network.setInputsNames(['conv_0'])
network.setInput(blobInputimage,name='conv_0')
output = network.forward('conv_81')

但仍然不能仅从排列层转发到yolo层

有人知道解决办法吗

Answer 1

到目前为止，我能够得到相同的结果，就像我通过推断常规网络得到的结果一样。因此，我通过删除除yolo entrie之外的所有条目，从.cfg文件中按比例创建了一个“新”网络，例如yolov3_scale_1.cfg如下所示：

[net]
# Testing
# batch=1
# subdivisions=1
# Training
batch=64
subdivisions=16
width=255
height=13
channels=13
momentum=0.9
decay=0.0005
angle=0
saturation = 1.5
exposure = 1.5
hue=.1

learning_rate=0.001
burn_in=1000
max_batches = 500200
policy=steps
steps=400000,450000
scales=.1,.1


[yolo]
mask = 6,7,8
anchors = 10,13,  16,30,  33,23,  30,61,  62,45,  59,119,  116,90,  156,198,  373,326
classes=80
num=9
jitter=.3
ignore_thresh = .7
truth_thresh = 1
random=1

验证输出的代码如下所示：

def getOutputLayerNames(network):
    layer_names = network.getLayerNames() 
    outputlayers=[layer_names[i[0] - 1] for i in network.getUnconnectedOutLayers()]
    return outputlayers

# Load Network
modelWeightPath = r".\yolov3.weights"
modelPath_scale_1 = r".\yolov3_scale_1.cfg"
network_scale_1 = cv2.dnn.readNetFromDarknet(modelPath_scale_1,modelWeightPath)

modelPath_scale_2 = r".\yolov3_scale_2.cfg"
network_scale_2 = cv2.dnn.readNetFromDarknet(modelPath_scale_2,modelWeightPath)

modelPath_scale_3 = r".\yolov3_scale_3.cfg"
network_scale_3 = cv2.dnn.readNetFromDarknet(modelPath_scale_3,modelWeightPath)

networks = [network_scale_1,network_scale_2,network_scale_3]

outputLayers1 = getOutputLayerNames(network_scale_1)
outputLayers2 = getOutputLayerNames(network_scale_2)
outputLayers3 = getOutputLayerNames(network_scale_3)



## Read FileStorage - Network input
pathToFile = r'.\previousLayerOutput.yml'
s = cv2.FileStorage()
s.open(pathToFile, cv2.FileStorage_READ)

# Get outputs to verify behaviour
inputScale1 = s.getNode('conv_81').mat()
inputScale2 = s.getNode('conv_93').mat()
inputScale3 = s.getNode('conv_105').mat()

ouputOfYoloScale1 = s.getNode('yolo_82').mat()
ouputOfYoloScale2 = s.getNode('yolo_94').mat()
ouputOfYoloScale3 = s.getNode('yolo_106').mat()

correctOutputs = [ouputOfYoloScale1,ouputOfYoloScale2,ouputOfYoloScale3]

inputs = [inputScale1,inputScale2,inputScale3]
outputs = []
for network_with_different_scales,imageInputScaled in zip(networks,inputs):
    network_with_different_scales.setInputsNames('permute_0')
    network_with_different_scales.setInput(imageInputScaled)
    outputs.append(network_with_different_scales.forward('yolo_0'))

strides = [32,16,8] # need to do it manually
for outputIdx,stride in zip(range(0,len(outputs)),strides):
    outputs[outputIdx][:,3] = outputs[outputIdx][:,3]/stride
    outputs[outputIdx][:,2] = outputs[outputIdx][:,2]/stride

for output, correctOutput in zip(outputs,correctOutputs):
    print(np.array_equal(output,correctOutput))

控制台输出：

True
True
True