均值交并比函数评估两幅图像相似性问题

Question

我被要求使用OpenCV的Python库开发一个图像处理流程，目的是从一组植物图片中分割出花朵，并将处理后的图片与真实的标准图片进行对比评估。然而，每当我遇到那些花朵边缘碰到图片边界的真实标准图片时，评估结果就会出错，产生非常小的相似度分数。有没有什么解决办法呢？

评估的代码

# Function to find the largest contour which is assumed to be the flower
def find_largest_contour(binary_image):
    # Find contours from the binary image
    contours, _ = cv2.findContours(binary_image, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

    if contours:
        return max(contours, key=cv2.contourArea)
    else:
        return None

# Function to calculate the Intersection over Union
def calculate_iou(contourA, contourB, shape):
    maskA = np.zeros(shape, dtype=np.uint8)
    maskB = np.zeros(shape, dtype=np.uint8)
    cv2.drawContours(maskA, [contourA], -1, color=255, thickness=cv2.FILLED)
    cv2.drawContours(maskB, [contourB], -1, color=255, thickness=cv2.FILLED)
    intersection = np.logical_and(maskA, maskB)
    union = np.logical_or(maskA, maskB)
    iou_score = np.sum(intersection) / np.sum(union)
    return iou_score

# Function to display similarity percentage based on IoU
def display_similarity(image_name, iou_score):
    similarity_percentage = round(iou_score, 2)
    print(f"Similarity for {image_name}: {similarity_percentage}%")

# Apply the processing and calculate IoU for each image
ious = []
for input_path, ground_truth_path in zip(image_paths, ground_truth_image_paths):
    image_name = os.path.basename(input_path)
    original_image = cv2.imread(input_path)  # Read the original image again for visualization
    processed_image = process_image(input_path, image_name)
    show_binary_image(processed_image, window_name=f"Binary: {image_name}")
    ground_truth_image = cv2.imread(ground_truth_path, cv2.IMREAD_GRAYSCALE)
    show_binary_image(ground_truth_image, window_name=f"Binary: {image_name}")

    if processed_image.shape != ground_truth_image.shape:
        ground_truth_image = cv2.resize(ground_truth_image, (processed_image.shape[1], processed_image.shape[0]))

    # Find largest contours
    contour_processed = find_largest_contour(processed_image)
    contours_ground_truth = process_red_edges(ground_truth_path)  # Fix: Pass path instead of image

# Find the largest contour among the contours found
contour_ground_truth = max(contours_ground_truth, key=cv2.contourArea)

# Calculate IoU
iou_score = calculate_iou(contour_processed, contour_ground_truth, ground_truth_image.shape) * 100
ious.append(iou_score)

从处理流程中生成的图片，用来与真实标准图片进行比较

真实标准图片

从上面的例子可以看到，真实标准图片中花朵的顶部和底部边缘正好碰到图片的边界，而当我运行代码时，得到的相似度只有1.58%。而其他那些边缘没有碰到图片边界的图片，相似度都超过了90%。

Answer 1

这并不是简单的“交集与并集”。你需要关注的是，模型是否把所有的正样本（真实的正例）都标记为正样本，负样本也是同样的道理。所以这就需要用“交集与真实标签”来衡量。

GT = cv.imread("GT.png", cv.IMREAD_GRAYSCALE)
GT_negative = cv.inRange(GT, 100, 125) # lighter gray
GT_positive = cv.inRange(GT, 25, 50) # darker gray
# GT_dontcare = cv.inRange(GT, 0, 25) # black

subj = cv.imread("subject.png", cv.IMREAD_GRAYSCALE)
(_, subj) = cv.threshold(subj, 128, 255, cv.THRESH_BINARY)

def intersection_over_GT(GT, test):
    intersection = cv.bitwise_and(GT, test)
    return cv.countNonZero(intersection) / cv.countNonZero(GT)

positive_ratio = intersection_over_GT(GT_positive, subj) # 0.99175
negative_ratio = intersection_over_GT(GT_negative, cv.bitwise_not(subj)) # 0.99997

所以，得分几乎完美。之所以不是完全完美，部分原因是你给我们的截图边缘有一些杂物。

所有这些遮罩操作其实也可以用普通的numpy和布尔数组来完成，而不是用包含0和255的uint8类型。