如何在opencv中对图像及其注释进行图像配准？

Question

我想把一个源图像上的多边形标签转移到一个目标图像上。目标图像其实就是源图像，只是稍微移动了一下。我找到了一段代码，可以用来把源图像和目标图像对齐。把这段代码写成一个函数后，变成了：

import numpy as np
import cv2

def register_images(
        align: np.ndarray,
        reference: np.ndarray,
):
    """
    Registers two RGB images with each other.

    Args:
        align: Image to be aligned. 
        reference: Reference image to be used for alignment.

    Returns:
        Registered image and transformation matrix.
    """
    # Convert to grayscale if needed
    _align = align.copy()
    _reference = reference.copy()
    if _align.shape[-1] == 3:
        _align = cv2.cvtColor(_align, cv2.COLOR_RGB2GRAY)
    if _reference.shape[-1] == 3:
        _reference = cv2.cvtColor(_reference, cv2.COLOR_RGB2GRAY)

    height, width = _reference.shape

    # Create ORB detector with 5000 features
    orb_detector = cv2.ORB_create(500)

    # Find the keypoint and descriptors
    # The first arg is the image, second arg is the mask (not required in this case).
    kp1, d1 = orb_detector.detectAndCompute(_align, None)
    kp2, d2 = orb_detector.detectAndCompute(_reference, None)

    # Match features between the two images
    # We create a Brute Force matcher with Hamming distance as measurement mode.
    matcher = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True)

    # Match the two sets of descriptors
    matches = list(matcher.match(d1, d2))

    # Sort matches on the basis of their Hamming distance and select the top 90 % matches forward
    matches.sort(key=lambda x: x.distance)
    matches = matches[:int(len(matches) * 0.9)]
    no_of_matches = len(matches)

    # Define empty matrices of shape no_of_matches * 2
    p1 = np.zeros((no_of_matches, 2))
    p2 = np.zeros((no_of_matches, 2))
    for i in range(len(matches)):
        p1[i, :] = kp1[matches[i].queryIdx].pt
        p2[i, :] = kp2[matches[i].trainIdx].pt

    # Find the homography matrix and use it to transform the colored image wrt the reference
    homography, mask = cv2.findHomography(p1, p2, cv2.RANSAC)
    transformed_img = cv2.warpPerspective(align, homography, (width, height))

    return transformed_img, homography

现在，我可以获取到变换后的图像和用来对齐这两张图像的变换矩阵。不过，我不太明白的是，怎么把同样的变换应用到用来标注图像的多边形和边界框上。

具体来说，标注是用COCO格式的，这意味着你可以这样访问坐标：

x0, y0, width, height = bounding_box

而标注是一个多边形坐标的列表：

segmentations = [poly1, poly2, poly3, ...]  # segmentations are a list of polygons
for poly in segmentations:
    x_coords = poly[0::2]  # x coordinates are integer values on the even index in the poly list
    y_coords = poly[1::2]  # y coordinates are integer values on the odd index in the poly list

一旦我获取到x和y坐标，怎么才能把变换矩阵应用上去呢？

图像处理计算机视觉 opencv 边界框变换矩阵图像配准多边形标注 COCO格式

如何在opencv中对图像及其注释进行图像配准？

1 个回答

多边形

盒子

撰写回答