Python高级Numpy切片和索引到矢量化方法

2024-03-28 23:50:09 发布

您现在位置:Python中文网/ 问答频道 /正文

我正在尝试将此方法矢量化,我正在使用此方法进行ML图像增强:

def random_erase_from_image(images, random_erasing, image_size):
#could probably be vectorized to speed up
to_return = images
for t in range(images.shape[0]):
    if np.random.randint(0, 2) == 0:#do random erasing
        x_erase_size = np.random.randint(0, random_erasing)
        y_erase_size = np.random.randint(0, random_erasing)

        x_erase_start = np.random.randint(0, image_size-x_erase_size)
        y_erase_start = np.random.randint(0, image_size-y_erase_size)

        shape = to_return[t, y_erase_start:y_erase_start+y_erase_size, x_erase_start:x_erase_start+x_erase_size, :].shape

        print(shape)

        to_return[t, y_erase_start:y_erase_start+y_erase_size, x_erase_start:x_erase_start+x_erase_size, :] = (np.random.random(shape) * 255).astype('uint8')

return images

这是我所能做到的,但我不知道如何正确地切割

def random_erase_vec(images, random_erasing, image_size):
    #could probably be vectorized to speed up
    to_return = images
    mask = np.random.choice(a=[False, True], size=images.shape[0], p=[.5, .5])  
    x_erase_size = np.random.randint(0, random_erasing, size=images.shape[0])
    y_erase_size = np.random.randint(0, random_erasing, size=images.shape[0])

    x_erase_start = np.random.randint(0, image_size-x_erase_size, size=images.shape[0])
    y_erase_start = np.random.randint(0, image_size-y_erase_size, size=images.shape[0])

    random_values = (np.random.random((images.shape))* 255).astype('uint8')

    to_return[:, [y_erase_start[:]]:[y_erase_start[:]+y_erase_size[:]], [x_erase_start[:]]:[x_erase_start[:]+x_erase_size[:]], :] = random_values[:, [y_erase_start[:]]:[y_erase_start[:]+y_erase_size[:]], [x_erase_start[:]]:[x_erase_start[:]+x_erase_size[:]], :]

    return images

我试图避免重塑,但如果这是需要的,我想它会做的。让我知道你能想到的任何加速原始方法的方法

我在切片线上遇到以下错误: “切片索引必须为整数或无,或具有索引方法”

我还想遮罩,所以不是所有的图像都是随机擦除的,但我想在切片部分完成后这样做

谢谢你的帮助

编辑:示例输入:

图像:大小为[#的图像、高度(32)、宽度(32)、通道(3)的numpy阵列

random_Erasting(随机擦除):名称不好,但要擦除的任意维图像的最大大小。当前设置为20

image_size:现在我想可能是从images数组中得到的,但是清理还不是优先事项


Tags: to方法图像imagesizereturnnp切片
1条回答
网友
1楼 · 发布于 2024-03-28 23:50:09

我稍微整理了一下你的函数,并尝试对它进行部分矢量化,但由于你想改变随机补丁的大小,这有点复杂

import numpy as np

def random_erase(images, random_erasing):
    n, *image_size, n_channels = images.shape
    to_return = images.copy()
    
    for t in range(n):
        x_erase_size = np.random.randint(1, random_erasing)
        y_erase_size = np.random.randint(1, random_erasing)

        x_erase_start = np.random.randint(1, image_size[0]-x_erase_size)
        y_erase_start = np.random.randint(1, image_size[1]-y_erase_size)

        x_erase_end = x_erase_start + x_erase_size
        y_erase_end = y_erase_start + y_erase_size
        
        shape = (x_erase_size, y_erase_size, n_channels)
        random_image = np.random.randint(0, 255, size=shape, dtype=np.uint8)
        to_return[t, x_erase_start:x_erase_end, y_erase_start:y_erase_end, :] = random_image
        
    return to_return


def random_erase_vec(images, random_erasing):
    n, *image_size, n_channels = images.shape

    to_return = images.copy()
    x_erase_size = np.random.randint(1, random_erasing, size=n)
    y_erase_size = np.random.randint(1, random_erasing, size=n)

    x_erase_start = np.random.randint(1, image_size[0]-x_erase_size, size=n)
    y_erase_start = np.random.randint(1, image_size[1]-y_erase_size, size=n)

    x_erase_end = x_erase_start + x_erase_size
    y_erase_end = y_erase_start + y_erase_size

    shapes = np.vstack((x_erase_size, y_erase_size))
    sizes = np.prod(shapes, axis=0)
    sizes_cs = np.cumsum(np.concatenate([[0], sizes]))
    total_size = np.sum(sizes)

    idx = np.empty((total_size, 3), dtype=int)
    for i in range(n):
        idx_x, idx_y = np.meshgrid(np.arange(x_erase_start[i], x_erase_end[i]), 
                                   np.arange(y_erase_start[i], y_erase_end[i]))
        idx[sizes_cs[i]:sizes_cs[i+1], 0] = i
        idx[sizes_cs[i]:sizes_cs[i+1], 1] = idx_x.flatten()
        idx[sizes_cs[i]:sizes_cs[i+1], 2] = idx_y.flatten()

    random_values = np.random.randint(0, 255, size=(total_size, n_channels), dtype=np.uint8)
    to_return[idx[:, 0], idx[:, 1], idx[:, 2], :] = random_values

    return to_return
# images = np.random.random((1000, 100, 100, 1))
# random_erasing = 32
a = random_erase(images, random_erasing)
b = random_erase_vec(images, random_erasing)
# a: 0.059 s
# b: 0.049 s

速度并不惊人(大约20%),因为这是在ML中进行预处理,所以最好的办法是使用更多的工作人员来准备数据,以便充分利用GPU

编辑: 是的,我使用.copy()来确保参数不会在例程之外发生变异。如果你愿意,你可以忽略这个

我在[tensorflow文档]中使用术语worker:(https://www.tensorflow.org/api_docs/python/tf/keras/Model

workers Used for generator or keras.utils.Sequence input only. Maximum number of processes to spin up when using process-based threading. If unspecified, workers will default to 1. If 0, will execute the generator on the main thread.

use_multiprocessing Used for generator or keras.utils.Sequence input only. If True, use process-based threading. If unspecified, use_multiprocessing will default to False. Note that because this implementation relies on multiprocessing, you should not pass non-picklable arguments to the generator as they can't be passed easily to children processes.

相关问题 更多 >