是否可以将此numpy函数转换为tensorflow？

Question

我有一个函数，它接受[32,32,3]张量，并输出[256256,3]张量

具体来说，该函数将较小的数组解释为.svg文件，并使用this algorithm将其“呈现”为256x256数组作为画布

有关我为什么要这样做的解释，请参见This question

函数的行为完全符合预期，直到我尝试将其包含在GAN的训练循环中。我看到的当前错误是：

NotImplementedError: Cannot convert a symbolic Tensor (mul:0) to a numpy array.

对于类似错误的许多其他答案似乎归结为“您需要使用tensorflow而不是numpy重新编写函数”

这是使用numpy的工作代码-是否可以重新编写它以专门使用tensorflow函数

def convert_to_bitmap(input_tensor, target, j):
    #implied conversion to nparray - the tensorflow docs seem to indicate this is okay, but the error is thrown here when training
    array = input_tensor
    outputArray = target
    output = target

    for i in range(32):
        col = float(array[i,0,j])
        if ((float(array[i,0,0]))+(float(array[i,0,1]))+(float(array[i,0,2]))/3)< 0:
            continue  

        #slice only the red channel from the i line, multiply by 255
        red_array = array[i,:,0]*255

        #slice only the green channel, multiply by 255
        green_array = array[i,:,1]*255

        #combine and flatten them
        combined_array = np.dstack((red_array, green_array)).flatten()

        #remove the first two and last two indices of the combined array
        index = [0,1,62,63]
        clipped_array = np.delete(combined_array,index)

        #filter array to remove values less than 0
        filtered = clipped_array > 0
        filtered_array = clipped_array[filtered]

        #check array has an even number of values, delete the last index if it doesn't
        if len(filtered_array) % 2 == 0: 
            pass
        else:
            filtered_array = np.delete(filtered_array,-1)

        #convert into a set of tuples
        l = filtered_array.tolist()
        t = list(zip(l, l[1:] + l[:1]))

        if not t:
            continue

        output = fill_polygon(t, outputArray, col)

    return(output)

“填充多边形”功能是从“mahotas”库复制的：

def fill_polygon(polygon, canvas, color):
if not len(polygon):
    return

min_y = min(int(y) for y,x in polygon)
max_y = max(int(y) for y,x in polygon)
polygon = [(float(y),float(x)) for y,x in polygon]

if max_y < canvas.shape[0]:
    max_y += 1 

for y in range(min_y, max_y):
    nodes = []
    j = -1
    for i,p in enumerate(polygon):
        pj = polygon[j]
        if p[0] < y and pj[0] >= y or pj[0] < y and p[0] >= y:
            dy = pj[0] - p[0]
            if dy:
                nodes.append( (p[1] + (y-p[0])/(pj[0]-p[0])*(pj[1]-p[1])) )
            elif p[0] == y:
                nodes.append(p[1])

        j = i

    nodes.sort()

    for n,nn in zip(nodes[::2],nodes[1::2]):
        nn += 1
        canvas[y, int(n):int(nn)] = color

return(canvas)

注意：我并不是想让别人为我改变整个事情！有些函数非常明显（tf.stack而不是np.dstack），但有些函数我甚至不知道如何开始，比如上面fill_多边形函数的最后几行

Answer 1

是的，你确实可以做到这一点，你可以在名为tf.pyfunc的sth中使用python函数。它是一个python包装器，但与普通tensorflow相比速度非常慢。然而，tensorflow和Cuda的速度非常快，因为它们使用了vectorization之类的东西，这意味着你可以用非常快的数学张量运算重写很多循环

一般来说：

如果您想使用自定义代码作为自定义层，我建议您重新思考这些循环背后的代数，并尝试以某种方式表达它们。如果只是在开始培训之前进行预处理，则可以使用tensorflow，但在numpy和其他库中也可以使用tensorflow

对于您的主要问题：是的，这是可能的，但最好不要使用循环。Tensorflow有一个内置的循环优化器，但是你必须使用tf.while()，这就是一切（可能只是为了我）。我只是浏览了一下您的代码，但是看起来您应该能够使用标准的tensorflow词汇表很好地对其进行矢量化。如果你想要它快，我的意思是GPU支持在tensorflow中写所有的东西，但是不像50/50在tf.convert_to_tensor()中那样快，因为它会再次变慢。因为您可以在GPU和CPU、普通Python解释器和tensorflow低级API之间切换。希望我至少能帮你一点忙

Answer 2

该代码“有效”，因为它只使用tensorflow函数，并且允许模型在训练循环中使用时进行训练：

def convert_image (x):

    #split off the first column of the generator output, and store it for later (remove the 'colours' column)
colours_column = tf.slice(img_to_convert, tf.constant([0,0,0], dtype=tf.int32), tf.constant([32,1,3], dtype=tf.int32)) 

    #split off the rest of the data, only keeping R + G, and discarding B
image_data_red = tf.slice(img_to_convert, tf.constant([0,1,0], dtype=tf.int32), tf.constant([32,31,1], dtype=tf.int32))
image_data_green = tf.slice(img_to_convert, tf.constant([0,1,1], dtype=tf.int32), tf.constant([32, 31,1], dtype=tf.int32))

    #roll each row by 1 position, and make two more 2D tensors
rolled_red = tf.roll(image_data_red, shift=-1, axis=0)
rolled_green = tf.roll(image_data_green, shift=-1, axis=0)

    #remove all values where either the red OR green channels are 0
zeroes = tf.constant(0, dtype=tf.float32)

    #this is for the 'count_nonzero' command
boolean_red_data = tf.not_equal(image_data_red, zeroes)
boolean_green_data = tf.not_equal(image_data_green, zeroes)
initial_data_mask = tf.logical_and(boolean_red_data, boolean_green_data)

    #count non-zero values per row and flatten it
count = tf.math.count_nonzero(initial_data_mask, 1)
count_flat = tf.reshape(count, [-1])

flat_red = tf.reshape(image_data_red, [-1])
flat_green = tf.reshape(image_data_green, [-1])

boolean_red = tf.math.logical_not(tf.equal(flat_red, tf.zeros_like(flat_red)))
boolean_green = tf.math.logical_not(tf.equal(flat_green, tf.zeros_like(flat_red)))
mask = tf.logical_and(boolean_red, boolean_green)

flat_red_without_zero = tf.boolean_mask(flat_red, mask) 
flat_green_without_zero = tf.boolean_mask(flat_green, mask) 

    # create a ragged tensor
X0_ragged = tf.RaggedTensor.from_row_lengths(values=flat_red_without_zero, row_lengths=count_flat)
Y0_ragged = tf.RaggedTensor.from_row_lengths(values=flat_green_without_zero, row_lengths=count_flat)

    #do the same for the rolled version
rolled_data_mask = tf.roll(initial_data_mask, shift=-1, axis=1)

flat_rolled_red = tf.reshape(rolled_red, [-1])
flat_rolled_green = tf.reshape(rolled_green, [-1])


    #from SO "shift zeros to the end"
boolean_rolled_red = tf.math.logical_not(tf.equal(flat_rolled_red, tf.zeros_like(flat_rolled_red)))
boolean_rolled_green = tf.math.logical_not(tf.equal(flat_rolled_green, tf.zeros_like(flat_rolled_red)))
rolled_mask = tf.logical_and(boolean_rolled_red, boolean_rolled_green)

flat_rolled_red_without_zero = tf.boolean_mask(flat_rolled_red, rolled_mask) 
flat_rolled_green_without_zero = tf.boolean_mask(flat_rolled_green, rolled_mask) 

    # create a ragged tensor
X1_ragged = tf.RaggedTensor.from_row_lengths(values=flat_rolled_red_without_zero, row_lengths=count_flat)
Y1_ragged = tf.RaggedTensor.from_row_lengths(values=flat_rolled_green_without_zero, row_lengths=count_flat)

    #available outputs for future use are:
X0 = X0_ragged.to_tensor(default_value=0.)
Y0 = Y0_ragged.to_tensor(default_value=0.)
X1 = X1_ragged.to_tensor(default_value=0.)
Y1 = Y1_ragged.to_tensor(default_value=0.)

    #Example tensor cel (replace with (x))
P = tf.cast(x, dtype=tf.float32)

    #split out P.x and P.y, and fill a ragged tensor to the same shape as Rx

Px_value = tf.cast(x, dtype=tf.float32) - tf.cast((tf.math.floor(x/255)*255), dtype=tf.float32)
Py_value = tf.cast(tf.math.floor(x/255), dtype=tf.float32)

Px = tf.squeeze(tf.ones_like(X0)*Px_value)
Py = tf.squeeze(tf.ones_like(Y0)*Py_value)

    #for each pair of values (Y0, Y1, make a vector, and check to see if it crosses the y-value (Py) either up or down

a = tf.math.less(Y0, Py)
b = tf.math.greater_equal(Y1, Py)
c = tf.logical_and(a, b)

d = tf.math.greater_equal(Y0, Py)
e = tf.math.less(Y1, Py)
f = tf.logical_and(d, e)

g = tf.logical_or(c, f)

    #Makes boolean bitwise mask

    #calculate the intersection of the line with the y-value, assuming it intersects
    #P.x <= (G.x - R.x) * (P.y - R.y) / (G.y - R.y + R.x)   - use tf.divide_no_nan for safe divide

h = tf.math.less(Px,(tf.math.divide_no_nan(((X1-X0)*(Py-Y0)),(Y1-Y0+X0))))

    #combine using AND with the mask above

i = tf.logical_and(g,h)



    #tf.count_nonzero
    #reshape to make a column tensor with the same dimensions as the colours
    #divide by 2 using tf.floor_mod (returns remainder of division - any remainder means the value is odd, and hence the point is IN the polygon)


final_count = tf.cast((tf.math.count_nonzero(i, 1)), dtype=tf.int32)

twos = tf.ones_like(final_count, dtype=tf.int32)*tf.constant([2], dtype=tf.int32)

divide = tf.cast(tf.math.floormod(final_count, twos), dtype=tf.int32)

index = tf.cast(tf.range(0,32, delta=1), dtype=tf.int32)

clipped_index = divide*index

sort = tf.sort(clipped_index)

reverse = tf.reverse(sort, [-1])

value = tf.slice(reverse, [0], [1])

pair = tf.constant([0], dtype=tf.int32)

slice_tensor = tf.reshape(tf.stack([value, pair, pair], axis=0),[-1])

output_colour = tf.slice(colours_column, slice_tensor, [1,1,3])

return output_colour

这是使用tf.vectorize_map应用“convert image”函数的地方：

def convert_images(image_to_convert):

global img_to_convert

img_to_convert = image_to_convert

process_list = tf.reshape((tf.range(0,65536, delta=1, dtype=tf.int32)), [65536, 1])

output_line = tf.vectorized_map(convert_image, process_list)

output_line_squeezed = tf.squeeze(output_line)

output_reshape = (tf.reshape(output_line_squeezed, [256,256,3])/127.5)-1

output = tf.expand_dims(output_reshape, axis=0)

return output

不过，它的速度非常慢——它似乎没有使用GPU，而且看起来也是单线程的

我添加它作为对我自己问题的回答，因为显然完全可以在tensorflow中完成这个numpy函数-它可能不应该这样做