我可以在python中创建一个共享的多数组或列表对象来进行多处理吗？

Question

我需要使多维数组或列表的共享对象对其他进程可用。有没有办法创造它，因为我所看到的是不可能的。我试过：

from multiprocessing import Process, Value, Array
arr = Array('i', range(10))
arr[:]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
arr[2]=[12,43]
TypeError: an integer is required

我听说numpy数组可以是多数组和共享对象，如果上面不可能有人告诉我如何使numpy数组成为共享对象？？

Answer 1

为什么不创建一个Arrays的列表？

 arrays = [Array('i', range(10))] * 10

Answer 2

要使numpy数组成为共享对象（full example）：

import ctypes as c
import numpy as np
import multiprocessing as mp

n, m = 2, 3
mp_arr = mp.Array(c.c_double, n*m) # shared, can be used from multiple processes
# then in each new process create a new numpy array using:
arr = np.frombuffer(mp_arr.get_obj()) # mp_arr and arr share the same memory
# make it two-dimensional
b = arr.reshape((n,m)) # b and arr share the same memory

如果您不需要shared（如“share the same memory”（共享同一内存）对象，并且只需要一个可以从多个进程使用的对象就足够了，那么您可以使用multiprocessing.Manager：

from multiprocessing import Process, Manager

def f(L):
    row = L[0] # take the 1st row
    row.append(10) # change it
    L[0] = row #NOTE: important: copy the row back (otherwise parent
               #process won't see the changes)

if __name__ == '__main__':
    manager = Manager()

    lst = manager.list()
    lst.append([1])
    lst.append([2, 3])
    print(lst) # before: [[1], [2, 3]]

    p = Process(target=f, args=(lst,))
    p.start()
    p.join()

    print(lst) # after: [[1, 10], [2, 3]]

来自the docs：

Server process managers are more flexible than using shared memory objects because they can be made to support arbitrary object types. Also, a single manager can be shared by processes on different computers over a network. They are, however, slower than using shared memory.