我正在运行以下多处理代码:
manager = mp.Manager()
returns = manager.dict()
state = manager.dict()
pool = manager.Pool(processes=workers)
results = pool.starmap_async(extract_traces, [(
v,
path,
masks[v],
state,
returns,
max_frames,
False
) for v in videos
])
在我增加masks
的大小之前,它工作得非常好,这是numpy数组。
然后starmap_异步方法抛出以下错误:
.../signal_extraction.py", line 197, in parallel_extract_traces_from_videos
) for v in videos
File "<string>", line 2, in starmap_async
File "/home/eule/anaconda3/envs/detector/lib/python3.7/multiprocessing/managers.py", line
795, in _callmethod
conn.send((self._id, methodname, args, kwds))
File "/home/eule/anaconda3/envs/detector/lib/python3.7/multiprocessing/connection.py", line
206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/home/eule/anaconda3/envs/detector/lib/python3.7/multiprocessing/connection.py", line
393, in _send_bytes
header = struct.pack("!i", n)
struct.error: 'i' format requires -2147483648 <= number <= 2147483647
我知道这与32位范围之外的内容有关,但我在任何地方都找不到解决方案
另外,当我从一个简单的for循环运行extract_traces
方法时,即使使用大的masks
数组,它也可以正常工作。
对于我正在超越的多处理过程,是否存在某种限制
目前没有回答
相关问题 更多 >
编程相关推荐