python的多处理抛出struct.error

2024-05-23 21:16:09 发布

您现在位置:Python中文网/ 问答频道 /正文

我正在运行以下多处理代码:

manager = mp.Manager()
returns = manager.dict()
state = manager.dict()
pool = manager.Pool(processes=workers)
results = pool.starmap_async(extract_traces, [(
        v,
        path,
        masks[v],
        state,
        returns,
        max_frames,
        False
    ) for v in videos
])

在我增加masks的大小之前,它工作得非常好,这是numpy数组。 然后starmap_异步方法抛出以下错误:

.../signal_extraction.py", line 197, in parallel_extract_traces_from_videos                                                                    
) for v in videos                                                                                                                                                                                       
File "<string>", line 2, in starmap_async                                                                                                                                                                 
File "/home/eule/anaconda3/envs/detector/lib/python3.7/multiprocessing/managers.py", line 
795, in _callmethod
conn.send((self._id, methodname, args, kwds))
File "/home/eule/anaconda3/envs/detector/lib/python3.7/multiprocessing/connection.py", line 
206, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "/home/eule/anaconda3/envs/detector/lib/python3.7/multiprocessing/connection.py", line 
393, in _send_bytes
header = struct.pack("!i", n)
struct.error: 'i' format requires -2147483648 <= number <= 2147483647

我知道这与32位范围之外的内容有关,但我在任何地方都找不到解决方案

另外,当我从一个简单的for循环运行extract_traces方法时,即使使用大的masks数组,它也可以正常工作。 对于我正在超越的多处理过程,是否存在某种限制


Tags: inpysendhomeforlinemanagerextract