所以我尝试在dict的dict上使用multiprocessing Manager,这是我的初步尝试:
from multiprocessing import Process, Manager
def task(stat):
test['z'] += 1
test['y']['Y0'] += 5
if __name__ == '__main__':
test = Manager().dict({'x': {'X0': 10, 'X1': 20}, 'y': {'Y0': 0, 'Y1': 0}, 'z': 0})
p = Process(target=task, args=(test,))
p.start()
p.join()
print(test)
当然,当我运行这个时,输出并不是我期望的,z
会正确更新,而{
然后我在google上找到了一个解释here,显然嵌套的dict也必须是Manager().dict()
s,而不是普通的python dict(可能是从python3.6开始)。所以我做了以下几点:
from multiprocessing import Process, Manager
def task(stat):
test['z'] += 1
test['y']['Y0'] += 5
if __name__ == '__main__':
test = Manager().dict({'x': Manager().dict({'X0': 10, 'X1': 20}), 'y': Manager().dict({'Y0': 0, 'Y1': 0}), 'z': 0})
p = Process(target=task, args=(test,))
p.start()
p.join()
print(test)
print(test['y'])
但是它没有正常工作,我得到了这个无法解释的错误,为了清楚起见,我把它分为三个部分。第一部分对应于test['y']['Y0'] += 5
,而第二部分只是print(test)
,最后一部分是{
Process Process-4:
Traceback (most recent call last):
File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "shit.py", line 5, in task
test['y']['Y0'] += 5
File "<string>", line 2, in __getitem__
File "/usr/lib/python3.7/multiprocessing/managers.py", line 796, in _callmethod
kind, result = conn.recv()
File "/usr/lib/python3.7/multiprocessing/connection.py", line 251, in recv
return _ForkingPickler.loads(buf.getbuffer())
File "/usr/lib/python3.7/multiprocessing/managers.py", line 920, in RebuildProxy
return func(token, serializer, incref=incref, **kwds)
File "/usr/lib/python3.7/multiprocessing/managers.py", line 770, in __init__
self._incref()
File "/usr/lib/python3.7/multiprocessing/managers.py", line 824, in _incref
conn = self._Client(self._token.address, authkey=self._authkey)
File "/usr/lib/python3.7/multiprocessing/connection.py", line 492, in Client
c = SocketClient(address)
File "/usr/lib/python3.7/multiprocessing/connection.py", line 619, in SocketClient
s.connect(address)
FileNotFoundError: [Errno 2] No such file or directory
{'x': <DictProxy object, typeid 'dict' at 0x7f01de2c5860>, 'y': <DictProxy object, typeid 'dict' at 0x7f01de2c5898>, 'z': 1}
Traceback (most recent call last):
File "test.py", line 16, in <module>
print(test['y'])
File "<string>", line 2, in __getitem__
File "/usr/lib/python3.7/multiprocessing/managers.py", line 796, in _callmethod
kind, result = conn.recv()
File "/usr/lib/python3.7/multiprocessing/connection.py", line 251, in recv
return _ForkingPickler.loads(buf.getbuffer())
File "/usr/lib/python3.7/multiprocessing/managers.py", line 920, in RebuildProxy
return func(token, serializer, incref=incref, **kwds)
File "/usr/lib/python3.7/multiprocessing/managers.py", line 770, in __init__
self._incref()
File "/usr/lib/python3.7/multiprocessing/managers.py", line 824, in _incref
conn = self._Client(self._token.address, authkey=self._authkey)
File "/usr/lib/python3.7/multiprocessing/connection.py", line 492, in Client
c = SocketClient(address)
File "/usr/lib/python3.7/multiprocessing/connection.py", line 619, in SocketClient
s.connect(address)
FileNotFoundError: [Errno 2] No such file or directory
我不知道为什么会这样。很明显,内部指令被创建了(如输出的第二部分所示)。但由于某些原因,它们根本不能读或写!。为什么会这样?在
Extra:如果我通过python控制台(而不是脚本)运行相同的python代码,错误将从FileNotFoundError
更改为ConnectionRefusedError
。但是同样的追踪!在
当
Manager()
在Manager().dict()
中时,你每次都在启动一个新的管理器进程,所以你实际上是在嵌套管理器(就像标题所说的那样),而这并不是它应该的方式。您需要做的是实例化一个管理器,然后在该管理器实例上创建字典:输出示例:
^{pr2}$您还需要使用
Manager.Lock
,以防您计划从多个进程修改管理器对象。在相关问题 更多 >
编程相关推荐