我正在处理一套冷冻设备。我试图在字典“output”中找到每个冻结集的最小集。我有7万个frozenset,所以我正在制作这个frozenset字典的一大块,并将这个任务并行化。当我试图把这本字典作为输入传递给我的函数时,只有密钥被发送,所以我得到了错误,有人能帮我找出其中的错误。
output => {frozenset({'rfid', 'zone'}): 0, frozenset({'zone'}): 0, frozenset({'zone', 'time'}): 0}
def reduce(prob,result,output):
print(output)
for k in output.keys():
#Function to do something
def reducer(prob,result,output):
print(output)
p = Pool(4) #number of processes = number of CPUs
func2 = partial(reduce,prob,result)
reduced_values= p.map( func2,output,chunksize=4)
p.close() # no more tasks
p.join() # wrap up current tasks
return reduced_values
if __name__ == '__main__':
final = reducer(prob,result,output)
{frozenset({'rfid', 'zone'}): 0, frozenset({'zone'}): 0, frozenset({'zone', 'time'}): 0}
frozenset({'rfid', 'zone'})
Error : AttributeError: 'frozenset' object has no attribute 'keys'
from multiprocessing import Pool
from functools import partial
import itertools
output = {frozenset({'rfid', 'zone'}): 0, frozenset({'zone'}): 0, frozenset({'zone', 'time'}): 0}
prob = {'3': 0.3, '1': 0.15, '2': 0.5, '4': 0.05}
result = {'2': {frozenset({'time', 'zone'}), frozenset({'time', 'rfid'})}, '3': {frozenset({'time', 'rfid'}), frozenset({'rfid', 'zone'})}}
def reduce(prob,result,output):
print(output)
for k in output.keys():
for ky,values in result.items():
if any(k>=l for l in values):
output[k] += sum((j for i,j in prob.items() if i == ky))
return output
def reducer(prob,result,output):
print(output)
p = Pool(4) #number of processes = number of CPUs
func2 = partial(reduce,prob,result)
reduced_values= p.map( func2,output,chunksize=4)
p.close() # no more tasks
p.join() # wrap up current tasks
return reduced_values
if __name__ == '__main__':
final = reducer(prob,result,output)
{frozenset({'zone', 'rfid'}): 0, frozenset({'zone'}): 0, frozenset({'time', 'zone'}): 0}
for k in output.keys():
AttributeError: 'frozenset' object has no attribute 'keys'
frozenset({'zone', 'rfid'})
{frozenset({'zone', 'time'}): 0, frozenset({'zone', 'rfid'}): 0, frozenset({'zone'}): 0}
frozenset({'zone', 'time'})
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "F:\Python34\lib\multiprocessing\pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "F:\Python34\lib\multiprocessing\pool.py", line 44, in mapstar
return list(map(*args))
File "C:\Users\Dell\workspace\key_mining\src\variable.py", line 16, in reduce
for k in output.keys():
AttributeError: 'frozenset' object has no attribute 'keys'
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\***\variable.py", line 33, in <module>
final = reducer(prob,result,output)
File "C:\***\variable.py", line 27, in reducer
reduced_values= p.map( func2,output,chunksize=4)
File "F:\Python34\lib\multiprocessing\pool.py", line 260, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "F:\Python34\lib\multiprocessing\pool.py", line 599, in get
raise self._value
AttributeError: 'frozenset' object has no attribute 'keys'
问题是您正在将
dict
对象传递给map
。当map
遍历output
中的项时,它将执行以下操作:所以每次调用
func2
时,包含在output
中的所有内容都是字典中的一个键(afrozenset
)。根据您上面的评论,似乎您想将整个字典传递给
func2
,但是如果您这样做了,您实际上根本没有并行地执行任何操作。我想也许你觉得这样做将导致
output
字典被分成四个字典,每个块被传递到func2
的实例。但事实并非如此。相反,字典中的每个键都是单独发送的。chunksize
只是用来告诉pool
一次通过进程间通信向每个子进程发送output
中的多少个元素。它只用于内部目的;不管您使用什么chunksize
,func2
将只使用单个元素output
调用。如果你真的想传递一段话,你需要这样做:
它将把
(key, value)
元组的列表从output
dict传递到func2
。然后,在func2
中,您可以将列表转换回dict:相关问题 更多 >
编程相关推荐