如何将关键字参数应用于多处理池中的函数?

2021-06-14 23:45:41 发布

您现在位置:Python中文网/ 问答频道 /正文

我有一个函数,它只接受关键字参数,并希望在进程池中运行它。如何将iteable中的条目作为关键字参数传递给进程中的函数?你知道吗

import multiprocessing

greetees = ('Foo', 'Bar')

def greet(*, greetee):
    return f'Hello, {greetee}!'

我试过用多处理.map地址:

greetings = multiprocessing.Pool(2).map(greet, greetees)
for greeting in greetings:
    print(greeting)

但正如预期的那样,这会引发一个例外:

multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
TypeError: greet() takes 0 positional arguments but 1 was given
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bengt/Projekte/gitlab.com/PFASDR/PFASDR.Code.Main/pfasdr/neural/multi_pool_kwargs.py", line 10, in <module>
    greetings = multiprocessing.Pool(2).map(greet, greetees)
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 266, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 644, in get
    raise self._value
TypeError: greet() takes 0 positional arguments but 1 was given

如果我删除星号以不要求参数仅为关键字,则效果很好:

[...]
def greet(greetee):
    return f'Hello, {greetee}!'
[...]

输出:

Hello, Foo!
Hello, Bar!
2条回答
网友
1楼 ·

这里的解决方案是使用^{} or ^{}

greetings = list(
    multiprocessing.Pool(2).apply(greet, kwds={'greetee': greetees[i]})
    for i in range(len(greetees))
)
for greeting in greetings:
    print(greeting)

输出:

Hello, Foo!
Hello, Bar!
网友
2楼 ·

借助Mad Physicistthis QnA,可以使用functools.partial将仅关键字参数注入函数:

from functools import partial
greetings = []
for i in range(len(greetees)):
    kwargs = {'greetee': greetees[i]}
    greet_partial = partial(greet, **kwargs)
    greetings.append(multiprocessing.Pool(2).apply(greet_partial))

或具有较少的可变出血:

from functools import partial
greetings = [
    multiprocessing.Pool(2).apply(
        partial(greet, **{'greetee': greetees[i]})
    )
    for i in range(len(greetees))
]

相关问题