在并行处理中如何使用期货

Question

我有以下代码。原则上，我想迭代long_list 并应用函数procedure()，该函数将一个值和另一个列表short_list作为参数：

def procedure(par1, the_short_list):
    """
    Description of procedure
    """
    o1, o2 = par1.split(" ")
    out = []
    for sl in the_short_list:
        output = sl * int(o1) * int(o2)
        out.append(output)
    return out



long_list = [str(v) + " " + str(w) for v, w in enumerate(range(0,10))]
short_list = range(10,15)

#--------------------------------------------------
# Sequential
#--------------------------------------------------
for i in long_list:
    out = procedure(i, short_list)
    print  (out)

它产生以下结果：

[0, 0, 0, 0, 0]
[10, 11, 12, 13, 14]
[40, 44, 48, 52, 56]
[90, 99, 108, 117, 126]
[160, 176, 192, 208, 224]
[250, 275, 300, 325, 350]
[360, 396, 432, 468, 504]
[490, 539, 588, 637, 686]
[640, 704, 768, 832, 896]
[810, 891, 972, 1053, 1134]

现在我要做的是通过打破long_list并行运行procedure()并最终收集结果来并行化进程。你知道吗

我试过这个代码：

#--------------------------------------------------
# Parallel
#--------------------------------------------------
import concurrent.futures
with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
    future = executor.submit(procedure, long_list, short_list)
    print(future.result())

但它给出了这个错误。你知道吗

Traceback (most recent call last):
  File "test.py", line 33, in <module>
    print(future.result())
  File "/home/ubuntu/anaconda2/lib/python2.7/site-packages/concurrent/futures/_base.py", line 462, in result
    return self.__get_result()
  File "/home/ubuntu/anaconda2/lib/python2.7/site-packages/concurrent/futures/thread.py", line 63, in run
    result = self.fn(*self.args, **self.kwargs)
  File "test.py", line 6, in procedure
    o1, o2 = par1.split(" ")
AttributeError: 'list' object has no attribute 'split'

正确的方法是什么？你知道吗

我希望输出与顺序版本相同。当应用更大的long_list时，它会运行得更快。你知道吗

Answer 1

这个应该适合你：

import concurrent.futures

futures = []
with concurrent.futures.ThreadPoolExecutor() as executor:
    for entry in long_list: 
         futures.append(executor.submit(procedure, entry, short_list))
for future in futures:
    print(future.result())