如何等待RxPy并行线程完成

2024-06-09 18:58:51 发布

您现在位置:Python中文网/ 问答频道 /正文

基于这个excellent SO answer我可以在RxPy中并行处理多个任务,我的问题是如何等待它们全部完成?我知道使用线程我可以做.join(),但在Rx调度器中似乎没有这样的选择。.to_blocking()也没有帮助,主线程在所有通知激发和完整处理程序被调用之前完成。下面是一个例子:

from __future__ import print_function
import os, sys
import time
import random
from rx import Observable
from rx.core import Scheduler
from threading import current_thread

def printthread(val):
    print("{}, thread: {}".format(val, current_thread().name))

def intense_calculation(value):
    printthread("calc {}".format(value))
    time.sleep(random.randint(5, 20) * .1)
    return value

if __name__ == "__main__":
    Observable.range(1, 3) \
        .select_many(lambda i: Observable.start(lambda: intense_calculation(i), scheduler=Scheduler.timeout)) \
        .observe_on(Scheduler.event_loop) \
        .subscribe(
            on_next=lambda x: printthread("on_next: {}".format(x)),
            on_completed=lambda: printthread("on_completed"),
            on_error=lambda err: printthread("on_error: {}".format(err)))

    printthread("\nAll done")
    # time.sleep(2)

预期产量

^{pr2}$

实际产量

calc 1, thread: Thread-1
calc 2, thread: Thread-2
calc 3, thread: Thread-3

All done, thread: MainThread

如果取消对睡眠调用的注释,则实际输出

calc 1, thread: Thread-1
calc 2, thread: Thread-2
calc 3, thread: Thread-3

All done, thread: MainThread
on_next: 2, thread: Thread-4
on_next: 3, thread: Thread-4
on_next: 1, thread: Thread-4
on_completed, thread: Thread-4

Tags: lambdafromimportformattimevalueoncalc
2条回答

在此处发布完整的解决方案:

from __future__ import print_function
import os, sys
import time
import random
from rx import Observable
from rx.core import Scheduler
from threading import current_thread
from rx.concurrency import ThreadPoolScheduler

def printthread(val):
    print("{}, thread: {}".format(val, current_thread().name))

def intense_calculation(value):
    printthread("calc {}".format(value))
    time.sleep(random.randint(5, 20) * .1)
    return value

if __name__ == "__main__":
    scheduler = ThreadPoolScheduler(4)

    Observable.range(1, 3) \
        .select_many(lambda i: Observable.start(lambda: intense_calculation(i), scheduler=scheduler)) \
        .observe_on(Scheduler.event_loop) \
        .subscribe(
            on_next=lambda x: printthread("on_next: {}".format(x)),
            on_completed=lambda: printthread("on_completed"),
            on_error=lambda err: printthread("on_error: {}".format(err)))

    printthread("\nAll done")
    scheduler.executor.shutdown()
    # time.sleep(2)

对于ThreadPoolScheduler,您可以:

  1. 调度程序=线程池调度程序(池大小)
  2. 并行调用。在
  3. scheduler.executor.shutdown()

然后,你就可以得到所有的结果了。在

相关问题 更多 >