Python 子进程：命令退出时的回调如何设置

Question

我现在正在使用 subprocess.Popen(cmd, shell=TRUE) 启动一个程序。

我对Python还比较陌生，但我感觉应该有一些API可以让我做类似下面的事情：

subprocess.Popen(cmd, shell=TRUE,  postexec_fn=function_to_call_on_exit)

我这样做是为了让 function_to_call_on_exit 能够根据命令是否结束来执行一些操作（比如统计当前正在运行的外部进程数量）。

我想我可以很简单地把subprocess封装成一个类，结合线程和 Popen.wait() 方法，但我还没有在Python中使用过线程，而且这看起来可能是一个常见的需求，所以我想先找找是否有现成的API。

提前谢谢你们 :)

Answer 1

我修改了Daniel G的回答，把subprocess.Popen的args和kwargs直接传进去，而不是放在一个单独的元组或列表里，因为我想在使用subprocess.Popen的时候用关键字参数。

在我的情况下，我有一个方法postExec()，我想在执行subprocess.Popen('exe', cwd=WORKING_DIR)之后运行这个方法。

用下面的代码，我只需要写成popenAndCall(postExec, 'exe', cwd=WORKING_DIR)就可以了。

import threading
import subprocess

def popenAndCall(onExit, *popenArgs, **popenKWArgs):
    """
    Runs a subprocess.Popen, and then calls the function onExit when the
    subprocess completes.

    Use it exactly the way you'd normally use subprocess.Popen, except include a
    callable to execute as the first argument. onExit is a callable object, and
    *popenArgs and **popenKWArgs are simply passed up to subprocess.Popen.
    """
    def runInThread(onExit, popenArgs, popenKWArgs):
        proc = subprocess.Popen(*popenArgs, **popenKWArgs)
        proc.wait()
        onExit()
        return

    thread = threading.Thread(target=runInThread,
                              args=(onExit, popenArgs, popenKWArgs))
    thread.start()

    return thread # returns immediately after the thread starts

Answer 2

在Python 3.2中，有一个叫做 concurrent.futures 的模块（如果你用的是老版本的Python，低于3.2，可以通过 pip install futures 来安装）：

pool = Pool(max_workers=1)
f = pool.submit(subprocess.call, "sleep 2; echo done", shell=True)
f.add_done_callback(callback)

当你使用 f.add_done_callback() 时，回调函数会在调用这个函数的同一个进程中被执行。

完整程序

import logging
import subprocess
# to install run `pip install futures` on Python <3.2
from concurrent.futures import ThreadPoolExecutor as Pool

info = logging.getLogger(__name__).info

def callback(future):
    if future.exception() is not None:
        info("got exception: %s" % future.exception())
    else:
        info("process returned %d" % future.result())

def main():
    logging.basicConfig(
        level=logging.INFO,
        format=("%(relativeCreated)04d %(process)05d %(threadName)-10s "
                "%(levelname)-5s %(msg)s"))

    # wait for the process completion asynchronously
    info("begin waiting")
    pool = Pool(max_workers=1)
    f = pool.submit(subprocess.call, "sleep 2; echo done", shell=True)
    f.add_done_callback(callback)
    pool.shutdown(wait=False) # no .submit() calls after that point
    info("continue waiting asynchronously")

if __name__=="__main__":
    main()

输出结果

$ python . && python3 .
0013 05382 MainThread INFO  begin waiting
0021 05382 MainThread INFO  continue waiting asynchronously
done
2025 05382 Thread-1   INFO  process returned 0
0007 05402 MainThread INFO  begin waiting
0014 05402 MainThread INFO  continue waiting asynchronously
done
2018 05402 Thread-1   INFO  process returned 0

Answer 3

你说得对，确实没有一个很好的接口来处理这个问题。你第二点说得也对，设计一个可以用线程来完成这个任务的函数其实非常简单。

import threading
import subprocess

def popen_and_call(on_exit, popen_args):
    """
    Runs the given args in a subprocess.Popen, and then calls the function
    on_exit when the subprocess completes.
    on_exit is a callable object, and popen_args is a list/tuple of args that 
    would give to subprocess.Popen.
    """
    def run_in_thread(on_exit, popen_args):
        proc = subprocess.Popen(*popen_args)
        proc.wait()
        on_exit()
        return
    thread = threading.Thread(target=run_in_thread, args=(on_exit, popen_args))
    thread.start()
    # returns immediately after the thread starts
    return thread

在Python中使用线程也很简单，但要注意，如果on_exit()这个函数计算量很大，你就应该考虑用多进程来处理，这样可以避免全局解释器锁（GIL）让你的程序变慢。其实这很简单，你只需要把所有调用threading.Thread的地方换成multiprocessing.Process就可以了，因为它们的使用方法（API）几乎是一样的。