获取分离子流程的返回代码？

Question

我正在尝试为作业调度程序编写提交程序。由于我不知道作业何时到来以及作业将运行多长时间，我使用多处理为每个作业生成一个进程，并使用子进程和分离来处理下一个作业。同时，这也很有效，但是我想在作业完成后获得返回代码，这可能吗？我尝试了几个子流程变体，但是那些返回RC的变体在作业运行时阻塞了流程

#!/usr/bin/python3
# coding=utf-8
import time
import multiprocessing
import subprocess

JobsList = []

def SubmitJob(jobname):
    """ Submit the next requested job """
    print(f"Starting job {jobname}...")
    JobDir ="/home/xxxxx/Jobs/"
    JobMem = "{}{}.sh".format(JobDir, jobname)
    SysoutFile = "./Sysout/{}.out".format(jobname)
    fh = open(SysoutFile, 'w')
    kwargs = {}
    kwargs.update(start_new_session=True)
    p = subprocess.Popen(JobMem, shell = False, stdout = fh, **kwargs)
    pid = p.pid
    print(f"Job {jobname} pid {pid} submitted...")

def PrepareSubmit():
    """ Create and start one process per job """
    jobs = []

    for Job in JobsList:
        process = multiprocessing.Process(target=SubmitJob, 
                                          args=(Job,))
        jobs.append(process)
        JobsList.remove(Job)

    for j in jobs:
        j.start()

    for j in jobs:
        j.join()

    print("All jobs submitted...")

def main():
    """ Check queue for new job requests """
    number_of_lines = 0
    jobs_list = []

    while 1:
        job_queue = open("/home/xxxxx/Development/Python/#Projects/Scheduler/jobs.que", 'r')
        lines = job_queue.readlines()

        if len(lines) > number_of_lines:
            jobs_list.append(lines[len(lines)-1])
            NewJob = lines[len(lines)-1][:-1]
            JobsList.append(NewJob)
            PrepareSubmit()
            number_of_lines = number_of_lines+1

        time.sleep(1)

if __name__ == "__main__":

    main()

main（）中的while循环仅用于测试目的

有谁能告诉我这是否可能，以及如何做到？提前谢谢

---

这是一个代码，它给了我一个返回代码，但在上一个作业完成之前不会发送作业。因此，如果我有一个长期运行的作业，它会延迟运行作业的过程，我称之为阻塞

def Submit(job):
    """ Submit the next requested job """
    print(f"Starting job {job}...")
    JobDir ="/home/uwe/Jobs/"
    JobMem = "{}{}.sh".format(JobDir, job)
    SysoutFile = "./Sysout/{}.out".format(job)
    fh = open(SysoutFile, 'w')
    kwargs = {}
    kwargs.update(start_new_session=True)
    p = subprocess.Popen(JobMem, shell = False, stdout = fh, **kwargs)
    pid = p.pid

    while p.poll() == None:
        a = p.poll()
        print(a)
        time.sleep(1)
    else:
        rc = p.returncode
        print(f"PID: {pid} rc: {rc}")

def main(): 
    JobsList = ['JOB90501','JOB00001','JOB00002','JOB00003']

    for Job in JobsList:
        Submit(Job)

Roy，这是我在你最后提示后的当前代码：

def SubmitJob(jobname):
    """ Submit the next requested job """
    JobDir ="/home/uwe/Jobs/"
    JobMem = "{}{}.sh".format(JobDir, jobname)
    SysoutFile = "./Sysout/{}.out".format(jobname)
    fh = open(SysoutFile, 'w')
    kwargs = {}
    kwargs.update(start_new_session=True)
    p = subprocess.Popen(JobMem, shell = False, stdout = fh, **kwargs)
    ProcessList[p] = p.pid
    print(f"Started job {jobname} - PID: {p.pid}")

def main(): 
    c_JobsList = ['JOB00001','JOB00002','JOB00003']

    for Job in c_JobsList:
        SubmitJob(Job)

    for p, pid in ProcessList.items():
        RcFile = "./Sysout/{}.rc".format(pid)
        f = open(RcFile, 'w')
        while p.poll() == None:
            a = p.poll()
            time.sleep(1)
        else:
            rc = p.returncode
            f.writelines(str(rc))
            print(f"PID: {pid} rc: {rc}")

        f.close()

以及输出：

Started job JOB00001 - PID: 5426
Started job JOB00002 - PID: 5427
Started job JOB00003 - PID: 5429
PID: 5426 rc: 0
PID: 5427 rc: 0
PID: 5429 rc: 8

Answer 1

编辑（以下原始答案供将来参考）

用于此目的的natuaram的意思是Popen.poll，但显然它在某些情况下不起作用（参见https://lists.gt.net/python/bugs/633489）。我想提出的解决方案是使用Popen.wait和非常短的超时，如以下代码示例所示：

import subprocess
import time

p = subprocess.Popen(["/bin/sleep", "3"])
print(f"Created process {p.pid}")

count = 0
while True: 
    try:
        ret = p.wait(.001) # wait for 1 ms
        print(f"Got a return code {ret}")
        break
    except subprocess.TimeoutExpired as e: 
        print("..", end = "")

    time.sleep(.5)
    print(f"Still waiting, count is {count}")
    count += 1

print ("Done!")

我得到的结果是：

Created process 30040
..Still waiting, count is 0
..Still waiting, count is 1
..Still waiting, count is 2
..Still waiting, count is 3
..Still waiting, count is 4
..Still waiting, count is 5
Got a return code 0
Done

原创创意-Popen.poll

您应该使用的方法是Popen.poll（documentation）。它返回进程的退出状态，如果进程仍在运行，则返回None

要使用它，您必须保留在调用subprocess.Popen时得到的“popen”对象，以及稍后在这些对象上调用poll时得到的对象