Python子进程:打开的文件太多

2024-03-28 13:18:32 发布

您现在位置:Python中文网/ 问答频道 /正文

我使用subprocess调用另一个程序并将其返回值保存到变量中。这个过程在一个循环中重复,经过几千次后,程序崩溃,并出现以下错误:

Traceback (most recent call last):
  File "./extract_pcgls.py", line 96, in <module>
    SelfE.append( CalSelfEnergy(i) )
  File "./extract_pcgls.py", line 59, in CalSelfEnergy
    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, shell=True)
  File "/usr/lib/python3.2/subprocess.py", line 745, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.2/subprocess.py", line 1166, in _execute_child
    errpipe_read, errpipe_write = _create_pipe()
OSError: [Errno 24] Too many open files

任何解决这个问题的办法都是非常感谢的!

注释提供的代码:

cmd = "enerCHARMM.pl -parram=x,xtop=topology_modified.rtf,xpar=lipid27_modified.par,nobuildall -out vdwaals {0}".format(cmtup[1])
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, shell=True)
out, err = p.communicate()

Tags: inpy程序cmdtruestdoutlineextract
3条回答

我想问题是因为我正在处理一个带有子进程的打开文件:

cmd = "enerCHARMM.pl -par param=x,xtop=topology_modified.rtf,xpar=lipid27_modified.par,nobuildall -out vdwaals {0}".format(cmtup[1])
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, shell=True)

这里的cmd变量包含刚创建但未关闭的文件的名称。然后subprocess.Popen调用该文件的系统命令。在多次执行此操作后,程序因该错误消息而崩溃。

所以我从中学到的信息是

Close the file you have created, then process it

在Mac OSX(El Capitan)中,请参阅当前配置:

#ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
file size               (blocks, -f) unlimited
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 256
pipe size            (512 bytes, -p) 1
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 709
virtual memory          (kbytes, -v) unlimited

将“打开文件”的值设置为10K:

#ulimit -Sn 10000

验证结果:

#ulimit -a

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
file size               (blocks, -f) unlimited
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 10000
pipe size            (512 bytes, -p) 1
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 709
virtual memory          (kbytes, -v) unlimited

您可以尝试提高操作系统的打开文件限制:

ulimit -n 2048

相关问题 更多 >