我可以在正在写入的文件上使用fdpexpect吗？

Question

我正在尝试在Python中等待一些文本写入一个实时的日志文件。

fdpexpect看起来是处理这个问题的合适工具，但它并没有等待。当它到达文件末尾时就结束了。

我在想fdpexpect是不是不支持这个功能，我是否需要找到其他方法来解决这个问题？

我现在的代码大致是这样的：

创建一个spawn对象：

# we're not using pexpect.spawn because we want
# all the output to be written to the logfile in real time, 
# which spawn doesn't seem to support.

p = subprocess.Popen(command,
                     shell=shell,
                     stdout=spawnedLog.getFileObj(),
                     stderr=subprocess.STDOUT)
# give fdspawn the same file object we gave Popen
return (p, pexpect.fdpexpect.fdspawn(spawnedLog.getFileObj()))

等待某些事情发生：

pexpectObj.expect('something')

这段代码基本上是立即退出，并且在“某些事情”发生之前就出现了EOF错误。

Answer 1

另一种方法是直接使用 'tail -f'，虽然这听起来有点奇怪，而且你需要确保你的系统里有 'tail' 这个工具。

p = subprocess.Popen(command,
                     shell=shell,
                     stdout=spawnedLog.getFileObj(),
                     stderr=subprocess.STDOUT)

# this seems really dumb, but in order to follow the log                                                                                                                                                     
# file and not have fdpexect quit because we encountered EOF                                                                                                                                                 
# we're going spawn *another* process to tail the log file                                                                                                                                                   
tailCommand = "tail -f %s" % spawnedLog.getPath()

# this is readonly, we're going to look at the output logfile                                                                                                                                                
# that's created                                                                                                                                                                                             
return (p, pexpect.spawn(tailCommand))

Answer 2

fdpexpect 这个工具并不是为了处理普通文件而设计的。pexpect 会一直从一个文件对象中读取数据，直到遇到文件结束符（EOF）。对于管道和套接字来说，只有在连接真正关闭时才会遇到这个结束符；但对于普通文件来说，一旦文件内容全部读取完，就会立刻遇到结束符。它无法知道这个文件是否正在被其他程序写入。

你可以通过使用 os.pipe 创建一个管道来解决这个问题，然后自己实现一个 tee 功能，把你的程序的标准输出（stdout）同时写入这个管道和日志文件。下面是一个简单的示例，似乎可以正常工作：

from subprocess import Popen, PIPE, STDOUT
from threading  import Thread
import os
import pexpect.fdpexpect

# tee and teed_call are based on http://stackoverflow.com/a/4985080/2073595

def tee(infile, *files):
    """Print `infile` to `files` in a separate thread."""
    def fanout(infile, *files):
        for line in iter(infile.readline, ''):
            for f in files:
                f.write(line)
        infile.close()
    t = Thread(target=fanout, args=(infile,)+files)
    t.daemon = True
    t.start()
    return t

def teed_call(cmd_args, files, **kwargs):
    p = Popen(cmd_args,
              stdout=PIPE,
              stderr=STDOUT,
              **kwargs)
    threads = []
    threads.append(tee(p.stdout, *files))
    return (threads, p)

with open("log.txt", 'w') as logf:
    # Create pipes for unbuffered reading and writing
    rpipe, wpipe = os.pipe()
    rpipe = os.fdopen(rpipe, 'r', 0)
    wpipe = os.fdopen(wpipe, 'w', 0)

    # Have pexpect read from the readable end of the pipe
    pobj = pexpect.fdpexpect.fdspawn(rpipe)

    # Call some script, and tee output to our log file and
    # the writable end of the pipe.
    threads, p = teed_call(["./myscript.sh"], [wpipe, logf])

    # myscript.sh will print 'hey'
    pobj.expect("hey")

    # orderly shutdown/cleanup
    for t in threads: t.join()
    p.wait()
    rpipe.close()
    wpipe.close()