嵌套的Luigi任务不会出现在执行摘要中

Question

我计划使用Luigi编写一个可复制且抗故障的超参数调优任务。因此，我在“parent”class HParamOptimizer中多次调用我的class TrainOneModel

为了简化问题，这里有一个更简单的hello world版本：

import luigi

# Child class
class HelloTask(luigi.Task):
    name = luigi.parameter.Parameter(default='Luigi')

    def run(self):
        print(f'Luigi says: Hello {self.name}!')

# Parent class
class ManyHellos(luigi.Task):

    def run(self):
        names = ['Marc', 'Anna', 'John']
        for name in names:
            hello = HelloTask(name=name)
            hello.run()

if __name__ == '__main__':
    luigi.run(['ManyHellos', '--workers', '1', '--local-scheduler'])

使用python filename.py运行脚本可以工作，并且进度看起来：）。名称也按预期打印，但是，执行摘要仅显示ManyHellos运行：

Scheduled 1 tasks of which:
* 1 ran successfully:
    - 1 ManyHellos()

是否有可能包含子class HelloTask以查看中央日程安排可视化工具中的进展情况

谢谢，BBQuercus

Answer 1

它们不会显示，因为您正在手动执行它们的run（），而不是通过调度程序执行。luigi的方法更像这样：

import luigi


# Child class
class HelloTask(luigi.Task):
    name = luigi.parameter.Parameter(default='Luigi')

    def run(self):
        with self.output().open('w') as fout:
            fout.write(f'Luigi says: Hello {self.name}!')

    def output(self):
        # An output target is needed for the scheduler to verify whether
        # the task was run.
        return luigi.LocalTarget(f'./names/{self.name}.txt')


# Parent class
class ManyHellos(luigi.Task):
    def run(self):
        names = ['Marc', 'Anna', 'John']
        for name in names:
            yield HelloTask(name=name)  # dynamically schedules a HelloTask


if __name__ == '__main__':
    luigi.run(['ManyHellos', ' workers', '1', ' local-scheduler'])

这导致了

===== Luigi Execution Summary =====

Scheduled 4 tasks of which:
* 4 ran successfully:
    - 3 HelloTask(name=Anna,John,Marc)
    - 1 ManyHellos()

This progress looks :) because there were no failed tasks or missing dependencies

执行时。注意，您还可以使用yield [HelloTask(name) for name in names]一次生成多个任务