一个高度兼容的用于linting python文件的“构建”系统。
makelint的Python项目详细描述
一个高度兼容的用于linting python文件的“构建”系统。
目的
这个程序的目的是对python代码进行lint/检查。你为什么 需要专门的软件来做这个?
- python静态分析工具(pylint、flake8等)速度很慢。如果你 有超过几十个文件,然后对每个文件进行linting可能需要相当长的时间 很久了。
- 这些工具通常加载模块以了解python代码 正在做。这意味着检查程序不仅仅取决于 正在检查,但其依赖项的完整传递闭包
这个程序维护一个依赖关系数据库,这样您就可以确信 一张特别的皮棉支票仍然有效。你可以肯定地知道不仅仅是 一个特定的文件没有改变,但是它所依赖的文件也没有改变 已经改变了。这允许您跳过重新检查未更改的文件。
本项目的目的是(最终)支持标准构建系统 类似于make或ninja,但它也实现了一个自包含的解决方案。 特别是,它允许“退出”工作流(而不是选择加入)的含义 两件事:
- 添加新文件时不需要重新运行configure(或cmake) (而且,让我们面对现实吧,您可能正受到一些配置的影响 你不是很胖吗?
- 当新的python被添加到代码基中时,它会被自动包含 用于检查。一个人必须做出明确的决定来排除它。 开发人员不能“忘记”将其添加到构建系统中进行linting。
用法
usage: pymakelint [-h] [-v] [-l {debug,info,warning,error}] [--dump-config] [-c CONFIG_FILE] [<config-overrides> [...]] Incremental execution system for python code analysis (linting). optional arguments: -h, --help show this help message and exit -v, --version show program's version number and exit -l {debug,info,warning,error}, --log-level {debug,info,warning,error} --dump-config If specified, print the default configuration to stdout and exit -c CONFIG_FILE, --config-file CONFIG_FILE path to configuration file Configuration: Override configfile options --include-patterns [INCLUDE_PATTERNS [INCLUDE_PATTERNS ...]] A list of python regular expression patterns which are used to include files during the directory walk. They are matched against relative paths of files (relative to the root of the search). They are not matched against directories. The default is `[".*\.py"]`. --exclude-patterns [EXCLUDE_PATTERNS [EXCLUDE_PATTERNS ...]] A list of python regular expression patterns which are used to exclude files during the directory walk. They are matched against relative paths of files (relative to the root of the search). If the pattern matches a directory the whole directory is skipped. If it matches an individual file then that file is skipped. --source-tree SOURCE_TREE The root of the search tree for inclusion. --target-tree TARGET_TREE The root of the tree where the outputs are written. --tools [TOOLS [TOOLS ...]] A list of tools to execute. The default is ["pylint", "flake8"]. This can either be a string (a simple command which takes one argument), or it can be an object with a get_stamp() and an execute() method. See SimpleTool for ane example. --fail-fast [FAIL_FAST] If true, exit on the first failure, don't keep going. Useful if you want a speedy CI gate. --merged-log MERGED_LOG If specified, output logs for failed jobs will be merged into a single file at this location. Useful if you have a large number of issues to del with. --quiet [QUIET] Don't print fancy progress bars to stdout. --jobs JOBS Number of parallel jobs to execute.
配置
大多数命令行选项也可以在配置文件中指定。 配置文件是python文件。如果未在命令行中指定, 工具将自动查找并加载配置文件 <source_tree>/.makelint.py。
可以使用--dump-config将默认配置转储到文件中,然后 以此为出发点。默认配置也粘贴在下面。
# A list of python regular expression patterns which are used to include files # during the directory walk. They are matched against relative paths of files # (relative to the root of the search). They are not matched against # directories. The default is `[".*\.py"]`. include_patterns = ['.*\\.py'] # A list of python regular expression patterns which are used to exclude files # during the directory walk. They are matched against relative paths of files # (relative to the root of the search). If the pattern matches a directory the # whole directory is skipped. If it matches an individual file then that file is # skipped. exclude_patterns = [] # The root of the search tree for inclusion. source_tree = None # The root of the tree where the outputs are written. target_tree = None # A list of tools to execute. The default is ["pylint", "flake8"]. This can # either be a string (a simple command which takes one argument), or it can be # an object with a get_stamp() and an execute() method. See SimpleTool for ane # example. tools = ['flake8', 'pylint'] # A dictionary specifying the environment to use for the tools. Add your # virtualenv configurations here. env = { "LANG": "en_US.UTF-8", "LANGUAGE": "en_US", "PATH": [ "/usr/local/sbin", "/usr/local/bin", "/usr/sbin", "/usr/bin", "/sbin", "/bin" ] } # If true, exit on the first failure, don't keep going. Useful if you want a # speedy CI gate. fail_fast = False # If specified, output logs for failed jobs will be merged into a single file # at this location. Useful if you have a large number of issues to del with. merged_log = None # Don't print fancy progress bars to stdout. quiet = False # Number of parallel jobs to execute. jobs = 12 # multiprocessing.cpu_count()
设计
发现步骤执行文件系统遍历以建立索引 要检查的文件。您可以使用配置文件或命令行 用于设置发现进程的包含和排除筛选器的选项。 不过,一般来说,扫描的每个目录都会生成一个文件列表 皮棉。如果跟踪目录的时间戳更改,则会重新扫描为新的 文件或新目录。
发现阶段的输出是每个跟踪目录的清单文件。 此清单的创建取决于目录的修改时间 它与目录对应,如果目录被更改,它将被重新构建。如果一个新的 添加子目录后,系统将递归索引该新目录。 如果目录被删除,它将递归地从 清单索引。
内容摘要
第二阶段是内容总结和摘要创建。每个人的sha1 跟踪文件被计算并存储在摘要文件中(每个源文件一个)。 摘要文件取决于源文件的修改时间。
依赖推断
第三阶段是依赖推理。在此阶段,每个跟踪 对源文件进行索引以获得完整的依赖关系示意图。注意这个 通过在一个干净的解释器进程中导入每个模块文件来完成,并且 然后检查加载的所有模块的__file__属性 翻译。请注意,这有两个含义:
- 动态加载的模块不能作为依赖项被发现
- 任何导入工作都将增加此阶段的运行时间
此阶段的输出是依赖项清单:每个源文件一个。这个 清单包含依赖项文件的列表。的依赖关系 此清单是 每个源文件本身,以及它的所有依赖项。如果有的话 修改这些摘要文件,重新生成清单。有一条捷径, 然而,如果没有任何消化物本身改变了显化 修改时间已更新,但跳过依赖项扫描。
执行工具
一旦依赖性封装被更新,我们就可以开始执行 实际工具。工具执行有两个输出:stampfile (每个源文件一个)和一个日志文件。失败时跳过stampfile,并且 成功时将删除日志文件。