Python BDD工具“behavior”的Bazel测试规则

Question

问题：

你好，亲爱的bazel英雄们！在

对于我们的系统测试，我试图让行为驱动开发工具behave与bazel as一起运行

• bazel test system/acceptance_criteria:nice

需要完成的工作：

• {{cd2>通过使用命令行来解决{cd3>
• 将文件夹steps添加到runfiles（通过在文件前面显式地添加路径来解决）
• 将相关Python步骤文件添加到runfiles（已解决）
• 将被测系统添加到PYTHONPATH（打开）

对于如何解决这个问题，您有什么建议或其他建议吗？在

• 使用Skylark作为一个新的规则（我已经尝试过了，但是与PYTHONPATH和runfiles做斗争。我得到了ModuleNotFoundError: No module named 'my_system'）
• 将Skylark用于带有genrule的宏
• 在py_test中调用脚本（这是一种可能但不太灵活的解决方法，因为我没有弄清楚如何将参数从bazel传递到脚本）
• 其他一些最佳实践？在

干杯， 克里斯

如果可能，请提供一个最小的示例来重现问题：

我所做的几乎就是复制Skylark中py_test的行为，并让它运行命令behave <feature_directory>，或者python -m behave <feature_directory>：

project/

• WORKSPACE（空）
• system/
  • BUILD
  • my_system.py
  • acceptance_criteria/
    • BUILD
    • nice.feature
    • steps/
      • reusable_steps.py
• tools/
  • BUILD（空）
  • behave_rule.bzl
  • bazel.rc

system/BUILD：

py_binary(
    name = "main",
    main = "my_system.py",
    srcs = ["my_system.py"],
    # deps = [], excluded for this example
    imports = ["."], # needed for adding PYTHONPATH in acceptance_criteria
    visibility = ["//system/acceptance_criteria:__pkg__"]
)

system/my_system.py：

^{pr2}$

system/acceptance_criteria/BUILD：

load("//tools:behave_rule.bzl", "py_bdd_test")

py_bdd_test(
    name = "nice",
    feats = [
        "nice.feature",
    ],
    steps = [
        "steps/reusable_steps.py",
    ],
    deps = [
        # TODO figure out how to add the implicit imports to the PYTHONPATH
        "//system:main",
    ],
    size = "small",
)

system/acceptance_criteria/nice.feature：

Feature: Multiprocess software

Scenario: 4 processes
  Given the device is powered on

steps/reusable_steps.py：

from behave import *
from my_system import MySystem

@Given("the device is powered on")
def step_impl(context):
    raise NotImplementedError(MySystem().yeah())

tools/behave_rule.bzl：

# =============================================================================
# Description: Adds a test rule for the BDD tool behave to the bazel rule set.
# Knowledge:
# * https://bazel.build/versions/master/docs/skylark/cookbook.html
# * https://bazel.build/versions/master/docs/skylark/rules.html
# * https://bazel.build/versions/master/docs/skylark/lib/ctx.html
# * http://pythonhosted.org/behave/gherkin.html
# =============================================================================

"""Private implementation of the rule py_bdd_test.
"""
def _rule_implementation(ctx):

  # Store the path of the first feature file
  features_dir = ctx.files.feats[0].dirname

  # We want a test target so make it create an executable output.
  # https://bazel.build/versions/master/docs/skylark/rules.html#test-rules
  ctx.file_action(
      # Access the executable output file using ctx.outputs.executable.
      output=ctx.outputs.executable,
      content="behave %s" % features_dir,
      executable=True
  )
  # The executable output is added automatically to this target.

  # Add the feature and step files for behave to the runfiles.
  # https://bazel.build/versions/master/docs/skylark/rules.html#runfiles
  return [DefaultInfo(
      # Create runfiles from the files specified in the data attribute.
      # The shell executable - the output of this rule - can use them at runtime.
      # It is also possible to define data_runfiles and default_runfiles.
      # However if runfiles is specified it's not possible to define the above
      # ones since runfiles sets them both.
      runfiles = ctx.runfiles(
          files = ctx.files.feats + ctx.files.steps + ctx.files.deps)
  )]

"""An example documentation.

Args:
  name:
    A unique name for this rule.
  feats:
    Feature files used to run this target.
  steps:
    Files containing the mapping of feature steps to actual system API calls.
    Note: Since this rule implicitely uses the BDD tool "behave" they must
be in the "steps" folder (https://pythonhosted.org/behave/gherkin.html).
  deps:
    System to test.
"""
py_bdd_test = rule(
    implementation=_rule_implementation,
    attrs={
      # Do not declare "name": It is added automatically.
      "feats": attr.label_list(allow_files=True),
      "steps": attr.label_list(allow_files=True),
      "deps":
        attr.label_list(
            mandatory=True,
            non_empty=True,)
    },
    test=True,
)

tools/bazel.rc：

test --test_output=errors

环境信息

• 操作系统：带docker的Windows 10（参见下面的Dockerfile）
• Bazel版本（输出bazel info release）：版本0.5.1rc1

Dockerfile：

FROM ubuntu:xenial

# Install essentials
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update \
  && apt-get install -y gnupg git \
  && apt-get clean

# -----------------------------------------------------------------------------

# Install the awesome build automation tool bazel
# https://bazel.build/
# version > 0.4.5 since it has a bug in the rule extension skylark
RUN echo "deb http://storage.googleapis.com/bazel-apt testing jdk1.8" \
  > /etc/apt/sources.list.d/bazel.list
RUN apt-key adv --keyserver pool.sks-keyservers.net --recv-key 3D5919B448457EE0
RUN apt-get update \
  && apt-get install -y openjdk-8-jdk bazel \
  && apt-get clean \
  && rm -rf /var/lib/apt/lists/* && mkdir /var/lib/apt/lists/partial
ENV DEBIAN_FRONTEND ""

# Setup environment for bazel
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64

# Run bazel a first time for it to self-extract
RUN /usr/bin/bazel version

# -----------------------------------------------------------------------------

# Install Python (using the non-default version 3.6)
RUN echo "deb http://ppa.launchpad.net/jonathonf/python-3.6/ubuntu xenial main" \
  > /etc/apt/sources.list.d/jonathonf-ubuntu-python-3_6-xenial.list \
  && apt-key adv --keyserver pool.sks-keyservers.net --recv-key 8CF63AD3F06FC659 \
  && apt-get update \
  && apt-get install -y python3.6

# Create symlink from python3.6 to python
RUN ln -s /usr/bin/python3.6 /usr/local/bin/python

# Install the Python package manager pip (for the non-default version 3.6)
# https://en.wikipedia.org/wiki/Pip_(package_manager)
RUN apt-get update \
  && apt-get install -y wget \
  && wget https://bootstrap.pypa.io/get-pip.py \
  && python3.6 get-pip.py \
  && rm get-pip.py \
  && pip install --upgrade pip

# -----------------------------------------------------------------------------

# Install the Gherkin-based BDD tool "behave" for Python
RUN pip install behave

# -----------------------------------------------------------------------------

# Our build environment is based on bazel. Now run tests with it.
ENTRYPOINT "/usr/bin/bazel"

你在网上找到了什么相关的东西吗？

不幸的是没有。正如我在调查的behave_rule.bzl的评论中提到的

• https://bazel.build/versions/master/docs/skylark/cookbook.html
• https://bazel.build/versions/master/docs/skylark/rules.html
• https://bazel.build/versions/master/docs/skylark/lib/ctx.html

我还发现

• https://github.com/bazelbuild/bazel/issues/702

还有其他有用的信息、日志或输出吗？

命令行输出：

root@eec1fa791491:/project# bazel test system/acceptance_criteria:nice
INFO: Found 1 test target...
FAIL: //system/acceptance_criteria:nice (see /root/.cache/bazel/_bazel_root/2ca1f4ebdc59348ffdc31d97a51a98d5/execroot/project/bazel-out/local-fastbuild/testlogs/system/acceptance_criteria/nice/test.log).
INFO: From Testing //system/acceptance_criteria:nice:
==================== Test output for //system/acceptance_criteria:nice:
Exception ModuleNotFoundError: No module named 'my_system'
Traceback (most recent call last):
  File "/usr/local/bin/behave", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.6/dist-packages/behave/__main__.py", line 109, in main
    failed = runner.run()
  File "/usr/local/lib/python3.6/dist-packages/behave/runner.py", line 672, in run
return self.run_with_paths()
File "/usr/local/lib/python3.6/dist-packages/behave/runner.py", line 678, in run_with_paths
    self.load_step_definitions()
File "/usr/local/lib/python3.6/dist-packages/behave/runner.py", line 658, in load_step_definitions
    exec_file(os.path.join(path, name), step_module_globals)
File "/usr/local/lib/python3.6/dist-packages/behave/runner.py", line 304, in exec_file
    exec(code, globals, locals)
File "system/acceptance_criteria/steps/reusable_steps.py", line 5, in <module>
    from my_system import MySystem
ModuleNotFoundError: No module named 'my_system'
================================================================================
Target //system/acceptance_criteria:nice up-to-date:
bazel-bin/system/acceptance_criteria/nice
INFO: Elapsed time: 0.538s, Critical Path: 0.22s
//system/acceptance_criteria:nice                                        FAILED in 0.2s
/root/.cache/bazel/_bazel_root/2ca1f4ebdc59348ffdc31d97a51a98d5/execroot/project/bazel-out/local-fastbuild/testlogs/system/acceptance_criteria/nice/test.log

Executed 1 out of 1 test: 1 fails locally.

来源：

• https://github.com/bazelbuild/bazel/issues/3194#issuecomment-309236573

Answer 1

所以目前我们可以为

• 在py_测试中调用脚本（这是一种可能但不太灵活的解决方法，因为我没有弄清楚如何将参数从bazel传递到脚本）

在弄清楚如何将参数从bazel传递到脚本之后。方法如下：

project/WORKSPACE:
new_local_repository(
    name = "tools",
    build_file = "support/BUILD.tools",
    path = "../tools",
)


project/support/BUILD.tools:
py_library(
    name = "bdd_tool",
    srcs = ["run_bdd_test.py"],
    visibility = ["//visibility:public"],
)


project/acceptance_criteria/BUILD:
[py_test(
    name = filename[:filename.rfind(".feature")],
    main = "run_bdd_test.py",
    args = [
        filename[:filename.rfind(".feature")],
        package_name(),
        "$(location //cpp_system_simulator)",
    ],
    srcs = glob(["steps/*.py"]) + [
        "environment.py",
        "@tools//:bdd_tool",
    ],
    deps = [
        # fixtures and support files
    ],
    data = [
        filename,
        "//cpp_system_simulator"
    ],
for filename in glob(["*.feature"])]


tools/run_bdd_test.py:
"""
Calls behave from the command line for use in a bazel py_test.
Features tagged with '@wip' are skipped.
"""

import sys
import subprocess

BDD_TOOL = "behave"
COMMAND = [BDD_TOOL, " no-capture", " no-capture-stderr", " no-logcapture"]

if len(sys.argv) >= 3:
    FEATURE_NAME = sys.argv[1]
    FEATURES_PATH = sys.argv[2]
    COMMAND.append(FEATURES_PATH)
    COMMAND.append(" include")
    COMMAND.append(FEATURE_NAME)
    COMMAND.append(" tags=-wip")

if len(sys.argv) >= 4:
    COMMAND.append(" define")
    COMMAND.append("cpp_system_simulator_path=" + sys.argv[3])

subprocess.run(COMMAND, check=True)