Sphinx警告：无法从模块`pythontemplate`导入test.test_adder

Question

背景

我在创建一个 root_dir/docs/source/conf.py 文件，这个文件可以自动生成每个 .py 文件对应的 .rst 文件，文件位置在 root_dir/src 和 root_dir/test/ 目录下（以及它的子目录）时，遇到了一些困难。具体来说，我在 .rst 文件中链接 src/projectname/__main__.py 和 root_dir/test/<测试文件>.py 时出现了问题。

这个项目的文件结构如下：

src/projectname/__main__.py
src/projectname/helper.py
test/test_adder.py
docs/source/conf.py

(其中 projectname 是：pythontemplate.)

错误信息

当我使用命令 cd docs && make html 来构建 Sphinx 文档时，收到了以下“警告”：

WARNING: Failed to import pythontemplate.test.test_adder.
Possible hints:
* AttributeError: module 'pythontemplate' has no attribute 'test'
* ModuleNotFoundError: No module named 'pythontemplate.test'

...

WARNING: autodoc: failed to import module 'test.test_adder' from module 'pythontemplate'; the following exception was raised:
No module named 'pythontemplate.test'

设计选择

我知道有些项目把 test/ 文件放在 src/test 目录下，而有些则把测试文件放在根目录下，这个项目采用的是后者。通过将测试目录命名为 test 而不是 tests，这些文件会自动包含在用 pip install -e . 创建的 dist 中。这一点可以通过打开 dist/pythontemplate-1.0.tar.gz 文件来验证，里面的 pythontemplate-1.0 目录包含了 test 目录（还有 src 目录）。不过，test 目录并没有包含在 whl 文件中。（这是希望的，因为用户不需要运行测试，但如果他们想的话，可以通过 tar.gz 文件来运行测试。）

生成的 .rst 文档文件

对于测试文件 test/test_adder.py，我生成了 root_dir/docs/source/autogen/test/test_adder.rst，内容如下：


.. _test_adder-module:

test_adder Module
=================

.. automodule:: test.test_adder
   :members:
   :undoc-members:
   :show-inheritance:

但是它无法导入 test.test_adder.py 文件。（我也尝试过 .. automodule:: pythontemplate.test.test_adder，但那也没有成功导入。）

问题

我该如何在 docs/source/autogen/test/test_<某个东西>.rst 文件中引用 root_dir/test 文件夹里的 test_<某个东西>.py 文件，以便 Sphinx 能够导入它？

Conf.py

为了完整性，下面是 conf.py 文件：

"""Configuration file for the Sphinx documentation builder.

For the full list of built-in configuration values, see the documentation:
https://www.sphinx-doc.org/en/master/usage/configuration.html

-- Project information -----------------------------------------------------
https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
"""  #

# This file only contains a selection of the most common options. For a full
# list see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html

# -- Path setup --------------------------------------------------------------

# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
# import sys
# sys.path.insert(0, os.path.abspath('.'))


# -- Project information -----------------------------------------------------

import os
import shutil
import sys

# This makes the Sphinx documentation tool look at the root of the repository
# for .py files.
from datetime import datetime
from pathlib import Path
from typing import List, Tuple

sys.path.insert(0, os.path.abspath(".."))


def split_filepath_into_three(*, filepath: str) -> Tuple[str, str, str]:
    """Split a file path into directory path, filename, and extension.

    Args:
        filepath (str): The input file path.

    Returns:
        Tuple[str, str, str]: A tuple containing directory path, filename, and
        extension.
    """
    path_obj: Path = Path(filepath)
    directory_path: str = str(path_obj.parent)
    filename = os.path.splitext(path_obj.name)[0]
    extension = path_obj.suffix

    return directory_path, filename, extension


def get_abs_root_path() -> str:
    """Returns the absolute path of the root dir of this repository.

    Throws an error if the current path does not end in /docs/source.
    """
    current_abs_path: str = os.getcwd()
    assert_abs_path_ends_in_docs_source(current_abs_path=current_abs_path)
    abs_root_path: str = current_abs_path[:-11]
    return abs_root_path


def assert_abs_path_ends_in_docs_source(*, current_abs_path: str) -> None:
    """Asserts the current absolute path ends in /docs/source."""
    if current_abs_path[-12:] != "/docs/source":
        print(f"current_abs_path={current_abs_path}")
        raise ValueError(
            "Error, current_abs_path is expected to end in: /docs/source"
        )


def loop_over_files(*, abs_search_path: str, extension: str) -> List[str]:
    """Loop over all files in the specified root directory and its child
    directories.

    Args:
        root_directory (str): The root directory to start the traversal from.
    """
    filepaths: List[str] = []
    for root, _, files in os.walk(abs_search_path):
        for filename in files:
            extension_len: int = -len(extension)
            if filename[extension_len:] == extension:
                filepath = os.path.join(root, filename)
                filepaths.append(filepath)
    return filepaths


def is_unwanted(*, filepath: str) -> bool:
    """Hardcoded filter of unwanted datatypes."""
    base_name = os.path.basename(filepath)
    if base_name == "__init__.py":
        return True
    if base_name.endswith("pyc"):
        return True
    if "something/another" in filepath:
        return True
    return False


def filter_unwanted_files(*, filepaths: List[str]) -> List[str]:
    """Filters out unwanted files from a list of file paths.

    Unwanted files include:
    - Files named "init__.py"
    - Files ending with "swag.py"
    - Files in the subdirectory "something/another"

    Args:
        filepaths (List[str]): List of file paths.

    Returns:
        List[str]: List of filtered file paths.
    """

    return [
        filepath
        for filepath in filepaths
        if not is_unwanted(filepath=filepath)
    ]


def get_abs_python_filepaths(
    *, abs_root_path: str, extension: str, root_folder_name: str
) -> List[str]:
    """Returns all the Python files in this repo."""

    # Get the file lists.
    py_files: List[str] = loop_over_files(
        abs_search_path=f"{abs_root_path}docs/source/../../{root_folder_name}",
        extension=extension,
    )

    # Merge and filter to preserve the relevant files.
    filtered_filepaths: List[str] = filter_unwanted_files(filepaths=py_files)
    return filtered_filepaths


def abs_to_relative_python_paths_from_root(
    *, abs_py_paths: List[str], abs_root_path: str
) -> List[str]:
    """Converts the absolute Python paths to relative Python filepaths as seen
    from the root dir."""
    rel_py_filepaths: List[str] = []

    for abs_py_path in abs_py_paths:

        flattened_filepath = os.path.normpath(abs_py_path)
        print(f"flattened_filepath={flattened_filepath}")
        print(f"abs_root_path={abs_root_path}")
        if abs_root_path not in flattened_filepath:
            print(f"abs_root_path={abs_root_path}")
            print(f"flattened_filepath={flattened_filepath}")
            raise ValueError(
                "Error, root dir should be in flattened_filepath."
            )
        rel_py_filepaths.append(
            os.path.relpath(flattened_filepath, abs_root_path)
        )
    return rel_py_filepaths


def delete_directory(*, directory_path: str) -> None:
    """Deletes a directory and its contents.

    Args:
        directory_path (Union[str, bytes]): Path to the directory to be
        deleted.

    Raises:
        FileNotFoundError: If the specified directory does not exist.
        PermissionError: If the function lacks the necessary permissions to
          delete the directory.
        OSError: If an error occurs while deleting the directory.

    Returns:
        None
    """
    if os.path.exists(directory_path) and os.path.isdir(directory_path):
        shutil.rmtree(directory_path)


def create_relative_path(*, relative_path: str) -> None:
    """Creates a relative path if it does not yet exist.

    Args:
        relative_path (str): Relative path to create.

    Returns:
        None
    """
    if not os.path.exists(relative_path):
        os.makedirs(relative_path)
    if not os.path.exists(relative_path):
        raise NotADirectoryError(f"Error, did not find:{relative_path}")


def create_rst(
    *,
    autogen_dir: str,
    rel_filedir: str,
    filename: str,
    pyproject_name: str,
    py_type: str,
) -> None:
    """Creates a reStructuredText (.rst) file with automodule directives.

    Args:
        rel_filedir (str): Path to the directory where the .rst file will be
        created.
        filename (str): Name of the .rst file (without the .rst extension).

    Returns:
        None
    """
    if py_type == "src":
        prelude: str = pyproject_name
    elif py_type == "test":
        prelude = f"{pyproject_name}.test"
    else:
        raise ValueError(f"Error, py_type={py_type} is not supported.")
    # if filename != "__main__":
    title_underline = "=" * len(f"{filename}-module")

    rst_content = f"""
.. _{filename}-module:

{filename} Module
{title_underline}

.. automodule:: {prelude}.{filename}
   :members:
   :undoc-members:
   :show-inheritance:
"""
    # .. automodule:: {rel_filedir.replace("/", ".")}.{filename}
    rst_filepath: str = os.path.join(
        f"{autogen_dir}{rel_filedir}", f"{filename}.rst"
    )

    with open(rst_filepath, "w", encoding="utf-8") as rst_file:
        rst_file.write(rst_content)


def generate_rst_per_code_file(
    *, extension: str, pyproject_name: str
) -> List[str]:
    """Generates a parameterised .rst file for each .py file of the project, to
    automatically include its documentation in Sphinx.

    Returns rst filepaths.
    """

    abs_root_path: str = get_abs_root_path()
    abs_src_py_paths: List[str] = get_abs_python_filepaths(
        abs_root_path=abs_root_path,
        extension=extension,
        root_folder_name="src",
    )
    abs_test_py_paths: List[str] = get_abs_python_filepaths(
        abs_root_path=abs_root_path,
        extension=extension,
        root_folder_name="test",
    )
    current_abs_path: str = os.getcwd()
    autogen_dir: str = f"{current_abs_path}/autogen/"
    prepare_rst_directories(autogen_dir=autogen_dir)
    rst_paths: List[str] = []

    rst_paths.extend(
        create_rst_files(
            pyproject_name=pyproject_name,
            abs_root_path=abs_root_path,
            autogen_dir=autogen_dir,
            abs_py_paths=abs_src_py_paths,
            py_type="src",
        )
    )
    rst_paths.extend(
        create_rst_files(
            pyproject_name=pyproject_name,
            abs_root_path=abs_root_path,
            autogen_dir=autogen_dir,
            abs_py_paths=abs_test_py_paths,
            py_type="test",
        )
    )
    return rst_paths


def prepare_rst_directories(*, autogen_dir: str) -> None:
    """Creates the output directory for the auto-generated .rst documentation
    files."""
    delete_directory(directory_path=autogen_dir)
    create_relative_path(relative_path=autogen_dir)


def create_rst_files(
    *,
    pyproject_name: str,
    abs_root_path: str,
    autogen_dir: str,
    abs_py_paths: List[str],
    py_type: str,
) -> List[str]:
    """Loops over the python files of py_type src or test, and creates the .rst
    files that point to the actual .py file such that Sphinx can generate its
    documentation on the fly."""
    rel_root_py_paths: List[str] = abs_to_relative_python_paths_from_root(
        abs_py_paths=abs_py_paths, abs_root_path=abs_root_path
    )
    rst_paths: List[str] = []

    # Create file for each py file.
    for rel_root_py_path in rel_root_py_paths:
        rel_filedir: str
        filename: str
        rel_filedir, filename, _ = split_filepath_into_three(
            filepath=rel_root_py_path
        )

        create_relative_path(relative_path=f"{autogen_dir}{rel_filedir}")

        create_rst(
            autogen_dir=autogen_dir,
            rel_filedir=rel_filedir,
            filename=filename,
            pyproject_name=pyproject_name,
            py_type=py_type,
        )

        rst_path: str = os.path.join(f"autogen/{rel_filedir}", f"{filename}")
        rst_paths.append(rst_path)
    return rst_paths


def generate_index_rst(*, filepaths: List[str]) -> str:
    """Generates the list of all the auto-generated rst files."""
    now = datetime.now().strftime("%a %b %d %H:%M:%S %Y")
    content = f"""\
.. jsonmodipy documentation main file, created by
   sphinx-quickstart on {now}.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.

.. include:: manual.rst

Auto-generated documentation from Python code
=============================================
.. toctree::
   :maxdepth: 2
"""

    for filepath in filepaths:
        content += f"\n   {filepath}"

    content += """

Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
"""

    return content


def write_index_rst(*, filepaths: List[str], output_file: str) -> None:
    """Creates an index.rst file that is used to generate the Sphinx
    documentation."""
    index_rst_content = generate_index_rst(filepaths=filepaths)

    with open(output_file, "w", encoding="utf-8") as index_file:
        index_file.write(index_rst_content)


# Call functions to generate rst Sphinx documentation structure.
# Readthedocs sets it to contents.rst, but it is index.rst in the used example.
# -- General configuration ---------------------------------------------------
project: str = "Decentralised-SAAS-Investment-Structure"
main_doc: str = "index"
PYPROJECT_NAME: str = "pythontemplate"

# pylint:disable=W0622
copyright: str = "2024, a-t-0"
author: str = "a-t-0"

the_rst_paths: List[str] = generate_rst_per_code_file(
    extension=".py", pyproject_name=PYPROJECT_NAME
)
if len(the_rst_paths) == 0:
    raise ValueError(
        "Error, did not find any Python files for which documentation needs"
        + " to be generated."
    )
write_index_rst(filepaths=the_rst_paths, output_file="index.rst")


# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions: List[str] = [
    "sphinx.ext.duration",
    "sphinx.ext.doctest",
    "sphinx.ext.autodoc",
    "sphinx.ext.autosummary",
    "sphinx.ext.intersphinx",
    # Include markdown files in Sphinx documentation
    "myst_parser",
]


# Add any paths that contain templates here, relative to this directory.
templates_path: List[str] = ["_templates"]

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns: List[str] = []


# -- Options for HTML output -------------------------------------------------

# The theme to use for HTML and HTML Help pages.  See the documentation for
# a list of builtin themes.
#
html_theme: str = "alabaster"

# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path: List[str] = ["_static"]

注意

我知道错误信息是因为测试文件不在 pythontemplate 的 pip 包中。正如上面所解释的，这是一个设计选择。我的问题是如何在不将 test 包含到 pip 包中的情况下，从 .rst 文件中导入这些测试文件。

我可以在 .rst 文件中导入 test_adder.py 文件的内容，应该可以通过以下方式实现自动文档生成：

.. _test_adder-module:

test_adder Module
=================
Hello
=====
.. include:: ../../../../test/test_adder.py
.. automodule:: ../../../../test/test_adder.py
   :members:
   :undoc-members:
   :show-inheritance:

但是 automodule 无法识别这个路径，automodule ........test/test_adder 也不行。

文件结构文档生成导入错误 sphinx 测试文件 automodule pip 包设计选择

Sphinx警告：无法从模块`pythontemplate`导入test.test_adder

背景

错误信息

设计选择

生成的 .rst 文档文件

问题

Conf.py

注意

1 个回答

更好的答案

不太好的答案

撰写回答