在需要在导入期间创建文件的Singularity容器中安装Python包（例如启用GPU渲染的mujoco_py）

Question

请注意，下面的部分描述特定于Python包mujoco_py，但问题本身是一般性的

我按照here和here中描述的步骤构建了一个容器，其中安装了mujoco-py，使用gpu进行渲染。但是，当import mujoco_py尝试创建使用GPU渲染所需的一些文件/目录时，我会遇到以下错误。但是，由于文件系统变为只读，因此在构建容器后无法在容器内创建任何文件，除非这些文件将存储在/tmp中。另外，在我的例子中，将容器构建为可写的不是一个选项。我还尝试通过python3 -m pip install . -e而不是python3 setup.py install安装mujoco-py，但在安装import mujoco_py时遇到了一些其他类似的错误

有没有办法改变创建这些文件的路径，或者将这些文件保存在内存中

import mujoco_py
running build_ext
building 'mujoco_py.cymj' extension
creating /usr/local/lib/python3.7/dist-packages/mujoco_py-2.0.2.13-py3.7.egg/mujoco_py/generated/_pyxbld_2.0.2.13_37_linuxgpuextensionbuilder
Traceback (most recent call last):
  File "/usr/lib/python3.7/distutils/dir_util.py", line 70, in mkpath
    os.mkdir(head, mode)
OSError: [Errno 30] Read-only file system: '/usr/local/lib/python3.7/dist-packages/mujoco_py-2.0.2.13-py3.7.egg/mujoco_py/generated/_pyxbld_2.0.2.13_37_linuxgpuextensionbuilder'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.7/dist-packages/mujoco_py-2.0.2.13-py3.7.egg/mujoco_py/__init__.py", line 3, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/usr/local/lib/python3.7/dist-packages/mujoco_py-2.0.2.13-py3.7.egg/mujoco_py/builder.py", line 510, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/usr/local/lib/python3.7/dist-packages/mujoco_py-2.0.2.13-py3.7.egg/mujoco_py/builder.py", line 105, in load_cython_ext
    cext_so_path = builder.build()
  File "/usr/local/lib/python3.7/dist-packages/mujoco_py-2.0.2.13-py3.7.egg/mujoco_py/builder.py", line 221, in build
    built_so_file_path = self._build_impl()
  File "/usr/local/lib/python3.7/dist-packages/mujoco_py-2.0.2.13-py3.7.egg/mujoco_py/builder.py", line 291, in _build_impl
    so_file_path = super()._build_impl()
  File "/usr/local/lib/python3.7/dist-packages/mujoco_py-2.0.2.13-py3.7.egg/mujoco_py/builder.py", line 244, in _build_impl
    dist.run_commands()
  File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.7/dist-packages/Cython/Distutils/old_build_ext.py", line 186, in run
    _build_ext.build_ext.run(self)
  File "/usr/lib/python3.7/distutils/command/build_ext.py", line 340, in run
    self.build_extensions()
  File "/usr/local/lib/python3.7/dist-packages/mujoco_py-2.0.2.13-py3.7.egg/mujoco_py/builder.py", line 144, in build_extensions
    build_ext.build_extensions(self)
  File "/usr/local/lib/python3.7/dist-packages/Cython/Distutils/old_build_ext.py", line 195, in build_extensions
    _build_ext.build_ext.build_extensions(self)
  File "/usr/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
    self._build_extensions_serial()
  File "/usr/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
    self.build_extension(ext)
  File "/usr/lib/python3.7/distutils/command/build_ext.py", line 534, in build_extension
    depends=ext.depends)
  File "/usr/lib/python3.7/distutils/ccompiler.py", line 566, in compile
    depends, extra_postargs)
  File "/usr/lib/python3.7/distutils/ccompiler.py", line 348, in _setup_compile
    self.mkpath(os.path.dirname(obj))
  File "/usr/lib/python3.7/distutils/ccompiler.py", line 916, in mkpath
    mkpath(name, mode, dry_run=self.dry_run)
  File "/usr/lib/python3.7/distutils/dir_util.py", line 74, in mkpath
    "could not create '%s': %s" % (head, exc.args[-1]))
distutils.errors.DistutilsFileError: could not create '/usr/local/lib/python3.7/dist-packages/mujoco_py-2.0.2.13-py3.7.egg/mujoco_py/generated/_pyxbld_2.0.2.13_37_linuxgpuextensionbuilder': Read-only file system

下面是我如何构建的

Answer 1

现在可以通过奇点容器中的 overlay在容器中写入[临时]文件：

https://sylabs.io/guides/3.7/user-guide/persistent_overlays.html