我正在构建我自己的PyTorch CUDA扩展,并尝试将其安装到Docker容器中。在本例中,当容器在计算集群上启动时,代码将编译(实际上由代码生成器生成)。在
由于集群上的执行方式,我可以在启动自己的应用程序脚本之前以root用户身份运行一个脚本。此脚本无法访问python扩展的代码,因为代码是由代码生成器动态生成的。为此,我执行以下操作以允许安装包(容器中的python解释器位于/opt/conda/bin中):
chmod 777 /opt/conda/lib/python3.6/site-packages
然后使用启动容器的用户的权限来构建PyTorch扩展。此时,脚本只需在代码生成器的输出文件夹中运行以下命令:
^{pr2}$代码构建得很好,我看到正在创建以下目录:
/opt/conda/lib/python3.6/site-packages/foobar-1.2.4-py3.6-linux-x86_64.egg
然后,以下代码在容器内工作:
import sys
sys.path.insert(0, "/opt/conda/lib/python3.6/site-packages/foobar-1.2.4-py3.6-linux-x86_64.egg")
import foobar
但是,如果我把系统路径插入,我得到:
ModuleNotFoundError: No module named 'foobar'
似乎站点包中的egg文件不能自动发现?文件夹的内容是:
/opt/conda/lib/python3.6/site-packages/foobar-1.2.4-py3.6-linux-x86_64.egg/__pycache__
/opt/conda/lib/python3.6/site-packages/foobar-1.2.4-py3.6-linux-x86_64.egg/__pycache__/foobar.cpython-36.pyc
/opt/conda/lib/python3.6/site-packages/foobar-1.2.4-py3.6-linux-x86_64.egg/foobar.cpython-36m-x86_64-linux-gnu.so
/opt/conda/lib/python3.6/site-packages/foobar-1.2.4-py3.6-linux-x86_64.egg/foobar.py
/opt/conda/lib/python3.6/site-packages/foobar-1.2.4-py3.6-linux-x86_64.egg/EGG-INFO
/opt/conda/lib/python3.6/site-packages/foobar-1.2.4-py3.6-linux-x86_64.egg/EGG-INFO/SOURCES.txt
/opt/conda/lib/python3.6/site-packages/foobar-1.2.4-py3.6-linux-x86_64.egg/EGG-INFO/top_level.txt
/opt/conda/lib/python3.6/site-packages/foobar-1.2.4-py3.6-linux-x86_64.egg/EGG-INFO/dependency_links.txt
/opt/conda/lib/python3.6/site-packages/foobar-1.2.4-py3.6-linux-x86_64.egg/EGG-INFO/not-zip-safe
/opt/conda/lib/python3.6/site-packages/foobar-1.2.4-py3.6-linux-x86_64.egg/EGG-INFO/native_libs.txt
/opt/conda/lib/python3.6/site-packages/foobar-1.2.4-py3.6-linux-x86_64.egg/EGG-INFO/PKG-INFO
我看到有些包在site包下有pth文件,但没有一个是由python setup.py install
生成的。我在跑步的日志里看到了设置.py以下内容:
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing foobar-1.2.4-py3.6-linux-x86_64.egg
creating /opt/conda/lib/python3.6/site-packages/foobar-1.2.4-py3.6-linux-x86_64.egg
Extracting foobar-1.2.4-py3.6-linux-x86_64.egg to /opt/conda/lib/python3.6/site-packages
Adding foobar 1.2.4 to easy-install.pth file
但是,实际上我没有看到创建任何pth文件。有什么办法让包裹看得见吗?也没有权限被拒绝的错误,将整个site packages文件夹改为递归全局可写也不能解决问题。在
编辑:有一件有趣的事要注意,我找不到一个名为“easy”的文件-安装.pth“docker图像中的任何地方。这样的文件应该存在于站点包中吗?以下是唯一的pth文件:
root@cb54e4e35141:/opt/conda/lib/python3.6/site-packages# ls *.pth
matplotlib-3.1.1-py3.6-nspkg.pth sphinxcontrib_htmlhelp-1.0.2-py3.7-nspkg.pth
protobuf-3.8.0-py3.6-nspkg.pth sphinxcontrib_jsmath-1.0.1-py3.7-nspkg.pth
pytest-cov.pth sphinxcontrib_qthelp-1.0.2-py3.7-nspkg.pth
sphinxcontrib_applehelp-1.0.1-py3.7-nspkg.pth sphinxcontrib_serializinghtml-1.1.3-py3.7-nspkg.pth
sphinxcontrib_devhelp-1.0.1-py3.7-nspkg.pth
更新:结果是pip install .
而不是{
目前没有回答
相关问题 更多 >
编程相关推荐