在Windows DLL加载上使用MKL编译NumPy失败

2024-03-28 11:06:52 发布

您现在位置:Python中文网/ 问答频道 /正文

我一直在尝试使用MSVC编译器和英特尔MKL在Windows 10上从源代码编译NumPy。 我正在使用Microsoft Visual Studio 2019(16.8.4)和英特尔MKL 2017.8.275运行Windows 10.0.18363

我使用最新的Python和NumPy,以最小的设置成功地重现了这个问题

  1. 下载最新的Python(3.9.1)和最新的NumPy(1.20.1)源代码

  2. 打开VS命令提示符,解压缩Python源代码,使用PCbuild\build.bat构建

  3. 运行mklvars.bat intel64以设置正确的环境变量

  4. 将英特尔编译器(ifort所需)添加到路径:

    set PATH=C:\Program Files (x86)\IntelSWTools\compilers_and_libraries\windows\bin\intel64;%PATH%
    
  5. 创建虚拟环境,从Python构建中复制一些文件并激活虚拟环境:

    copy Python\PCbuild\amd64\python39.dll venv\Scripts
    copy Python\PC\pyconfig.h venv\Include
    
  6. 从源代码构建NumPy并安装:pip install . -v

  7. 尝试导入NumPy:python -c "import numpy"

错误消息

Traceback (most recent call last):
  File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\__init__.py", line 22, in <module>
    from . import multiarray
  File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\multiarray.py", line 12, in <module>
    from . import overrides
  File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\overrides.py", line 7, in <module>
    from numpy.core._multiarray_umath import (
ImportError: DLL load failed while importing _multiarray_umath: The specified module could not be found.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\__init__.py", line 145, in <module>
    from . import core
  File "C:\cygwin\home\user\numpy_clean_env\venv\lib\site-packages\numpy\core\__init__.py", line 48, in <module>
    raise ImportError(msg)
ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.9 from "C:\cygwin\home\user\numpy_clean_env\venv\Scripts\python.exe"
  * The NumPy version is: "1.20.1"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: DLL load failed while importing _multiarray_umath: The specified module could not be found.

其他信息

MKL库在编译期间被拾取,因为它返回:

FOUND:
        libraries = ['mkl_rt']
        library_dirs = ['C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl\\lib\\intel64']
        define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
        include_dirs = ['C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl', 'C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl\\include', 'C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries\\windows\\mkl\\lib']

我试图分析带有依赖项的DLL解析(较新版本的Dependency Walker),但似乎MKL DLL加载良好。有些DLL似乎没有正确加载,但据我所知,这是由于Windows API设置的检查软件限制造成的,而不是由于此DLL的实际问题造成的,因此我认为系统设置正确。 以下是在分析_multiarray_umath.pyd时导致依赖项中出现NOT_FOUND的唯一DLL列表:

EMCLIENT.dll (NOT_FOUND) :
HvsiFileTrust.dll (NOT_FOUND) :
UpdateAPI.dll (NOT_FOUND) :
api-ms-win-core-comm-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-psapi-ansi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-psapi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-stringansi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-versionansi-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-versionansi-l1-1-1.dll (NOT_FOUND) :
api-ms-win-core-winrt-string-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-wow64-l1-1-0.dll (NOT_FOUND) :
api-ms-win-core-wow64-l1-1-1.dll (NOT_FOUND) :
api-ms-win-core-xstate-l2-1-0.dll (NOT_FOUND) :
api-ms-win-core-xstate-l2-1-1.dll (NOT_FOUND) :
api-ms-win-coreui-secruntime-l1-1-0.dll (NOT_FOUND) :
ext-ms-mf-pal-l2-1-0.dll (NOT_FOUND) :
ext-ms-onecore-appmodel-emclient-l1-1-0.dll (NOT_FOUND) :
ext-ms-onecore-defaultdiscovery-l1-1-0.dll (NOT_FOUND) :
ext-ms-onecore-orientation-l1-1-0.dll (NOT_FOUND) :
ext-ms-onecore-shellchromeapi-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-appmodel-deployment-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-appmodel-usercontext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-appmodel-viewscalefactor-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-audiocore-pal-l1-2-0.dll (NOT_FOUND) :
ext-ms-win-com-suspendresiliency-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-core-winrt-remote-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-dwmapi-ext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-dxcore-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-networking-wlanstorage-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-1.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-2.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-3.dll (NOT_FOUND) :
ext-ms-win-ntuser-window-l1-1-4.dll (NOT_FOUND) :
ext-ms-win-ras-tapi32-l1-1-1.dll (NOT_FOUND) :
ext-ms-win-ro-typeresolution-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-rtcore-minuser-input-l1-1-1.dll (NOT_FOUND) :
ext-ms-win-rtcore-minuser-private-ext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-security-capauthz-ext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-security-chambers-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell-knownfolderext-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell-shell32-l1-2-0.dll (NOT_FOUND) :
ext-ms-win-shell-shlwapi-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell-tabbedtitlebar-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell32-shellcom-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-shell32-shellfolders-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-ui-viewmanagement-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-uiacore-l1-1-0.dll (NOT_FOUND) :
ext-ms-win-wer-xbox-l1-1-0.dll (NOT_FOUND) :
ext-ms-win32-subsystem-query-l1-1-0.dll (NOT_FOUND) :
ext-ms-windowscore-deviceinfo-l1-1-0.dll (NOT_FOUND) :

我还尝试遵循this guide,我可以确认os.path.exists('_multiarray_umath.pyd')返回True,但是ctypes.WinDLL('_multiarray_umath.pyd')无法加载DLL(或其依赖项之一)

工作示例

与上述步骤相同,但不执行mklvars.bat,因此不链接到英特尔MKL

在过去,此设置用于Python 3.6、VS2015和类似版本的英特尔MKL。 我可能遗漏了什么,任何建议都是非常受欢迎的


Tags: andcorenumpyapil1venvnotwin
1条回答
网友
1楼 · 发布于 2024-03-28 11:06:52

按照Numpy邮件列表(permalink)上的建议解决

Python >= 3.8 will no longer use PATH for resolving dependencies of extension modules. Use os.add_dll_directory(mkl_bin_path) https://docs.python.org/3/library/os.html#os.add_dll_directory in all your scripts before importing numpy or add the call to a _distributor_init.py file in the numpy package directory.

相关问题 更多 >