成功安装变压器后无法导入管道

Question

环境信息

• transformers版本：4.9.0
• 平台：Linux-4.15.0-151-generic-x86_64-with-glibc2.27
• Python版本：3.9.2
• PyTorch版本（GPU？）：1.7.1+cu101（错误）
• Tensorflow版本（GPU？）：2.5.0（错误）
• 亚麻版本（CPU？/GPU？/TPU？）：0.3.4（CPU）
• Jax版本：0.2.17
• JaxLib版本：0.1.69
• 在脚本中使用GPU？：否
• 在脚本中使用分布式或并行设置？：否

详细信息

我正在尝试使用新安装的transformers库，但在使用pip成功完成安装后，我无法运行测试脚本：python -c "from transformers import pipeline; print(pipeline('sentiment-analysis')('we love you'))"

相反，我看到以下输出：

/home/shushan/.conda/envs/ccg_parser/lib/python3.9/site-packages/gensim/similarities/init.py:15: UserWarning: The gensim.similarities.levenshtein submodule is disabled, because the optional Levenshtein package <https://pypi.org/proje$ t/python-Levenshtein/> is unavailable. Install Levenhstein (e.g. pip install python-Levenshtein) to suppress this warning.
warnings.warn(msg)
Traceback (most recent call last):
File "", line 1, in
File "/home/shushan/.conda/envs/ccg_parser/lib/python3.9/site-packages/transformers/file_utils.py", line 1977, in getattr
module = self._get_module(self._class_to_module[name])
File "/home/shushan/.conda/envs/ccg_parser/lib/python3.9/site-packages/transformers/file_utils.py", line 1986, in _get_module
return importlib.import_module("." + module_name, self.name)
File "/home/shushan/.conda/envs/ccg_parser/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/home/shushan/.conda/envs/ccg_parser/lib/python3.9/site-packages/transformers/pipelines/init.py", line 25, in
from ..models.auto.configuration_auto import AutoConfig
File "/home/shushan/.conda/envs/ccg_parser/lib/python3.9/site-packages/transformers/models/init.py", line 19, in
from . import (
File "/home/shushan/.conda/envs/ccg_parser/lib/python3.9/site-packages/transformers/models/layoutlm/init.py", line 22, in
from .configuration_layoutlm import LAYOUTLM_PRETRAINED_CONFIG_ARCHIVE_MAP, LayoutLMConfig
File "/home/shushan/.conda/envs/ccg_parser/lib/python3.9/site-packages/transformers/models/layoutlm/configuration_layoutlm.py", line 19, in
from ..bert.configuration_bert import BertConfig
File "/home/shushan/.conda/envs/ccg_parser/lib/python3.9/site-packages/transformers/models/bert/configuration_bert.py", line 21, in
from ...onnx import OnnxConfig
File "/home/shushan/.conda/envs/ccg_parser/lib/python3.9/site-packages/transformers/onnx/init.py", line 16, in
from .config import EXTERNAL_DATA_FORMAT_SIZE_LIMIT, OnnxConfig, OnnxConfigWithPast
File "/home/shushan/.conda/envs/ccg_parser/lib/python3.9/site-packages/transformers/onnx/config.py", line 18, in
from transformers import PretrainedConfig, PreTrainedTokenizer, TensorType
File "/home/shushan/.conda/envs/ccg_parser/lib/python3.9/site-packages/transformers/file_utils.py", line 1977, in getattr
module = self._get_module(self._class_to_module[name])
File "/home/shushan/.conda/envs/ccg_parser/lib/python3.9/site-packages/transformers/file_utils.py", line 1986, in _get_module
return importlib.import_module("." + module_name, self.name)
File "/home/shushan/.conda/envs/ccg_parser/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/home/shushan/.conda/envs/ccg_parser/lib/python3.9/site-packages/transformers/tokenization_utils.py", line 26, in
from .tokenization_utils_base import (
File "/home/shushan/.conda/envs/ccg_parser/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 74, in
from tokenizers import AddedToken
File "/home/shushan/tokenization_experiments/tokenizers.py", line 26, in
from transformers import BertTokenizer File "/home/shushan/.conda/envs/ccg_parser/lib/python3.9/site-packages/transformers/file_utils.py", line 1978, in getattr value = getattr(module, name) File "/home/shushan/.conda/envs/ccg_parser/lib/python3.9/site-packages/transformers/file_utils.py", line 1977, in getattr module = self._get_module(self._class_to_module[name]) File "/home/shushan/.conda/envs/ccg_parser/lib/python3.9/site-packages/transformers/file_utils.py", line 1986, in _get_module return importlib.import_module("." + module_name, self.name) File "/home/shushan/.conda/envs/ccg_parser/lib/python3.9/importlib/init.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/home/shushan/.conda/envs/ccg_parser/lib/python3.9/site-packages/transformers/models/bert/tokenization_bert.py", line 23, in from ...tokenization_utils import PreTrainedTokenizer, _is_control, _is_punctuation, _is_whitespace ImportError: cannot import name 'PreTrainedTokenizer' from partially initialized module 'transformers.tokenization_utils' (most likely due to a circular import) (/home/shushan/.conda/envs/ccg_parser/lib/python3.9/site-packages/transformer s/tokenization_utils.py)

我曾尝试卸载变压器并重新安装它们，但我找不到更多关于问题所在的信息，也找不到如何解决我所看到的问题的信息。唯一可疑的行为是上面的transformers-cli env命令的输出指定pytorch版本不能与GPU一起工作，而实际上，我安装了一个与GPU一起工作的pytorch。你能帮忙吗？ 提前谢谢 蜀山

Answer 1

可能是Pytorch和TensorFlow的存在或环境的不正确创建导致了问题。尝试重新创建环境，同时安装最少的裸包，只需保留Pytorch或TensorFlow中的一个即可

使用以下配置对我来说效果非常好：

 - transformers version: 4.9.0
 - Platform: macOS-10.14.6-x86_64-i386-64bit
 - Python version: 3.9.2 
 - PyTorch version (GPU?): 1.7.1 (False) 
 - Tensorflow version (GPU?): not installed (NA) 
 - Flax version (CPU?/GPU?/TPU?): not installed (NA) 
 - Jax version: not installed 
 - JaxLib version: not installed 
 - Using GPU in script?: <fill in> 
 - Using distributed or parallel set-up in script?: <fill in>