从拥抱面权重建立tensorflow模型的问题

Question

我需要处理来自Huggingface和Tensorflow（在this链接）的预训练BERT模型（'dbmdz/bert-base-italian-xxl-cased'）

在网站上读到这个之后,

Currently only PyTorch-Transformers compatible weights are available. If you need access to TensorFlow checkpoints, please raise an issue!

我提出了这个问题，并立即向我提供了一个包含以下文件的档案下载链接。这些文件如下所示：

$ ls bert-base-italian-xxl-cased/
config.json                    model.ckpt.index               vocab.txt
model.ckpt.data-00000-of-00001 model.ckpt.meta

我现在尝试加载模型并使用它，但我尝试的一切都失败了

我试图从一个Huggingface讨论网站上得到以下建议：

bert_folder = str(Config.MODELS_CONFIG.BERT_CHECKPOINT_DIR) # folder in which I have the files extracted from the archive
from transformers import BertConfig, TFBertModel
config = BertConfig.from_pretrained(bert_folder) # this gets loaded correctly

在此之后，我尝试了几种组合以加载模型，但总是失败

例如：

model = TFBertModel.from_pretrained("../../models/pretrained/bert-base-italian-xxl-cased/model.ckpt.index", config=config)

model = TFBertModel.from_pretrained("../../models/pretrained/bert-base-italian-xxl-cased/model.ckpt.index", config=config, from_pt=True)

model = TFBertModel.from_pretrained("../../models/pretrained/bert-base-italian-xxl-cased/model.ckpt.index", config=config, from_pt=True)

model = TFBertModel.from_pretrained("../../models/pretrained/bert-base-italian-xxl-cased", config=config, local_files_only=True)

始终会导致此错误：

404 Client Error: Not Found for url: https://huggingface.co/models/pretrained/bert-base-italian-xxl-cased/model.ckpt.index/resolve/main/tf_model.h5
...
...
OSError: Can't load weights for '../../models/pretrained/bert-base-italian-xxl-cased/model.ckpt.index'. Make sure that:

- '../../models/pretrained/bert-base-italian-xxl-cased/model.ckpt.index' is a correct model identifier listed on 'https://huggingface.co/models'

- or '../../models/pretrained/bert-base-italian-xxl-cased/model.ckpt.index' is the correct path to a directory containing a file named one of tf_model.h5, pytorch_model.bin.

所以我的问题是：如何从这些文件中加载这个预先训练好的BERT模型，并在tensorflow中使用它？

Answer 1

您可以尝试以下代码段在tensorflow中加载dbmdz/bert-base-italian-xxl-cased

from transformers import AutoTokenizer, TFBertModel
model_name = "dbmdz/bert-base-italian-cased"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = TFBertModel.from_pretrained(model_name)

如果要从给定的tensorflow checkpoint加载，可以这样尝试：

model = TFBertModel.from_pretrained("../../models/pretrained/bert-base-italian-xxl-cased/model.ckpt.index", config=config, from_tf=True)