擅长:python、mysql、java
<p>如果你想调整伯特模型本身,你需要修改模型的参数。要做到这一点,您很可能希望使用PyTorch完成您的工作。下面是一些粗略的psuedo代码来说明:</p>
<pre><code>from torch.optim import SGD
model = ... # whatever model you are using
parameters = model.parameters() # or some more specific set of parameters
optimizer = SGD(parameters,lr=.01) # or whatever optimizer you want
optimizer.zero_grad() # boiler-platy pytorch function
input = ... # whatever the appropriate input for your task is
label = ... # whatever the appropriate label for your task is
loss = model(**input, label) # usuall loss is the first item returned
loss.backward() # calculates gradient
optim.step() # runs optimization algorithm
</code></pre>
<p>我省略了所有相关的细节,因为它们非常乏味,而且对于您的具体任务都是特定的。Huggingface有一篇很好的文章介绍了这是一篇更详细的文章<a href="https://huggingface.co/transformers/training.html" rel="nofollow noreferrer">here</a>,当您使用任何pytorch内容时,您肯定会想参考一些pytorch文档。我强烈推荐<a href="https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html" rel="nofollow noreferrer">pytorch blitz</a>,然后再尝试做任何严肃的事情</p>