我正在培训关于LeNet CNN模型的CIFAR10数据集。我正在谷歌Colab上使用PyTorch。只有当我使用带有model.parameters()的Adam优化器作为唯一参数时,代码才会运行。但是,当我改变我的优化器或使用权重衰减参数时,在所有时间段内,精度都保持在10%。我不明白为什么会这样
# CNN Model - LeNet
class LeNet_ReLU(nn.Module):
def __init__(self):
super().__init__()
self.cnn_model = nn.Sequential(nn.Conv2d(3,6,5),
nn.ReLU(),
nn.AvgPool2d(2, stride=2),
nn.Conv2d(6,16,5),
nn.ReLU(),
nn.AvgPool2d(2, stride=2))
self.fc_model = nn.Sequential(nn.Linear(400, 120),
nn.ReLU(),
nn.Linear(120,84),
nn.ReLU(),
nn.Linear(84,10))
def forward(self, x):
x = self.cnn_model(x)
x = x.view(x.size(0), -1)
x = self.fc_model(x)
return x
# Importing dataset and creating dataloader
batch_size = 128
trainset = torchvision.datasets.CIFAR10(root='./data', train=True, download=True,
transform=transforms.ToTensor())
trainloader = utils_data.DataLoader(trainset, batch_size=batch_size, shuffle=True)
testset = torchvision.datasets.CIFAR10(root='./data', train=False, download=True,
transform=transforms.ToTensor())
testloader = utils_data.DataLoader(testset, batch_size=batch_size, shuffle=False)
# Creating instance of the model
net = LeNet_ReLU()
# Evaluation function
def evaluation(dataloader):
total, correct = 0, 0
for data in dataloader:
inputs, labels = data
outputs = net(inputs)
_, pred = torch.max(outputs.data, 1)
total += labels.size(0)
correct += (pred==labels).sum().item()
return correct/total * 100
# Loss function and optimizer
loss_fn = nn.CrossEntropyLoss()
opt = optim.Adam(net.parameters(), weight_decay = 0.9)
# Model training
loss_epoch_arr = []
max_epochs = 16
for epoch in range(max_epochs):
for i, data in enumerate(trainloader, 0):
inputs, labels = data
outputs = net(inputs)
loss = loss_fn(outputs, labels)
loss.backward()
opt.step()
opt.zero_grad()
loss_epoch_arr.append(loss.item())
print('Epoch: %d/%d, Test acc: %0.2f, Train acc: %0.2f'
% (epoch,max_epochs, evaluation(testloader), evaluation(trainloader)))
plt.plot(loss_epoch_arr)
权重衰减机制为高值wieghts设置惩罚,即通过将权重之和乘以您给出的
weight_decay
参数,将权重限制为具有相对较小的值。这可以看作是一个二次正则化项当传递较大的
weight_decay
值时,您可能会过于严格您的网络,阻止它学习,这可能是它有10%的准确率的原因,这与根本不学习和猜测答案有关(因为您有10个类,您会收到10%的acc,而输出根本不是您输入的函数)解决方案是使用不同的值,训练
weight_decay
或该区域的其他值。请注意,当您达到接近零的值时,您应该得到更接近初始序列的结果,而不使用权重衰减希望有帮助
相关问题 更多 >
编程相关推荐