具有Logloss和L2正则化的SGD分类器，使用SGD而不使用sklearn python

Question

我正在用python处理SGD手动实现中的一个赋值问题。我被困在dw导数函数上

import numpy as np 
import pandas as pd 
from sklearn.datasets import make_classification

X, y = make_classification(n_samples=50000, n_features=15, n_informative=10, n_redundant
=5,n_classes=2, weights=[0.7], class_sep=0.7, random_state=15)

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=15)

def initialize_weights(dim):
    w=np.zeros_like(dim)
    b=0
    return w,b
dim=X_train[0] 
w,b = initialize_weights(dim)
print('w =',(w))
print('b =',str(b))

import math
def sigmoid(z):
''' In this function, we will return sigmoid of z'''
# compute sigmoid(z) and return
    test_neg_int = -z
    sig_z=1/(1+(math.exp(test_neg_int )))

    return sig_z

import math
def logloss(y_true,y_pred):
'''In this function, we will compute log loss '''
    n=len(y_true)
    loss= -(1.0/n)*sum([y_true[i]*math.log(y_pred[i],10)+ (1.0-y_true[i])*math.log(1.0-y_pred[i],10) 
    for i in range(len(y_true))])
    return loss

def gradient_dw(x,y,w,b,alpha,N):
'''In this function, we will compute the gardient w.r.to w '''
    for n in range(0,len(x)):
        dw=[] 
 # y=0, x= 15 array values, w= 15 array values of 0, b=0, alpha=0.0001, n=len(X_train)=37500
        lambda_val = 0.01
        d = x[n]*((y-alpha*((w.T)*x[n]+b)) - ((lambda_val*w)/N))
        dw.append(d)
    print (dw)

def grader_dw(x,y,w,b,alpha,N):
    grad_dw=gradient_dw(x,y,w,b,alpha,N)
    assert(np.sum(grad_dw)==2.613689585)
    return True
grad_x=np.array([-2.07864835,  3.31604252, -0.79104357, -3.87045546, -1.14783286,
   -2.81434437, -0.86771071, -0.04073287,  0.84827878,  1.99451725,
    3.67152472,  0.01451875,  2.01062888,  0.07373904, -5.54586092])
grad_y=0
grad_w,grad_b=initialize_weights(grad_x)
alpha=0.0001
N=len(X_train)
grader_dw(grad_x,grad_y,grad_w,grad_b,alpha,N)

结果我得到了

[array([-0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0., -0.,
     -0., -0.])]
  ---------------------------------------------------------------------------
 AssertionError                            Traceback (most recent call last)
<ipython-input-168-a3ed60706dc2> in <module>
     10 alpha=0.0001
     11 N=len(X_train)
---> 12 grader_dw(grad_x,grad_y,grad_w,grad_b,alpha,N)

<ipython-input-168-a3ed60706dc2> in grader_dw(x, y, w, b, alpha, N)
      1 def grader_dw(x,y,w,b,alpha,N):
      2     grad_dw=gradient_dw(x,y,w,b,alpha,N)
----> 3     assert(np.sum(grad_dw)==2.613689585)
      4     return True
      5 grad_x=np.array([-2.07864835,  3.31604252, -0.79104357, -3.87045546, -1.14783286,

AssertionError: 

预期结果：

True

你能告诉我我对梯度函数的理解是否有误吗？我试图应用这个公式：

dw(t) = xn * (yn − σ * (((w(t))Transpose) * xn + b(t))) − (λ * w(t)) / N)

我试图在gradient_dw函数中计算gradient w.r.t‘w’，以便稍后在主代码中使用它。我不理解的是w是一个0的数组，y=0，所以当我们应用dw（t）公式并返回dw时，我们很可能得到一个0的数组，但为什么它会说 “断言（np.sum（grad_dw）==2.613689585）”。我们怎么可能得到2.613689585

Answer 1

你走错路了

1. 在迭代过程中，我们以随机梯度下降的方式迭代“n”个点（批量大小为1），而不是“d”维。在这里，您将遍历“d”维

2. 梯度x=np.数组（[-2.07864835,3.31604252，-0.79104357，-3.87045546，-1.14783286， -2.81434437, -0.86771071, -0.04073287, 0.84827878, 1.99451725, 3.67152472,0.01451875,2.01062888,0.07373904，-5.54586092]）

它是一个具有15个维度的单点。 因此，请修改您的查询，如下所示。这会奏效的

    def gradient_dw(x,y,w,b,alpha,N):
       '''In this function, we will compute the gardient w.r.to w '''
       dw=x * (y-sigmoid(np.dot(w.T,x)+b)) -(alpha * w)/N

       return dw