逻辑回归在python中不起作用

Question

我试图用python实现logistic回归科学优化我在下面描述了一个错误

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import scipy.optimize as sci

data=pd.read_csv("data.txt")
X=data.iloc[:,:-1]
y=data.iloc[:,-1]
admitted=data.loc[y==1]
not_admitted=data.loc[y==0]

plotting the data
plt.scatter(admitted.iloc[:,0],admitted.iloc[:,1],color='red',marker='X')
plt.scatter(not_admitted.iloc[:,0],not_admitted.iloc[:,1],color='green',marker='o')
plt.show()

X=np.c_[np.ones((X.shape[0],1)),X]
y=y[:,np.newaxis]
theta=np.zeros((X.shape[1],1))

def sigmoid(x):
    return 1/1+np.exp(-x)

def input(theta,x):
    return np.dot(x,theta)

def probablity(theta,x):
    return sigmoid(input(theta,x))

def cost_func(self,theta,x,y):
    m=x.shape[0]
    cost=-(1/m)*sum(y*np.log(probablity(theta,x))+(1-y)*np.log(1-probablity(theta,x)))
    return cost

def gradient(theta,x,y):
    m=x.shape[0]
    grad=(1/m)*np.dot(x.T,probablity(theta,x)-y)

def fit(self, x, y, theta):
    opt_weights=sci.fmin_tnc(func=cost_func,x0=theta,fprime=gradient,args=(x,y.flatten()))
    return opt_weights[0]

parameters = fit(X, y, theta)

print('The value of parameters are '+str(parameters))

我得到的错误是

  RuntimeWarning: invalid value encountered in log
  total_cost = -(1 / m) * np.sum(y * np.log(probability(theta, x)) + (1 - y) * np.log(1 - probability(theta, x)))
  NIT   NF   F                       GTG
    0    1                    NAN   1.52331587E+04
tnc: fscale = 0.00810224
    0   66                    NAN   1.52331587E+04
tnc: Linear search failed

我知道对数不能作为负值，但我从来没有得到过八度音阶的错误，有人能帮我吗

Answer 1

这是因为你定义了你的乙状结肠，没有任何边界。数字或小数字，你会得到+inf和-inf，这会导致你的问题。它可以被称为量化问题，或者sigmoid的估计。你知道吗

正如前面提到的here，您可以像这样修改sigmoid函数（我只是在这里复制该解决方案），您的问题将得到解决：

def sigmoid(x):
    "Numerically-stable sigmoid function."
    if x >= 0:
        z = np.exp(-x)
        return 1 / (1 + z)
    else:
        z = np.exp(x)
        return z / (1 + z)