我有这样一个管道模型:
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer, make_column_transformer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
# define preprocessor
preprocess = make_column_transformer(
(StandardScaler(), ['attr1', 'attr2', 'attr3', 'attr4', 'attr5',
'attr6', 'attr7', 'attr8', 'attr9']),
(OneHotEncoder(categories='auto'), ['attrcat1', 'attrcat2'])
)
# define train and test datasets
X_train, X_test, y_train, y_test =
train_test_split(features, target, test_size=0.3, random_state=0)
当我在没有过度采样的情况下执行管道时,我得到:
# don't do over-sampling in this case
os_X_train = X_train
os_y_train = y_train
print('Training data is type %s and shape %s' % (type(os_X_train), os_X_train.shape))
logreg = LogisticRegression(penalty='l2',solver='lbfgs',max_iter=1000)
model = make_pipeline(preprocess, logreg)
model.fit(os_X_train, np.ravel(os_y_train))
print("The coefficients shape is: %s" % logreg.coef_.shape)
print("Model coefficients: ", logreg.intercept_, logreg.coef_)
print("Logistic Regression score: %f" % model.score(X_test, y_test))
输出为:
Training data is type <class 'pandas.core.frame.DataFrame'> and shape (87145, 11)
The coefficients shape is: (1, 47)
Model coefficients: [-7.51822124] [[ 0.10011794 0.10313989 ... -0.14138371 0.01612046 0.12064405]]
Logistic Regression score: 0.999116
这意味着我得到了一个训练集的47个模型系数87145个样本,这是有意义的考虑到定义的预处理。OneHotEncoder
在attrcat1
和attrcat2
上工作,它们总共有31+7个类别,加上38列,加上我已经拥有的9列,总共有47个特性。你知道吗
现在如果我也这么做,但是这次用SMOTE进行过度采样,像这样:
from imblearn.over_sampling import SMOTE
# balance the classes by oversampling the training data
os = SMOTE(random_state=0)
os_X_train,os_y_train=os.fit_sample(X_train, y_train.ravel())
os_X_train = pd.DataFrame(data=os_X_train, columns=X_train.columns)
os_y_train = pd.DataFrame(data=os_y_train, columns=['response'])
输出变为:
Training data is type <class 'pandas.core.frame.DataFrame'> and shape (174146, 11)
The coefficients shape is: (1, 153024)
Model coefficients: [12.02830778] [[ 0.42926969 0.14192505 -1.89354062 ... 0.008847 0.00884372 -8.15123962]]
Logistic Regression score: 0.997938
在这种情况下,我得到了大约两倍的训练样本大小来平衡我想要的反应类,但我的逻辑回归模型爆炸到153024系数。这没有任何意义。。。知道为什么吗?你知道吗
好吧,我找到了这个问题的罪魁祸首。问题是SMOTE将所有特性列转换为float(包括这两个分类特性)。因此,当对column types float应用columns transformer
OneHotEncoder
时,会将列数分解为样本数,即它将相同float值的每次出现视为不同的类别。你知道吗解决方案只是在运行管道之前将这些分类列类型转换回int:
相关问题 更多 >
编程相关推荐