如何在pandas中使用sklearn-fit_转换并返回dataframe而不是numpy数组?

2024-06-16 11:29:19 发布

您现在位置:Python中文网/ 问答频道 /正文

我想将scaling(使用sklearn.preprocessing中的StandardScaler())应用于pandas数据帧。下面的代码返回一个numpy数组,因此我丢失了所有的列名和索引。这不是我想要的。

features = df[["col1", "col2", "col3", "col4"]]
autoscaler = StandardScaler()
features = autoscaler.fit_transform(features)

我在网上找到的一个“解决方案”是:

features = features.apply(lambda x: autoscaler.fit_transform(x))

它似乎起作用,但会导致一个弃用警告:

/usr/lib/python3.5/site-packages/sklearn/preprocessing/data.py:583: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.

因此我试图:

features = features.apply(lambda x: autoscaler.fit_transform(x.reshape(-1, 1)))

但这给了我们:

Traceback (most recent call last): File "./analyse.py", line 91, in features = features.apply(lambda x: autoscaler.fit_transform(x.reshape(-1, 1))) File "/usr/lib/python3.5/site-packages/pandas/core/frame.py", line 3972, in apply return self._apply_standard(f, axis, reduce=reduce) File "/usr/lib/python3.5/site-packages/pandas/core/frame.py", line 4081, in _apply_standard result = self._constructor(data=results, index=index) File "/usr/lib/python3.5/site-packages/pandas/core/frame.py", line 226, in init mgr = self._init_dict(data, index, columns, dtype=dtype) File "/usr/lib/python3.5/site-packages/pandas/core/frame.py", line 363, in _init_dict dtype=dtype) File "/usr/lib/python3.5/site-packages/pandas/core/frame.py", line 5163, in _arrays_to_mgr arrays = _homogenize(arrays, index, dtype) File "/usr/lib/python3.5/site-packages/pandas/core/frame.py", line 5477, in _homogenize raise_cast_failure=False) File "/usr/lib/python3.5/site-packages/pandas/core/series.py", line 2885, in _sanitize_array raise Exception('Data must be 1-dimensional') Exception: Data must be 1-dimensional

如何对pandas数据帧应用缩放,使数据帧保持完整?如果可能的话不复制数据。


Tags: inpycorepandasdatalibpackagesusr
2条回答
import pandas as pd    
from sklearn.preprocessing import StandardScaler

df = pd.read_csv('your file here')
ss = StandardScaler()
df_scaled = pd.DataFrame(ss.fit_transform(df),columns = df.columns)

df_scaled将是“相同”的数据帧,现在只有scaled值

可以使用^{}将数据帧转换为numpy数组。随机数据集上的示例:

编辑: 根据上述as_matrix()文档的最后一句话,将as_matrix()更改为values,(不会更改结果):

Generally, it is recommended to use ‘.values’.

import pandas as pd
import numpy as np #for the random integer example
df = pd.DataFrame(np.random.randint(0.0,100.0,size=(10,4)),
              index=range(10,20),
              columns=['col1','col2','col3','col4'],
              dtype='float64')

注:指数为10-19:

In [14]: df.head(3)
Out[14]:
    col1    col2    col3    col4
    10  3   38  86  65
    11  98  3   66  68
    12  88  46  35  68

现在fit_transform数据帧获得scaled_featuresarray

from sklearn.preprocessing import StandardScaler
scaled_features = StandardScaler().fit_transform(df.values)

In [15]: scaled_features[:3,:] #lost the indices
Out[15]:
array([[-1.89007341,  0.05636005,  1.74514417,  0.46669562],
       [ 1.26558518, -1.35264122,  0.82178747,  0.59282958],
       [ 0.93341059,  0.37841748, -0.60941542,  0.59282958]])

将缩放数据分配给数据帧(注意:使用indexcolumns关键字参数保留原始索引和列名:

scaled_features_df = pd.DataFrame(scaled_features, index=df.index, columns=df.columns)

In [17]:  scaled_features_df.head(3)
Out[17]:
    col1    col2    col3    col4
10  -1.890073   0.056360    1.745144    0.466696
11  1.265585    -1.352641   0.821787    0.592830
12  0.933411    0.378417    -0.609415   0.592830

编辑2:

遇到了sklearn-pandas包。它的重点是使scikit更易于与熊猫一起使用。sklearn-pandas在需要对DataFrame(一种更常见的方案)的列子集应用多种类型的转换时尤其有用。这是有文档记录的,但这是实现我们刚才执行的转换的方法。

from sklearn_pandas import DataFrameMapper

mapper = DataFrameMapper([(df.columns, StandardScaler())])
scaled_features = mapper.fit_transform(df.copy(), 4)
scaled_features_df = pd.DataFrame(scaled_features, index=df.index, columns=df.columns)

相关问题 更多 >