删除python pandas中的NaN值

Question

数据是来自人口普查数据的成年人收入，行看起来像：

31, Private, 84154, Some-college, 10, Married-civ-spouse, Sales, Husband, White, Male, 0, 0, 38, NaN, >50K
48, Self-emp-not-inc, 265477, Assoc-acdm, 12, Married-civ-spouse, Prof-specialty, Husband, White, Male, 0, 0, 40, United-States, <=50K

我正在尝试从pandas中的CSV文件加载的数据帧中删除所有带有NaNs的行。

>>> import pandas as pd
>>> income = pd.read_csv('income.data')
>>> income['type'].unique()
array([ State-gov,  Self-emp-not-inc,  Private,  Federal-gov,  Local-gov,
    NaN,  Self-emp-inc,  Without-pay,  Never-worked], dtype=object)
>>> income.dropna(how='any') # should drop all rows with NaNs
>>> income['type'].unique()
array([ State-gov,  Self-emp-not-inc,  Private,  Federal-gov,  Local-gov,
    NaN,  Self-emp-inc,  Without-pay,  Never-worked], dtype=object)
    Self-emp-inc, nan], dtype=object) # what??
>>> income = income.dropna(how='any') # ok, maybe reassignment will work?
>>> income['type'].unique()
array([ State-gov,  Self-emp-not-inc,  Private,  Federal-gov,  Local-gov,
    NaN,  Self-emp-inc,  Without-pay,  Never-worked], dtype=object) # what??

我试着用更小的example.csv：

label,age,sex
1,43,M
-1,NaN,F
1,65,NaN

而且dropna()在这里对分类和数值的nan都很有效。怎么回事？我对熊猫还不熟悉，只是在学绳子。

Answer 1

正如我在评论中所写：“NaN”有一个前导空格（至少在您提供的数据中）。因此，需要在read_csv函数中指定na_values参数。

试试这个：

df = pd.read_csv("income.csv",header=None,na_values=" NaN")

这就是为什么您的第二个示例有效，因为这里没有前导空格。