从两个数据帧中获取不常见的记录

Question

我试图得到两个数据帧之间的差异。因此，我想删除不同的记录数，并从中生成不同的数据帧

train_abusive=pd.read_csv('train_abusive.csv',low_memory=False)
train_non_abusive=pd.read_csv('train_non_abusive.csv',low_memory=False)
print len(train_abusive),len(train_non_abusive)

val_abusive=train_abusive.sample(frac=0.1)
val_non_abusive=train_non_abusive.sample(frac=0.2)

train_abusive=pd.concat([val_abusive,train_abusive],ignore_index=True)
train_abusive=train_abusive.drop_duplicates(keep=False)

train_non_abusive=pd.concat([val_non_abusive,train_non_abusive],ignore_index=True)
train_non_abusive=train_non_abusive.drop_duplicates(keep=False)

print len(train_abusive),len(train_non_abusive)

它提供以下输出：

50000 200000
44596 155010

但这算不出来。我不知道为什么。你知道吗

Answer 1

编辑：如果你只想比较2个数据帧，你可以使用断言。你知道吗

train_abusive=pd.read_csv('train_abusive.csv',low_memory=False)
train_non_abusive=pd.read_csv('train_non_abusive.csv',low_memory=False)

from pandas.util.testing import assert_frame_equal
assert_frame_equal(train_abusive, train_non_abusive)

另外，我在另一个post中看到了Tom Chapin的答案，你可能会感兴趣。你知道吗

def get_different_rows(train_abusive, train_non_abusive):
    """Returns just the rows from the new dataframe that differ from the source dataframe"""
    merged_df = train_abusive.merge(train_non_abusive, indicator=True, how='outer')
    changed_rows_df = merged_df[merged_df['_merge'] == 'right_only']
    return changed_rows_df.drop('_merge', axis=1)