使用python或C#删除csv中的行？

Question

我有一个csv文件，它会这样重复：

"col1", "col2","col3"
Integer, Integer, Varchar(50)
7, 8, 21554
24, 25, 36544
"col1", "col2","col3"
Integer, Integer, Varchar(50)
7, 8, 21554
24, 25, 36544

如何剥离复制的部分，包括后面的头、数据类型行和数据行？
我只想要这个：

"col1", "col2","col3"
Integer, Integer, Varchar(50)
7, 8, 21554
24, 25, 36544

Answer 1

我们甚至不需要为此使用csv模块。我们将记住文件的第一行是什么，然后写几行，直到我们再次看到它，此时我们将停止，截断文件。你知道吗

with open('infile.csv', newline='') as infile, open('outfile.csv', 'w+',  newline='')as outfile:
     first = next(infile)
     outfile.write(first)
     for line in infile:
         if line == first:
             break
         outfile.write(line)

Answer 2

您可以使用csv模块（假设python2.x）这样做：

import csv

seen = set()
with open('duplicates.csv', 'rb') as infile, open('cleaned.csv', 'wb') as outfile:
    reader = csv.reader(infile, skipinitialspace=True)
    writer = csv.writer(outfile)
    for row in (tuple(row) for row in reader):
        if row not in seen:
            writer.writerow(row)
            seen.add(row)

print('done')