读取以\n\x00结尾的csv行时出现编码问题

Question

我在尝试用pandas读取csv文件时遇到了一些问题，因为只有第一行正确地解析了日期（下一行出现在NaN或NaT）。我已尝试直接打开该文件以查看其外观：

f = open('20191122.csv', "r", encoding='ascii')
f.read(300)

前300个字符表示行以\n\x00结尾：

'20191122 21:29,1,59,-999,42,-999.9,-999.9,37,34,1,0.0,0.4,0.4,0.4,0,0,0,0,0,10.1,9.6,0.0,0,33.7,36.0,75.4,29.6,14.0,59.5,32.7,6.7,6.8,0.2,-\n\x0020191122 21:30,1,59,-999,42,-999.9,-999.9,37,34,1,0.0,0.4,0.4,0.4,0,0,0,0,0,10.0,9.8,0.0,0,33.4,35.9,74.9,29.0,13.9,59.6,32.7,6.6,6.6,0.2,-\n\x0020191122 21:30,1,5'

逐行拉动时，第一行可以：

data = f.readlines()
data[0]
'20191122 21:29,1,59,-999,42,-999.9,-999.9,37,34,1,0.0,0.4,0.4,0.4,0,0,0,0,0,10.1,9.6,0.0,0,33.7,36.0,75.4,29.6,14.0,59.5,32.7,6.7,6.8,0.2,-\n'

但其余行以\x00开头，因此无法解析日期：

data[1]
'\x0020191122 21:30,1,59,-999,42,-999.9,-999.9,37,34,1,0.0,0.4,0.4,0.4,0,0,0,0,0,10.0,9.8,0.0,0,33.4,35.9,74.9,29.0,13.9,59.6,32.7,6.6,6.6,0.2,-\n'

所以问题似乎与编码有关？我在csv文件上尝试了chardet package，得到了相同的结果：ascii with confidence 1.0，但是我似乎找不到解决\x00问题的答案。。。你知道吗

Answer 1

以UTF-16结尾的行是“\n\x00”。 f、 readlines（）将\n作为行尾。你知道吗

所以试试看：

data = open(...).read()
data.encode().decode("utf-16")