来自一列的CSV读取器，该列的元素表示列表/数组不是一个值

Question

以下内容来自我的文件.csv你知道吗

  1st        2nd     3rd      4th                     5th
2061100   10638650  -8000     25         [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
2061800   10639100  -8100     26         [2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0]
2061150   10638750  -8250     25         [3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0]
2061650   10639150  -8200     25         [4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0]
2061350   10638800  -8250     3          [5.0, 5.0, 5.0]
2060950   10638700  -8000     1          [1.0]
2061700   10639100  -8100     11         [2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0]
2061050   10638800  -8250     6          [3.0, 3.0, 3.0, 3.0, 3.0, 3.0]
2061500   10639150  -8200     1          [4.0]
2061250   10638850  -8150     16         [5.0, 5.0, 5.0, 5.0]

我的当前代码：

from numpy import genfromtxt
mydata = genfromtxt('myfile.csv', delimiter=',')
arr = np.array(mydata)
col5 = arr[:,4]

但是，我想把第5列作为一个列表来读，然后从列表中读取所有元素来做进一步的计算。我该怎么办？你知道吗

Answer 1

Pandas可以读取固定宽度的文件（与制表符/逗号分隔的文件不同），如您的文件：

import pandas as pd
import ast

df = pd.read_fwf('test.txt', colspecs=[(41,100)])['5th']\
       .apply(lambda x: ast.literal_eval(x))

你会得到：

>>> df

0         [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
1         [2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0]
2         [3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0]
3         [4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0]
4                             [5.0, 5.0, 5.0]
5                                       [1.0]
6    [2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0]
7              [3.0, 3.0, 3.0, 3.0, 3.0, 3.0]
8                                       [4.0]
9                        [5.0, 5.0, 5.0, 5.0]
Name: 5th, dtype: object

Answer 2

如果列之间的空白是制表符：

import csv, ast, pprint
result = list()
with open('in.txt') as in_file:
    reader = csv.reader(in_file, delimiter = '\t')
    for line in reader:
        line[:4] = map(int, line[:4])
        line[4] = ast.literal_eval(line[4])
        result.append(line)    

pprint.pprint(result)

>>> 
[[2061100, 10638650, -8000, 25, [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]],
 [2061800, 10639100, -8100, 26, [2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0]],
 [2061150, 10638750, -8250, 25, [3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0]],
 [2061650, 10639150, -8200, 25, [4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0]],
 [2061350, 10638800, -8250, 3, [5.0, 5.0, 5.0]],
 [2060950, 10638700, -8000, 1, [1.0]],
 [2061700, 10639100, -8100, 11, [2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0]],
 [2061050, 10638800, -8250, 6, [3.0, 3.0, 3.0, 3.0, 3.0, 3.0]],
 [2061500, 10639150, -8200, 1, [4.0]],
 [2061250, 10638850, -8150, 16, [5.0, 5.0, 5.0, 5.0]]]
>>>

这个主题的一个变体：

with open('in.txt') as in_file:
    reader = csv.reader(in_file, delimiter = '\t')
    result = [[ast.literal_eval(item) for item in line] for line in reader]

Answer 3

我想我很想手动完成这项工作：

with open(fn) as f:
    header=next(f).strip()
    print(header)
    for row in f:
        row=row.rstrip()
        lp,_,rp=row.partition('[')
        rp=rp.strip(']')
        lp_data=list(map(int, lp.split()))
        rp_data=list(map(float, rp.split(',')))
        print(lp_data+[rp_data])

印刷品：

1st        2nd     3rd      4th                     5th
[2061100, 10638650, -8000, 25, [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]]
[2061800, 10639100, -8100, 26, [2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0]]
[2061150, 10638750, -8250, 25, [3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0]]
[2061650, 10639150, -8200, 25, [4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0]]
[2061350, 10638800, -8250, 3, [5.0, 5.0, 5.0]]
[2060950, 10638700, -8000, 1, [1.0]]
[2061700, 10639100, -8100, 11, [2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0]]
[2061050, 10638800, -8250, 6, [3.0, 3.0, 3.0, 3.0, 3.0, 3.0]]
[2061500, 10639150, -8200, 1, [4.0]]
[2061250, 10638850, -8150, 16, [5.0, 5.0, 5.0, 5.0]]