如何仅在使用Python找到特定模式后才能读取csv文件？

Question

因此，我有几个表示一些数据的csv文件，每个文件可能有不同的初始注释行

table_doi: 10.17182/hepdata.52402.v1/t7
name: Table 7
...
ABS(YRAP), < 0.1
SQRT(S) [GeV], 1960
PT [GEV], PT [GEV] LOW, PT [GEV] HIGH, D2(SIG)/DYRAP/DPT [NB/GEV]
67, 62, 72, 6.68
...
613.5, 527, 700, 1.81E-07

我只想读入相关的数据和它们的标题，它们从行开始

PT [GEV], PT [GEV] LOW, PT [GEV] HIGH, D2(SIG)/DYRAP/DPT [NB/GEV]

因此，我想的策略是找到模式PT [GEV]，然后从那里开始阅读

然而，我不知道如何在Python中实现这一点，有人能在这方面帮助我吗

提前谢谢你

---

顺便说一下，我现在的功能是

import os
import glob
import csv

def read_multicolumn_csv_files_into_dictionary(folderpath, dictionary):
    filepath = folderpath + '*.csv'
    files = sorted(glob.glob(filepath))
    for file in files:
        data_set = file.replace(folderpath, '').replace('.csv', '')
        dictionary[data_set] = {}
        with open(file, 'r') as data_file:
            data_pipe = csv.DictReader(data_file)
            dictionary[data_set]['pt'] = []
            dictionary[data_set]['sigma'] = []
            for row in data_pipe:
                dictionary[data_set]['pt'].append(float(row['PT [GEV]']))
                dictionary[data_set]['sigma'].append(float(row['D2(SIG)/DYRAP/DPT [NB/GEV]']))
    return dictionary

这只有在我手动删除csv文件中的初始注释时才有效

Answer 1

签出startswith。此外，您可以在这里找到详细的解释https://cmdlinetips.com/2018/01/3-ways-to-read-a-file-and-skip-initial-comments-in-python/

Answer 2

您可以使用file.tell方法在读取和跳过行时保存文件指针位置，直到找到标题行，此时您可以使用file.seek方法将文件指针重置回标题行的开头，以便csv.DictReader可以将文件的其余部分解析为有效的CSV：

with open(file, 'r') as data_file:
    while True:
        position = data_file.tell()
        line = next(data_file)
        if line.count(',') == 3: # or whatever condition your header line satisfies
            data_file.seek(position) # reset file pointer to the beginning of the header line
            break
    data_pipe = csv.DictReader(data_file)
    ...

Answer 3

假设每个文件都有一行以PT [GEV]开头：

import os
import pandas as pd

...
csvs = []
for file in files:
    with open(file) as f:
        for i, l in enumerate(f):
            if l.startswith('PT [GEV]'):
                csvs.append(pd.read_csv(file, skiprows = i))
                break
df = pd.concat(csvs)