使用Beauty Soup在元内容中查找对应的兄弟姐妹

Question

原始html如下所示：

  <meta content="A" name="citation_author"/>
  <meta content="Axxxxx" name="citation_author_institution"/>
  <meta content="Aorcid" name="citation_author_orcid"/>
  <meta content="B" name="citation_author"/>
  <meta content="Bxxx1" name="citation_author_institution"/>
  <meta content="Bxxx2" name="citation_author_institution"/>
  <meta content="C" name="citation_author"/>
  <meta content="D" name="citation_author"/>
  <meta content="Dorcid" name="citation_author_orcid">
  <meta content="E" name="citation_author"/>
  <meta content="Eyyyyy" name="citation_email"/>

输出结果应如下所示：

^{tb1}$

我正在使用Python 3.7

我尝试使用“find_all”获取所有的名字，然后使用find_next_sibling（'meta'，'name'：'xxx'）获取特定作者的对应列。但就拿ORCID为例，因为作者BC没有ORCID，所以我编写的代码将返回D的ORCID

AU_names = soup.find_all('meta', {'name': 'citation_author'})
for name in AU_names:
    AU_name = name.attrs['content']
    ORCID = name.find_next_sibling('meta', {'name': 'citation_author_orcid'})
    ORCID = ORCID.attrs['content'] if ORCID else ''
    print(AU_name, ORCID)

有人能帮我吗？谢谢你

Answer 1

这是个有趣的问题

试着做以下几点：

orc = """<doc>[your html above]</doc>""" #we need to wrap the html in a root element

from bs4 import BeautifulSoup as bs
import pandas as pd

soup = bs(orc,'lxml')
targets = ["citation_author_institution","citation_author_orcid","citation_email"]
entries = []
for met in soup.select('meta[name=citation_author]'): 
    person = []
    for m in met.findAllNext():
        row  = []
        if (m.attrs['name'])=="citation_author":
            break
        else:
            row.append(m)
        person.append(row)
    if len(person)==0: #this to account for authors like C, with no data
        entry = [met.attrs['content'],'NA','NA','NA']
        entries.append(entry)
    else:
        entry = [met.attrs['content']]
        for target in targets:  
            mini_t = []
            for p in person:                
                if p[0].attrs['name']==target:
                    #mini_t.append(p[0].attrs['content']) #old version
                    mini_t.append(p[0].attrs['content']+' ')#edited version
            entry.append(mini_t)
        for tf in entry:
            if len(tf)==0: #this is to account for missing data
                tf.append('NA')
        #entry =[' '.join(tf) for tf in entry] #convert all inside lists to text - old version
        entry =[''.join(tf) for tf in entry] #edited version
        entries.append(entry)
#finally, create the dataframe           
columns = ['name','inst','orcid','email']
pd.DataFrame(entries,columns=columns)

输出：

name    inst    orcid   email
0   A   Axxxxx  Aorcid  NA
1   B   Bxxx1 Bxxx2 NA  NA
2   C   NA      NA     NA
3   D   NA     Dorcid   NA
4   E   NA     NA      Eyyyyy