当某些行包含其他格式时，使用mechanize&beautiful跳过表

Question

以下是我要刮的内容（为了便于阅读，缩短了一吨）：

<table class="sortable  row_summable stats_table" id="per_game">
<colgroup><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col><col></colgroup>
<thead>
<tr class="">
  <th data-stat="season" align="center"  class="tooltip sort_default_asc"  tip="If listed as single number, the year the season ended.<br>&#x2605; - Indicates All-Star for league.<br>Only on regular season tables.">Season</th>
  <th data-stat="age" align="center"  class="tooltip sort_default_asc"  tip="Age of Player at the start of February 1st of that season.">Age</th>
</tr>
</thead>
<tbody>
<tr  class="full_table" id="per_game.2009">
   <td align="left" ><a href="/players/r/rondora01/gamelog/2009/">2008-09</a></td>
   <td align="right" >22</td>
</tr>
<tr  class="full_table" id="per_game.2010">
   <td align="left" ><a href="/players/r/rondora01/gamelog/2010/">2009-10</a><span class="bold_text" style="color:#c0c0c0">&nbsp;&#x2605;</span></td>
   <td align="right" >23</td>
</tr>
</tfoot>
</table>

下面是我使用的代码：

^{pr2}$

但是，如果您在HTML中注意到，与第一行相比，第二行还有一个额外的span。它创造了一颗小星星。我的代码运行直到有这个额外参数的任何行都会找到，然后崩溃。关于让代码足够灵活以忽略额外的span块的想法？在

Answer 1

您可以通过以下方法改进代码：首先，将所有标题读入列表，并逐行读取所有参数，使用zip()将每个标题与值匹配，并生成字典：

headers = [item.text for item in table('th')]
for row in table('tr')[1:]:
    params = [item.text.strip() for item in row('td')]
    print dict(zip(headers, params))

印刷品：

^{pr2}$

---

如果要从参数值中去掉不可打印的字符，可以依赖^{}：

import string

params = [filter(lambda x: x in string.printable, item.text) 
          for item in row.find_all('td')] 

{另请参见^

---

输出到csv的完整代码（带播放器名）：

import csv
import string
from bs4 import BeautifulSoup
from mechanize import Browser

mech = Browser()
url = "http://www.basketball-reference.com/players/r/rondora01.html"
RR = mech.open(url)

html = RR.read()
soup = BeautifulSoup(html)
table = soup.find(id="per_game")
player_name = soup.select('div#info_box h1')[0].text.strip()

with open('result.csv', 'w') as f:
    writer = csv.writer(f)

    writer.writerow(['Name'] + [item.text for item in table('th')])

    for row in table('tr')[1:]:
        writer.writerow([player_name] + [filter(lambda x: x in string.printable, item.text)
                                         for item in row('td')])

Answer 2

我建议做一些改变：

1. 由于您对与<a>元素相关联的文本感兴趣，请更改行： col[0].string到col[0].a.string。这样就能解决问题了。

2. 在第一个问题修复后，您将在该表的最后一行遇到一个错误（因为它的结构不同）。要解决这个问题，请将for row in table.findAll('tr')[1:]:更改为for row in table.findAll('tr')[1:-1]:。这样可以跳过最后一行。

进行上述更改：

for row in table.findAll('tr')[1:-1]: 
    col = row.findAll('td')
    season = col[0].a.string
    age = col[1].string
    team = col[2].string
    pos = col[3].string
    games_played = col[4].string
    record = (season, age, team, pos, games_played)
    print "|".join(record)

印刷品：

^{pr2}$