如何解析网页中的特定节数据？

Question

我正在用靓汤解析网页的具体内容，你能告诉我，我怎么能做到这一点？ 代码：

import re
import pytz
import requests
import datetime
from flask import url_for
from bs4 import BeautifulSoup
from urllib.parse import urljoin


link = "http://www.espncricinfo.com/series/_/id/8038/season/2018/icc-world-cup-qualifiers/"

r = requests.get(link)
bigbash_article_html = r.text

soup = BeautifulSoup(bigbash_article_html, "html.parser")


details = soup.find("div",{"class":"module-list performers"})
bigbash_article_dict = {}


for div in details:
    image_div = div.find("div", {"class": "img-container player"})

我不知道如何进一步进行，我期望输出如下

预期产量：

最佳得分手：

[{'playerimage':'http://a.espncdn.com/combiner/i?img=/i/headshots/cricket/players/default-player-logo-500.png&h=55&w=40&scale=crop&transparent=true','playername':'TP Ura','player-details':'PNG, Right-hand bat','runs':'188','innings':'2','Average':'94.00'},..............................................................................................}]

另一列也一样 最佳小门接受者：

[{'playerimage':'http://a.espncdn.com/combiner/i?img=/i/headshots/cricket/players/default-player-logo-500.png&h=55&w=40&scale=crop&transparent=true','playername':'Ehsan Khan','player-details':'HKG, Right-arm offbreak','wickets':'9','innings':'3','Average':'12.55'},..............................................................................................}]

Answer 1

首先，你找错标签了。所需内容位于<ul class="module-list performers">内，而不是具有相同类名的div标记。你知道吗

Top Run Scorers表位于<div id="r-0">标记内。每个播放器都位于li标记中。您可以在li标记中获得播放器的所有详细信息。你知道吗

我将向您展示如何获得最佳得分手的图像、姓名和球员详细信息。你知道吗

r = requests.get('http://www.espncricinfo.com/series/_/id/8038/season/2018/icc-world-cup-qualifiers')
soup = BeautifulSoup(r.text, 'lxml')

top_run_scorers = []
for player in soup.find('div', id='r-0').find_all('li'):
    image = player.find('img')['src']
    info = player.find('div', class_='content-meta')
    name = info.find('a').text
    details = info.p.contents[-1]
    top_run_scorers.append({'playerimage': image, 'playername': name, 'player-details': details})

print(top_run_scorers)

输出：

[{'player-details': ', PNG, Right-hand bat',
  'playerimage': 'http://a.espncdn.com/combiner/i?img=/i/headshots/cricket/players/default-player-logo-500.png&h=55&w=40&scale=crop&transparent=true',
  'playername': 'TP Ura'},
 {'player-details': ', AFG, Right-hand bat',
  'playerimage': 'http://a.espncdn.com/combiner/i?img=/i/headshots/cricket/players/25913.png&h=55&w=40&scale=crop&transparent=true',
  'playername': 'Mohammad Nabi'},
 {'player-details': ', WI, Left-hand bat',
  'playerimage': 'http://a.espncdn.com/combiner/i?img=/i/headshots/cricket/players/default-player-logo-500.png&h=55&w=40&scale=crop&transparent=true',
  'playername': 'SO Hetmyer'}]

Answer 2

选择元素中所有类名为sub-module和performers的列表项，然后分析每个列表项的播放器详细信息。e、 g.

r = requests.get("http://www.espncricinfo.com/series/_/id/8038/season/2018/icc-world-cup-qualifiers/"
)

soup = BeautifulSoup(r.text, "html.parser")

toprunners = soup.select(".sub-module.performers li")

def player(li):
    name_and_details = li.select_one('p')
    name = name_and_details.a
    details = name.nextSibling
    stats = li.select_one('.overall-stats p')
    img = li.select_one('.focus-image')

    return {
        'player_name': name.text,
        'player_details': details.strip(', '),
        'player_image': img.attrs['src'],
        'runs': name_and_details.nextSibling.text,
        'innings': stats.span.text,
        'average': stats.nextSibling.span.text,
    }

players = [player(li) for li in toprunners]

In[2]: print(players)

[{'player_name': 'TP Ura', 'player_details': 'PNG, Right-hand bat', 'player_image': 'http://a.espncdn.com/combiner/i?img=/i/headshots/cricket/players/default-player-logo-500.png&h=55&w=40&scale=crop&transparent=true', 'runs': '188', 'innings': '2', 'average': '94.00'}, {'player_name': 'Mohammad Nabi', 'player_details': 'AFG, Right-hand bat', 'player_image': 'http://a.espncdn.com/combiner/i?img=/i/headshots/cricket/players/25913.png&h=55&w=40&scale=crop&transparent=true', 'runs': '181', 'innings': '3', 'average': '60.33'}, {'player_name': 'SO Hetmyer', 'player_details': 'WI, Left-hand bat', 'player_image': 'http://a.espncdn.com/combiner/i?img=/i/headshots/cricket/players/default-player-logo-500.png&h=55&w=40&scale=crop&transparent=true', 'runs': '171', 'innings': '3', 'average': '57.00'}, {'player_name': 'Ehsan Khan', 'player_details': 'HKG, Right-arm offbreak', 'player_image': 'http://a.espncdn.com/combiner/i?img=/i/headshots/cricket/players/default-player-logo-500.png&h=55&w=40&scale=crop&transparent=true', 'runs': '9', 'innings': '3', 'average': '12.55'}, {'player_name': 'Mujeeb Ur Rahman', 'player_details': 'AFG, Right-arm offbreak', 'player_image': 'http://a.espncdn.com/combiner/i?img=/i/headshots/cricket/players/default-player-logo-500.png&h=55&w=40&scale=crop&transparent=true', 'runs': '8', 'innings': '3', 'average': '15.25'}, {'player_name': 'JO Holder', 'player_details': 'WI, Right-arm medium-fast', 'player_image': 'http://a.espncdn.com/combiner/i?img=/i/headshots/cricket/players/391485.png&h=55&w=40&scale=crop&transparent=true', 'runs': '7', 'innings': '3', 'average': '21.28'}]