向web爬网时不存在的项添加值

Question

我正在从IMDB提取评论数据。你知道吗

然而，有时也有没有排名的数据。你知道吗

对于这样的数据，我想将Rank视为0，并将其添加到数组中。你知道吗

我不知道怎么做。你知道吗

你能帮我吗？你知道吗

非常感谢！你知道吗

web image

这样提取时，秩值较低。你知道吗

for star in soup.select('span:has(~ .point-scale)'):
    Star.append(star.text.strip());

for title in soup.find_all('a', {'class' : 'title'}):
    Title.append(title.text.strip())

for content in soup.find_all(True,{'class' :[text show-more__control'
                                   ,'text show-more__control clickable]}):
    Content.append(content.text.strip())
    print(range(len(Content)))

len(list : rank, title, content)

How elements in a site fit into a list.

Answer 1

不是所有的评论都会有评分，所以你需要考虑到这一点：

$ python3 test.py https://www.imdb.com/title/tt5113040/reviews
Got response: 200
Title: The Secret Life of Pets 2
# (8/10) Not as bad as some reviews on here
Let's get this straight it a film made for childre...
  -
ddriver385, 26 May 2019

# (7/10) A Good Film for the kids
This film is a good film to watch with the kids. C...
  -
xxharriet_hobbsxx, 27 May 2019

# (7/10) Worth a watch
Admittedly, it probably wasn't necessary to follow...
  -
MythoGenesis, 24 May 2019

# (No rating) Intense and entertaining
Narratively, the film is not without fault. In par...
  -
TheBigSick, 26 May 2019
...

测试.py

import requests
import sys
import time

from bs4 import BeautifulSoup


def fetch(url):
    with requests.Session() as s:
        r = s.get(url, timeout=5)
        return r


def main(url):
    start_t = time.time()
    resp = fetch(url)
    print(f'Got response: {resp.status_code}')
    html = resp.content
    bs = BeautifulSoup(html, 'html.parser')
    title = bs.find('h3', attrs={'itemprop': 'name'})
    print(f'Title: {title.a.text}')
    reviews = bs.find_all('div', class_='review-container')
    for review in reviews:
        title = review.find('a', class_='title').text.strip()
        rating = review.find('span', class_='rating-other-user-rating')
        if rating:
            rating = ''.join(i.text for i in rating.find_all('span'))
        rating = rating if rating else 'No rating'
        user = review.find('span', class_='display-name-link').text
        date = review.find('span', class_='review-date').text
        content = review.find('div', class_='content').div.text
        print(
            f'# ({rating}) {title}\n'
            f'{content[:50]}...\n'
            f'{"-" * 5}\n'
            f'{user}, {date}\n'
        )
    end_t = time.time()
    elapsed_t = end_t - start_t
    r_time = resp.elapsed.total_seconds()
    print(f'Total: {elapsed_t:.2f}s, request: {r_time:.2f}s')


if __name__ == '__main__':
    if len(sys.argv) > 1:
        url = sys.argv[1]
        main(url)
    else:
        print('URL is required.')
        sys.exit(1)