当我使用beatiful soup获取url的id时，它返回none，我如何才能获取其id的内容

Question

当我使用Beautiful Soup获取url的id时，它返回none，我如何才能获取其id的内容

import requests
import json
from bs4 import BeautifulSoup

URL = 'https://www.amazon.com/Ozeri-Digital-Multifunction-Kitchen-Elegant/dp/B01LAVADW2?pf_rd_p=3e7c8265-9bb7-5ab2-be71-1af95f06a1ad&pf_rd_r=52Z7DNQGKGV31B114R1K&pd_rd_wg=IAKey&ref_=pd_gw_ri&pd_rd_w=rDONb&pd_rd_r=b6b3cf66-c4a8-449a-8676-9027e8922b96'
headers = {"User-Agent":'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.100 Safari/537.36'}

page = requests.get(URL)

soup = BeautifulSoup(page.content, 'html.parser')

title = soup.find(id="productTitle")

print(title)

Answer 1

您已经创建了一个变量头，但没有将其添加到请求中，而且，您没有检查请求-响应状态代码（503）

修复代码时，应该如下所示：

import requests
import json
from bs4 import BeautifulSoup

URL = 'https://www.amazon.com/Ozeri-Digital-Multifunction-Kitchen-Elegant/dp/B01LAVADW2?pf_rd_p=3e7c8265-9bb7-5ab2-be71-1af95f06a1ad&pf_rd_r=52Z7DNQGKGV31B114R1K&pd_rd_wg=IAKey&ref_=pd_gw_ri&pd_rd_w=rDONb&pd_rd_r=b6b3cf66-c4a8-449a-8676-9027e8922b96'
headers = {"User-Agent":'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.100 Safari/537.36'}

r = requests.get(URL, headers=headers)
if r.status_code == 200:
    soup = BeautifulSoup(r.text)
    title = soup.find(id="productTitle")
    print(title.next)