如何使用python从列表中获取url

Question

我想抓取列表中的url。基本上我是在刮一个网站，我是从中刮出一个链接，我正在寻找特定的链接 一个刮那些链接，我搜索另一个特定的链接一个刮它。 我的代码：

from bs4 import BeautifulSoup
import urllib.request
import re
r = urllib.request.urlopen('http://i.cantonfair.org.cn/en/ExpExhibitorList.aspx?k=glassware')
soup = BeautifulSoup(r, "html.parser")
links = soup.find_all("a", href=re.compile(r"expexhibitorlist\.aspx\?categoryno=[0-9]+"))
linksfromcategories = ([link["href"] for link in links])

string = "http://i.cantonfair.org.cn/en/"
linksfromcategories = [string + x for x in linksfromcategories]
subcatlinks = list()
for link in linksfromcategories:
  response = urllib.request.urlopen(link)
  soup2 = BeautifulSoup(response, "html.parser")
  links2 = soup2.find_all("a", href=re.compile(r"ExpExhibitorList\.aspx\?categoryno=[0-9]+"))
  linksfromsubcategories = ([link["href"] for link in links2])
  subcatlinks.append(linksfromsubcategories)
responses = urllib.request.urlopen(subcatlinks)
soup3 = BeautifulSoup(responses, "html.parser")
print (soup3)

我得到了错误

^{pr2}$

Answer 1

一次只能传入一个链接到urllib.request.urlopen，而不是它们的整个列表。在

所以你需要另一个类似这样的循环：

for link in subcatlinks:
    response = urllib.request.urlopen(link)
    soup3 = BeautifulSoup(response, "html.parser")
    print(soup3)