如何使用BeautifulSoup4将<br>标记前的所有文本放入数据框

Question

我试图通过抓取一个网站来提取悉尼的邮政编码和郊区作为数据框

我已经在beautifulsup中找到了相应的标签，但无法完成该过程

import requests
from bs4 import BeautifulSoup

url = 'https://data.mongabay.com/igapo/australia/postcodes/sydney-numeric.html'
res = requests.get(url)
soup = BeautifulSoup(res.text,'html5lib')

table = soup.find_all('table')[10]

rows = table.find_all('td')[2]

for br in rows.find_all("br"):
    br.replace_with("\n")

parsedText = rows.get_text()

我期待一个数据帧，例如：

postcode suburbs
2000 Australia Square Post Office
2000 Circular Quay
2000 Clarence Street Post Office
...

谢谢你的帮助

Answer 1

有点冗长的方法。bs4.7.1标准

import requests
import pandas as pd
​
url = 'https://data.mongabay.com/igapo/australia/postcodes/sydney-numeric.html'
r = requests.get(url, headers = {'User-Agent' : 'Mozilla/5.0'})
soup = bs(r.content, 'lxml')
codes = []
names = []
​
for line in soup.select_one('h1:contains("Postal codes") ~ font [size="2"]').text.split('\n'):
    if line[:4]:
        codes.append(line[:4]), names.append(line[5:])
df = pd.DataFrame(list(zip(codes, names)), columns = ['code', 'name'])

​