如何使用python bs4获取wikipedia表中的第一列值?

2024-06-01 01:38:59 发布

您现在位置:Python中文网/ 问答频道 /正文

我正在尝试使用PythonBS4在wikipedia中对数据表进行web抓取。但是我被这个问题困住了。获取数据值时,我的代码没有获取第一列或索引零。我觉得索引有问题,但我想不出来。请帮忙。见

enter image description here

response_obj = requests.get('https://en.wikipedia.org/wiki/Metro_Manila').text
soup = BeautifulSoup(response_obj,'lxml')
Neighborhoods_MM_Table = soup.find('table', {'class':'wikitable sortable'})

rows = Neighborhoods_MM_Table.select("tbody > tr")[3:8]

cities = []
for row in rows:
    city = {}
    tds = row.select('td')
    city["City or Municipal"] = tds[0].text.strip()
    city["%_Population"] = tds[1].text.strip()
    city["Population"] = float(tds[2].text.strip().replace(",",""))
    city["area_sqkm"] = float(tds[3].text.strip().replace(",",""))
    city["area_sqm"] = float(tds[4].text.strip().replace(",",""))
    city["density_sqm"] = float(tds[5].text.strip().replace(",",""))
    city["density_sqkm"] = float(tds[6].text.strip().replace(",",""))

    cities.append(city)

print(cities)

df=pd.DataFrame(cities)

df.head() 


Tags: textobjcityresponsetablewikipediafloatreplace
1条回答
网友
1楼 · 发布于 2024-06-01 01:38:59
import requests
from bs4 import BeautifulSoup
import pandas as pd


def main(url):
    r = requests.get(url)
    soup = BeautifulSoup(r.content, 'html.parser')
    target = [item.get_text(strip=True) for item in soup.findAll(
        "td", style="text-align:right") if "%" in item.text] + [""]
    df = pd.read_html(r.content, header=0)[5]
    df = df.iloc[1: -1]
    df['Population (2015)[3]'] = target
    print(df)
    df.to_csv("data.csv", index=False)


main("https://en.wikipedia.org/wiki/Metro_Manila")

输出:view-online

enter image description here

相关问题 更多 >