从静态网站中删除表

Question

我需要从iana.org中使用顶级域刮表

我的代码：

import requests
from bs4 import BeautifulSoup

URL = 'https://www.iana.org/domains/root/db'
page = requests.get(URL)

soup = BeautifulSoup(page.content, 'html.parser')

results = soup.find(id='tld-table')

我如何才能将其以网站上的结构（域、类型、TLD管理器）发送到pandas DataFrame

Answer 1

您可以使用pandaspd.read_html

import pandas as pd

URL = "https://www.iana.org/domains/root/db"

df = pd.read_html(URL)[0]

print(df.head())
    Domain     Type                            TLD Manager
0     .aaa  generic  American Automobile Association, Inc.
1    .aarp  generic                                   AARP
2  .abarth  generic         Fiat Chrysler Automobiles N.V.
3     .abb  generic                                ABB Ltd
4  .abbott  generic              Abbott Laboratories, Inc.

Answer 2

Pandas已经提供了一些可阅读的表格from html，无需使用BeautifulSoup：

import pandas as pd

url = "https://www.iana.org/domains/root/db"
# This returns a list of DataFrames with all tables in the page. 
df = pd.read_html(url)[0]