对于span标记，从bs4获取\u text（）是否不同？无法删除跨度标记

Question

在制作网页刮板时，我能够找到一个可用的刮板数据。 在两个数据字段上，我可以使用beautifulsoup get_text（）从数据中重新生成html

但是当我使用get\u text（）时，第三个字段将不起作用。我可以让它给我整个span标签，而不是里面的文本。你知道吗

我已经尝试了不同的迭代来获得相同的数据，它将给我整个跨度标签，即东西

正在尝试将busnumber设置为此span标记内的电话号码

<span class="business--telephoneNumber" itemprop="telephone">01430 422826 </span>

我试过了

from bs4 import BeautifulSoup
import requests
import csv

data_list=[]
url = 'https://www.yell.com/ucs/UcsSearchAction.do?keywords=farmer&location=leeds'
headers = {
        'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36',
    }
site = requests.get(url, headers=headers, timeout=5);
if site.status_code is 200:
    content = BeautifulSoup(site.content, 'html.parser')
    #print(content)
    questions = content.find_all(class_='businessCapsule') 
for question in questions:
        busname = question.find(class_='businessCapsule--name').get_text()
        bustype =   question.find(class_='businessCapsule--classification').get_text()
        busnum = question.find('span', {'itemprop': 'telephone'})
        print(busnum)
        busnumber = busnum.get_text()
        new_data = {"busname": busname, "bustype": bustype, "busnumber": busnumber}
        data_list.append(new_data)
    with open ('selector.csv','w') as file:
        writer = csv.DictWriter(file, fieldnames = ["busname", "bustype", "busnumber"], delimiter = ';')
        writer.writeheader()
        for row in data_list:
            writer.writerow(row)

以及

from bs4 import BeautifulSoup
import requests
import csv

data_list=[]
url = 'https://www.yell.com/ucs/UcsSearchAction.do?keywords=farmer&location=leeds'
headers = {
        'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36',
    }
site = requests.get(url, headers=headers, timeout=5);
if site.status_code is 200:
    content = BeautifulSoup(site.content, 'html.parser')
    #print(content)
    questions = content.find_all(class_='businessCapsule')
    for question in questions:
        busname = question.find(class_='businessCapsule--name').get_text()
        bustype =   question.find(class_='businessCapsule--classification').get_text()
        busnumber = question.find('span', {'itemprop': 'telephone'}).get_text()
        new_data = {"busname": busname, "bustype": bustype, "busnumber": busnumber}
        data_list.append(new_data)
    with open ('selector.csv','w') as file:
        writer = csv.DictWriter(file, fieldnames = ["busname", "bustype", "busnumber"], delimiter = ';')
        writer.writeheader()
        for row in data_list:
            writer.writerow(row)

在这两种情况下，get\u text（）都会出现此错误

Traceback (most recent call last):
  File "webscraper2.py", line 22, in <module>
    busnumber = busnum.get_text()
AttributeError: 'NoneType' object has no attribute 'get_text'

如果删除get\文本，它将给出整个标签

<span class="business--telephoneNumber" itemprop="telephone">01430 422826 </span>

我只需要插入的电话号码。你知道吗

更新-最新代码

from bs4 import BeautifulSoup as bs
import requests
import csv

data_list=[]
url = 'https://www.yell.com/ucs/UcsSearchAction.do?keywords=farmer&location=leeds'
headers = {
        'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36',
    }
site = requests.get(url, headers=headers, timeout=5)
soup = bs(site.content, 'html.parser')
questions = soup.select('.businessCapsule--mainContent')
for question in questions:
    busname = question.find(class_='businessCapsule--name').get_text()
    bustype =   question.find(class_='businessCapsule--classification').get_text()
    busnumber = question.select_one('span.business--telephoneNumber').text
    print(busnumber)
    new_data = {"busname": busname, "bustype": bustype, "busnumber": busnumber}
    data_list.append(new_data)
with open ('selector.csv','w') as file:
    writer = csv.DictWriter(file, fieldnames = ["busname", "bustype", "busnumber"], delimiter = ';')
    writer.writeheader()
    for row in data_list:
        writer.writerow(row)

Answer 1

您需要获取不同的父级，以便选择适当的子级并更改子级的选择器，如下所示：

import requests
from bs4 import BeautifulSoup as bs

url = 'https://www.yell.com/ucs/UcsSearchAction.do?keywords=farmer&location=leeds'
headers = {
        'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36',
    }
site = requests.get(url, headers=headers, timeout=5)
soup = bs(site.content, 'lxml')
questions = soup.select('.businessCapsule mainContent:has(span.business telephoneNumber)')
for question in questions:
    print(question.select_one('span.business telephoneNumber').text)

如果您选中这个不同的父选择器，您将看到它选择了整个框中的信息，这样您就可以选择您的各种子

如果这是太报复你可以测试，如果电话是存在的

import requests
from bs4 import BeautifulSoup as bs

url = 'https://www.yell.com/ucs/UcsSearchAction.do?keywords=farmer&location=leeds'
headers = {
        'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36',
    }
site = requests.get(url, headers=headers, timeout=5)
soup = bs(site.content, 'lxml')
questions = soup.select('.businessCapsule mainContent')
for question in questions:
    tel = question.select_one('span.business telephoneNumber')
    if tel is None:
        tel = 'Not present'
    else:
        tel = tel.text
    print(tel)