Python用beauthoulsoup嵌套html标记

2024-04-23 10:04:51 发布

您现在位置：Python中文网/ 问答频道 /正文
3907 3

我试图从嵌套的html代码中获取一些所有的href URL:

...
<li class="dropdown">
<a href="#" class="dropdown-toggle wide-nav-link" data-toggle="dropdown">TEXT_1 <b class="caret"></b></a>
<ul class="dropdown-menu">
<li class="class_A"><a title="Title_1" href="http://www.customurl_1.com">Title_1</a></li>
<li class="class_B"><a title="Title_2" href="http://www.customurl_2.com">Title_2</a></li>
...
<li class="class_A"><a title="Title_X" href="http://www.customurl_X.com">Title_X</a></li>
</ul>
</li>
...
<li class="dropdown">
<a href="#" class="dropdown-toggle wide-nav-link" data-toggle="dropdown">TEXT_2 <b class="caret"></b></a>
<ul class="dropdown-menu">
<li class="class_A"><a title="Title_1" href="http://www.customurl_1.com">Title_1</a></li>
<li class="class_B"><a title="Title_2" href="http://www.customurl_2.com">Title_2</a></li>
...
<li class="class_A"><a title="Title_X" href="http://www.customurl_X.com">Title_X</a></li>
</ul>
</li>
...

在原始的html代码中，大约有15个“li”块带有类“dropdown”， 但是我只想从text=text_1的块中获取url。 有没有可能用beauthoulsoup绘制所有这些嵌套的url？在

谢谢你的帮助


Tags： 代码comhttptitlehtmlwwwliul
2条回答
网友
1楼 · 编辑于 2024-04-23 10:04:51

虽然没有Xpath那么优雅，但是您可以始终使用日常Python迭代来编写逻辑。beauthoulsoup允许将函数作为一个过滤器传递给find_all，当您有一个像这样的复杂情况时。

from bs4 import BeautifulSoup

html_doc = """<html>..."""
soup = BeautifulSoup(html_doc)

def matches_block(tag):
    return matches_dropdown(tag) and tag.find(matches_text) != None

def matches_dropdown(tag):
    return tag.name == 'li' and tag.has_attr('class') and 'dropdown' in tag['class']

def matches_text(tag):
    return tag.name == 'a' and tag.get_text().startswith('TEXT_1')

for li in soup.find_all(matches_block):
    for ul in li.find_all('ul', class_='dropdown-menu'):
        for a in ul.find_all('a'):
            if a.has_attr('href'):
                print (a['href'])
网友
2楼 · 编辑于 2024-04-23 10:04:51

Xpath和lxan示例：

from lxml import etree
from io import StringIO

parser = etree.HTMLParser()
tree   = etree.parse(StringIO(html), parser)
hrefs = tree.xpath('//li[@class="dropdown" and a[starts-with(.,"TEXT_1")]]/ul[@class="dropdown-menu"]/li/a/@href')

print hrefs

其中，html是包含html内容的unicode字符串。结果：

^{pr2}$

注意：我在XPath查询中使用starts-with函数更精确，但是如果TEXT_1不总是在文本节点的开头，则可以以相同的方式使用contains。

查询详细信息：

//              # anywhere in the domtree
li              # a li tag with the following conditions:
[                               # (opening condition bracket for li)
    @class="dropdown"           # li has a class attribute equal to "dropdown" 
  and                           # and
    a                           # a child tag "a"
    [                           # (open a condition for "a")
        starts-with(.,"TEXT_1") # that the text starts with "TEXT_1"
    ]                           # (close a condition for "a")
]                               # (close the condition for li)
/                            # li's child (/ stands for immediate descendant)
ul[@class="dropdown-menu"]   # "ul" with class equal to "dropdown-menu"
/li                          # "li" children of "ul"
/a                           # "a" children of "li"
/@href                       # href attributes children of "a"

相关问题 更多 >

    编程相关推荐