pythonselenium自动加载更多页面

Question

我正在尝试一个废弃的网站，“www.jabong.com网站“在这里，当我们到达页面底部时，每个产品线都会加载更多的产品。我想取消所有的链接。我正在尝试的代码如下所示：

from time import sleep
from selenium import webdriver
from selenium.webdriver.chrome.options import Options


def fetch_links(url, product_line_name):

    chrome_options = Options()
    chrome_options.add_argument("--disable-notifications")
    chrome_path = r"D:\chromedriver.exe"
    driver = webdriver.Chrome(chrome_path, chrome_options=chrome_options)
    driver.get(url)
    button="load-more-products"
    while True:
        element=driver.find_element_by_class_name(button).click()
        driver.execute_script("arguments[0].scrollIntoView();", element)



link_list=["https://www.jabong.com/women/clothing/trousers-jeans/trousers/?source=topnav_women"]
product_line=["trousers"]

fetch_links(link_list[0],product_line[0])

这里的问题是找不到元素。我也尝试过使用class\u name、css\u selector，但它们似乎都不起作用。我想在浏览器上连续自动加载产品。在

Answer 1

试试这个。它将获取您所需的数据。在

import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.keys import Keys

link_list = ["https://www.jabong.com/women/clothing/trousers-jeans/trousers/?source=topnav_women",]

def fetch_links(link):
    chrome_options = Options()
    chrome_options.add_argument(" disable-notifications")
    driver = webdriver.Chrome(chrome_options=chrome_options)
    driver.get(link)
    actions = ActionChains(driver)
    for _ in range(10):   #Adjust this range according to your need, I meant how far you wanna go down.
        actions.send_keys(Keys.SPACE).perform()
        time.sleep(1)

    for item in driver.find_elements_by_css_selector(".h4"):
        print(item.text)

fetch_links(link_list[0])

部分结果：

^{pr2}$