Python的threading.enumerate函数返回意外结果

Question

我正在使用 多线程 来同时运行多个 Selenium 驱动程序。每个驱动程序都有一个独立的线程。当我有 70 个驱动实例时（ThreadPoolExecutor(max_workers=70)），len(threading.enumerate()) 应该返回 70 + 1（包括 MainThread），对吧？但是在程序运行一段时间后，len(threading.enumerate()) 返回的结果却不对（比如 80 或 90 个线程）。我使用的是 seleniumbase / undetected-chromedriver。也许这些库在创建额外的线程？

import concurrent.futures
import threading
import time
from seleniumbase import Driver


# Pseudocode
def run_driver(driver, url):
    try:
        driver.get(url=url)
        print(url, len(threading.enumerate()))  # Wrong result
        time.sleep(10)
    except Exception as exc:
        driver.close()
        driver.quit()


urls = ['https://google.com' for i in range(70)]
with concurrent.futures.ThreadPoolExecutor(max_workers=len(urls)) as executor:
    for url in urls:
        driver = Driver(uc=True, headed=True)
        executor.submit(run_driver, driver, url)

Answer 1

你需要把 driver.quit() 放在 finally 这个块里，不然的话，只有在你的代码出现错误的时候，驱动才会被关闭。我对代码做了一些更新，还加了 sys.argv.append("-n")，这样在修补驱动的时候，SeleniumBase UC 模式下的线程锁定就能正确处理了。

import concurrent.futures
import threading
import time
from seleniumbase import Driver
import sys
sys.argv.append("-n")


def run_driver(url):
    driver = Driver(uc=True)
    try:
        driver.get(url=url)
        print(url, len(threading.enumerate()))
        time.sleep(3)
    finally:
        driver.quit()


urls = ['https://seleniumbase.io/demo_page' for i in range(8)]
with concurrent.futures.ThreadPoolExecutor(max_workers=len(urls)) as executor:
    for url in urls:
        executor.submit(run_driver, url)