特定网页上的Python WebDriverException问题

Question

我正试图使用selenium:https://candidat.pole-emploi.fr/offres/emploi/horticulteur/s1m1在这个网页上获取就业机会

但它返回了一个问题。但是，当我尝试使用另一个页面（例如https://www.youtube.com/）时，它工作得很好，我可以正常获得html

所以我不明白为什么这个网页会阻碍整个过程

这是我的密码：

from bs4 import BeautifulSoup
import requests
import csv
import pandas as pd
from selenium import webdriver
from selenium.webdriver.chrome.service import Service

options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')

site = 'https://candidat.pole-emploi.fr/offres/emploi/horticulteur/s1m1'

wd = webdriver.Chrome('chromedriver', options=options)
wd.get(site)

下面是错误消息：

WebDriverException                        Traceback (most recent call last)
<ipython-input-45-d232051ff849> in <module>()
      1 wd = webdriver.Chrome('chromedriver', options=options)
----> 2 wd.get(site)

2 frames
/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py in get(self, url)
    428         Loads a web page in the current browser session.
    429         """
--> 430         self.execute(Command.GET, {'url': url})
    431 
    432     @property

/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py in execute(self, driver_command, params)
    416         response = self.command_executor.execute(driver_command, params)
    417         if response:
--> 418             self.error_handler.check_response(response)
    419             response['value'] = self._unwrap_value(
    420                 response.get('value', None))

/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/errorhandler.py in check_response(self, response)
    241                 alert_text = value['alert'].get('text')
    242             raise exception_class(message, screen, stacktrace, alert_text)  # type: ignore[call-arg]  # mypy is not smart enough here
--> 243         raise exception_class(message, screen, stacktrace)
    244 
    245     def _value_or_default(self, obj: Mapping[_KT, _VT], key: _KT, default: _VT) -> _VT:

WebDriverException: Message: unknown error: net::ERR_CONNECTION_RESET
  (Session info: headless chrome=94.0.4606.71)
Stacktrace:
#0 0x563e4590b6e3 <unknown>
#1 0x563e4561c9f3 <unknown>
#2 0x563e45619be6 <unknown>
#3 0x563e4560c0a5 <unknown>
#4 0x563e4560cfd5 <unknown>
#5 0x563e4560c366 <unknown>
#6 0x563e4560b945 <unknown>
#7 0x563e4560a86c <unknown>
#8 0x563e4560ab21 <unknown>
#9 0x563e4561e50c <unknown>
#10 0x563e456840c3 <unknown>
#11 0x563e4566f212 <unknown>
#12 0x563e456839bc <unknown>
#13 0x563e4566f643 <unknown>
#14 0x563e456468ec <unknown>
#15 0x563e45647db5 <unknown>
#16 0x563e4592fa14 <unknown>
#17 0x563e4593f15d <unknown>
#18 0x563e4593ee7b <unknown>
#19 0x563e4593f7a2 <unknown>
#20 0x563e4597852b <unknown>
#21 0x563e4593fa01 <unknown>
#22 0x563e459248f1 <unknown>
#23 0x563e45948408 <unknown>
#24 0x563e4594859a <unknown>
#25 0x563e4596232f <unknown>
#26 0x7f08e1bb86db start_thread