如何在scray中指定TLS版本或禁用TLS验证

2024-04-19 19:38:15 发布

您现在位置:Python中文网/ 问答频道 /正文

对于站点:https://www.cnbanbao.cn/

我在我的MAC上试过这个命令

openssl s_client  -connect www.cnbanbao.cn:443 -msg

结果是:

^{pr2}$

我认为问题可能是该网站使用TLS1.2用于ServerHelloTLS1.0用于TLS握手,当我试图在scrapy下载该网站时,会出现问题

scrapy shell 'https://www.cnbanbao.cn/'
2019-01-24 11:49:57 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET https://www.cnbanbao.cn/> (failed 1 times): [<twisted.python.failure.Failure twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion: Connection lost.>]
2019-01-24 11:49:58 [scrapy.downloadermiddlewares.retry] DEBUG: Retrying <GET https://www.cnbanbao.cn/> (failed 2 times): [<twisted.python.failure.Failure twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion: Connection lost.>]
2019-01-24 11:49:59 [scrapy.downloadermiddlewares.retry] DEBUG: Gave up retrying <GET https://www.cnbanbao.cn/> (failed 3 times): [<twisted.python.failure.Failure twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion: Connection lost.>]

我试图指定TLS1.0版本:SSL issue when scraping website,它似乎不起作用

我也尝试了Disable SSL certificate verification in Scrapy,但是我不知道如何定义一个httpsdownloaderignorecError来禁用ssl验证

有什么办法让下面的命令生效吗?在

scrapy shell 'https://www.cnbanbao.cn/'

Tags: inhttpsdebuggetwwwtwistedconnectioncn