雪花python连接器在6小时后抛出403禁止

2024-04-29 03:16:40 发布

您现在位置:Python中文网/ 问答频道 /正文

我有一段代码,可以从snowflake下载大量数据(>;200M),并将其写入本地数据库。我正在使用Python雪花连接器v2.1.3来提取数据。它正常运行约6小时,之后它开始抛出403雪花错误,并最终关闭。我尝试过使用推荐的参数,如“unsecure_mode=True”和“client_session_keep_alive=True”,但没有任何效果

2021-05-12 09:45:40,061 [INFO/ForkPoolWorker-2/pid(26)][/app/.venv/lib/python3.7/site-packages/snowflake/connector/ssl_wrap_socket.py:427] THIS CONNECTION IS IN INSECURE MODE. IT MEANS THE CERTIFICATE WILL BE VALIDATED BUT THE CERTIFICATE REVOCATION STATUS WILL NOT BE CHECKED.
2021-05-12 09:45:40,279 [DEBUG/ForkPoolWorker-2/pid(26)][/app/.venv/lib/python3.7/site-packages/urllib3/connectionpool.py:393] https://sfc-va-ds1-2-customer-stage.s3.amazonaws.com:443 "GET /9jz1-s-vass3666/results/019c2ffa-0502-21c7-0000-39b10525bbee_0/main/data_0_0_18?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=<key_id>&Expires=1620808614&Signature=<signature> HTTP/1.1" 403 None
2021-05-12 09:45:40,280 [DEBUG/ForkPoolWorker-2/pid(26)][/app/.venv/lib/python3.7/site-packages/snowflake/connector/network.py:822] HTTP 403: Forbidden. Retrying...
2021-05-12 09:45:40,281 [ERROR/ForkPoolWorker-2/pid(26)][/app/.venv/lib/python3.7/site-packages/snowflake/connector/network.py:678] HTTP 403: Forbidden
Traceback (most recent call last):
  File "/app/.venv/lib/python3.7/site-packages/snowflake/connector/network.py", line 652, in _request_exec_wrapper
    **kwargs)
  File "/app/.venv/lib/python3.7/site-packages/snowflake/connector/network.py", line 913, in _request_exec
    raise err
  File "/app/.venv/lib/python3.7/site-packages/snowflake/connector/network.py", line 824, in _request_exec
    raise RetryRequest(exi)
snowflake.connector.network.RetryRequest: HTTP 403: Forbidden
2021-05-12 09:45:40,282 [DEBUG/ForkPoolWorker-2/pid(26)][/app/.venv/lib/python3.7/site-packages/snowflake/connector/network.py:953] Active requests sessions: 1, idle: 3
2021-05-12 09:45:40,282 [ERROR/ForkPoolWorker-2/pid(26)][/app/.venv/lib/python3.7/site-packages/snowflake/connector/chunk_downloader.py:148] Failed to fetch the large result set chunk 18/2035
Traceback (most recent call last):
  File "/app/.venv/lib/python3.7/site-packages/snowflake/connector/network.py", line 652, in _request_exec_wrapper
    **kwargs)
  File "/app/.venv/lib/python3.7/site-packages/snowflake/connector/network.py", line 913, in _request_exec
    raise err
  File "/app/.venv/lib/python3.7/site-packages/snowflake/connector/network.py", line 824, in _request_exec
    raise RetryRequest(exi)
snowflake.connector.network.RetryRequest: HTTP 403: Forbidden

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/.venv/lib/python3.7/site-packages/snowflake/connector/chunk_downloader.py", line 125, in _download_chunk
    result_data = self._fetch_chunk(self._chunks[idx].url, headers)
  File "/app/.venv/lib/python3.7/site-packages/snowflake/connector/chunk_downloader.py", line 258, in _fetch_chunk
    binary_data_handler=handler)
  File "/app/.venv/lib/python3.7/site-packages/snowflake/connector/network.py", line 612, in fetch
    **kwargs)
  File "/app/.venv/lib/python3.7/site-packages/snowflake/connector/network.py", line 692, in _request_exec_wrapper
    Error.errorhandler_wrapper(conn, None, cause)
  File "/app/.venv/lib/python3.7/site-packages/snowflake/connector/errors.py", line 100, in errorhandler_wrapper
    connection.errorhandler(connection, cursor, errorclass, errorvalue)
  File "/app/.venv/lib/python3.7/site-packages/snowflake/connector/errors.py", line 73, in default_errorhandler
    done_format_msg=errorvalue.get(u'done_format_msg'))
snowflake.connector.errors.ForbiddenError: HTTP 403: Forbidden

在这之后你所知道的就是

2021-05-12 09:45:46,405 [DEBUG/ForkPoolWorker-2/pid(26)][/app/.venv/lib/python3.7/site-packages/urllib3/connectionpool.py:393] https://sfc-va-ds1-2-customer-stage.s3.amazonaws.com:443 "GET /9jz1-s-vass3666/results/019c2ffa-0502-21c7-0000-39b10525bbee_0/main/data_0_0_19?x-amz-server-side-encryption-customer-algorithm=AES256&response-content-encoding=gzip&AWSAccessKeyId=<key>&Expires=1620808614&Signature=<signature> HTTP/1.1" 403 None
2021-05-12 09:45:46,406 [DEBUG/ForkPoolWorker-2/pid(26)][/app/.venv/lib/python3.7/site-packages/snowflake/connector/network.py:822] HTTP 403: Forbidden. Retrying...
2021-05-12 09:45:46,407 [DEBUG/ForkPoolWorker-2/pid(26)][/app/.venv/lib/python3.7/site-packages/snowflake/connector/network.py:706] retrying: errorclass=<class 'snowflake.connector.errors.ForbiddenError'>, error=HTTP 403: Forbidden, counter=136, sleeping=16(s)

在我看来,与snowflake的连接处于活动状态,但与stage的连接已过期,是否有任何方法可以解决此问题


Tags: inpyapphttpconnectorvenvlibpackages