使用urllib3.PoolManag时看到ClosedPoolError

2024-06-17 09:57:58 发布

您现在位置:Python中文网/ 问答频道 /正文

在使用urllib3.PoolManager(python 2.7,urllib3版本:1.21)时,我经常遇到ClosedPoolError异常。在阅读了代码之后,我倾向于相信这是意料之中的,因为返回的连接对象在使用时可能已经被池的缓存逐出。你知道吗

我已将错误复制到以下脚本中:

https://pastebin.com/aJggVHuv

#!/bin/bash

set -ex

TEST_DIR=/tmp/urllib3_bug
mkdir -p $TEST_DIR
cd $TEST_DIR
virtualenv .virtualenv
. .virtualenv/bin/activate
pip install urllib3==1.21

# Starts 3 threads with pool size = 2
echo 'from urllib3 import PoolManager
import threading
from threading import Thread

pool_manager = PoolManager(num_pools=2)


def start(url):
    print "Getting", url, "on", threading.current_thread().ident
    return pool_manager.urlopen("GET", url)


thread1 = Thread(target=start, args=["http://localhost:9998"])
thread2 = Thread(target=start, args=["http://localhost:9999"])
thread3 = Thread(target=start, args=["http://localhost:10000"])

thread1.start()
import time
time.sleep(1)
thread2.start()
thread3.start()
' > start.py

# Simulates preemption by sleeping the thread
echo '303a304,308
>         import time
>         import threading
>         print "Simulating preemption on", threading.current_thread().ident
>         time.sleep(3)
>         print "Waking up on", threading.current_thread().ident' | patch $TEST_DIR/.virtualenv/local/lib/python2.7/site-packages/urllib3/poolmanager.py -
rm -f $TEST_DIR/.virtualenv/local/lib/python2.7/site-packages/urllib3/poolmanager.pyc


python start.py

我创建一个大小为2的池,然后运行3个线程,每个线程连接到不同的URL。第一个线程应该遇到ClosedPoolError。你知道吗

(补丁程序插入sleep()以可靠地在第一个线程上复制抢占)。你知道吗


Tags: testimporturlvirtualenvtimedir线程thread