假设我想实现这段代码,但主要区别在于:
我考虑过让工人排成一列,然后继续把任务推给可用的工人,但是 我不知道怎么做/实施它。 非常感谢你。 我希望我的问题是清楚的
import concurrent.futures
import urllib.request
URLS = ['http://www.foxnews.com/',
'http://www.cnn.com/',
'http://europe.wsj.com/',
'http://www.bbc.co.uk/',
'http://some-made-up-domain.com/']
def load_url(url, timeout):
with urllib.request.urlopen(url, timeout = timeout) as conn:
return conn.read()
with concurrent.futures.ThreadPoolExecutor(max_workers = 5) as executor:
future_to_url = {executor.submit(load_url, url, 60): url for url in URLS}
for future in concurrent.futures.as_completed(future_to_url):
url = future_to_url[future]
try:
data = future.result()
except Exception as exc:
print('%r generated an exception: %s' % (url, exc))
else:
print('%r page is %d bytes' % (url, len(data)))
您可以将数字或URL除以10作为最大工作者标识符,因为他们的计数会影响您要提交的作业数:
相关问题 更多 >
编程相关推荐