PythonWebScraping对线程的多线程限制?

2024-03-28 13:50:08 发布

您现在位置:Python中文网/ 问答频道 /正文

我有一个从网站检索搜索结果的代码。你知道吗

ExtractDetails功能是一个搜索网站的大型搜索功能。你知道吗

LangList = ['91','32','69','A9','48','62','01','75','34','73','B1',
'68','61','20','03','35','36','86','87','AS','63','04','AO','65','57','05','AE','06','AC','07','08','A4','37','72','70',
'38','09','95','10','A1','B0','AL','A8','39','AT','94','11','84','12','59','AG','AB','13','40','14','AW','67','98','AJ',
'41','15','74','16','33','17','AU','AV','A0','42','92','AI','18','AD','A2','43','AM','44','45','A3','85','58','83','71',
'21','22','A6','90','AR','99','46','78','23','24','76','AP','25','26','AN','47','60','97','49','56','27','93','AZ','02',
'28','77','AA','29','50','A5','64','51','52','AQ','AY','A7','80','30','53','54','31','96','79','89','55','66','82','81']
Alphabet = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'
SearchCriteria=[Alphabet] * len(LangList)
for Lang , alphabets in LostConn.items():
    for item in alphabets:
        t = Thread(target=ExtractDetails,args=(Lang,item + '%',))
        t.start()
        th.append(t)
for thr in th:
    thr.join()

脚本的问题是,如果运行此代码,我的笔记本电脑将挂起,因为它将生成3406个线程。我的问题是如何确定特定笔记本电脑上的最佳线程数?你知道吗


Tags: 代码in功能langfor网站item线程
1条回答
网友
1楼 · 发布于 2024-03-28 13:50:08

可以并发运行的线程数取决于您的计算机和cpu的内核数。大多数机器每个核心可以运行2个线程。在Python的multiprocessing模块中,有一种简单的方法可以确定什么适用于您的机器:

import multiprocessing
multiprocessing.cpu_count()

此函数将显示可并发运行的最大线程数。希望这有帮助!你知道吗

相关问题 更多 >