Python线程会不断产生新的线程吗？

Question

我有以下代码：

while True:
    try:
        for i in range(2):
            t1 = Thread(target = grab_data_from_queue)
            t1.daemon = True
            t1.start() # start the thread
    except (KeyboardInterrupt, SystemExit):
      print '\n[!] Received keyboard interrupt, quitting threads.\n'
      exit()

正在运行的函数正在等待将项添加到队列中—我希望这是一个长时间运行的任务，因此该函数将处理任务。你知道吗

然而，这将继续产生新的线程不断，我不想让它运行一夜之间，有成千上万的处理线程。你知道吗

更新： 一个可能的解决方案是根据评论执行以下操作：

while True:
    if q.empty():
        time.sleep(5)
    else:
        try:
            for i in range(2):
                t1 = Thread(target = grab_data_from_queue)
                t1.daemon = True
                t1.start() # start the thread
        except (KeyboardInterrupt, SystemExit):
          print '\n[!] Received keyboard interrupt, quitting threads.\n'
          exit()

Answer 1

如果grab_data_from_queue func是cpu密集型作业，您可以尝试使用concurrent.futures.ProcessPoolExecutor()，那么您的代码可能如下所示：

pool = concurrent.futures.ProcessPoolExecutor(max_workers=3)
while True:
    if q.empty():
        time.sleep(5)
    else:
        futures = set()
        with pool as executor:
            future = executor.submit(grab_data_from_queue, data)
            futures.add(future)
        try:
            for future in concurrent.futures.as_completed(futures):
                exc = future.exception()
                if exc:
                    raise exc

        except (KeyboardInterrupt, SystemExit):
          print '\n[!] Received keyboard interrupt, quitting threads.\n'
          exit()