从线程派生线程

Question

我正在使用python3.4.2

1. 让线程列表中的线程（t1）处理一些数据
2. 让t1生成另一个线程（t2）来执行额外的处理

问题是proc_q1似乎被正确地调用添加id到t2\u q，但是在exit_flag被设置为True之前，t2线程没有从队列中提取任务。为什么proc_q2从来没有人打过电话？你知道吗

下面是一些模拟代码

import urllib.request
import threading

from queue import Queue
from xml.etree import ElementTree as ET
from urllib.parse import urlparse

class Scanner(object):
    @staticmethod
    def proc_q1(thread_id, q1, q2):
        while not exit_flag:
            t1_lck.acquire()
            if not q1.empty():
                new_id = q1.get()
                t1_lck.release()
                # add id to t2_queue for threads2 to pick up
                t2_lck.acquire()
                t2_q.put(new_id)
                t2_lck.release()
                print('proc_q1 - new_id: {}'.format(new_id))
            else:
                t1_lck.release()

    @staticmethod
    def proc_q2(thread_id, q):
        t2_lck.acquire()
        if not q.empty():
            new_id = q.get()
            t2_lck.release()
            print('proc_q2 - new_id: {}'.format(new_id))
        else:
            t2_lck.release()


class MyT1(threading.Thread):
    def __init__(self, thread_id, q1, q2):
        threading.Thread.__init__(self)
        self.thread_id = thread_id
        self.q1 = q1
        self.q2 = q2

    def run(self):
        print('Starting t1-thread {}'.format(self.thread_id))
        Scanner.proc_q1(thread_id=self.thread_id, q1=self.q1, q2=self.q2)
        print('Exiting t1-thread {}'.format(self.thread_id))


class MyT2(threading.Thread):
    def __init__(self, thread_id, q):
        threading.Thread.__init__(self)
        self.thread_id = thread_id
        self.q = q

    def run(self):
        print('Starting t2-thread {}'.format(self.thread_id))
        Scanner.proc_q2(thread_id=self.thread_id, q=self.q)
        print('Exiting t2-thread {}'.format(self.thread_id))

# Create new threads
thread_pool_size = 5
t1_list = list()
t2_list = list()
t1_q = Queue(2 * thread_pool_size)
t2_q = Queue(2 * thread_pool_size)
t1_lck = threading.Lock()
t2_lck = threading.Lock()

exit_flag = False

for i in range(thread_pool_size):
    t1 = MyT1(thread_id=i, q1=t1_q, q2 = t2_q)
    t1.start()
    t1_list.append(t1)

    t2 = MyT2(thread_id=i, q=t2_q)
    t2.start()
    t2_list.append(t2)

# put data on the queue
t1_lck.acquire()
for i in range(10):
    t1_q.put(i)
t1_lck.release()

# Wait for empty queue
while not (t1_q.empty() and t2_q.empty()):
    pass

exit_flag = True

# wait for threads2 to finish
for t in t2_list:
    t.join()

# wait for threads1 to finish
for t in t1_list:
    t.join()

print("Done")

下面是输出（脚本没有正确退出）

Starting t1-thread 0
Starting t2-thread 0
Exiting t2-thread 0
Starting t1-thread 1
Starting t2-thread 1
Exiting t2-thread 1
Starting t1-thread 2
Starting t2-thread 2
Starting t1-thread 3
Exiting t2-thread 2
Starting t2-thread 3
Starting t1-thread 4
Exiting t2-thread 3
Starting t2-thread 4
Exiting t2-thread 4
proc_q1 - new_id: 0
proc_q1 - new_id: 1
proc_q1 - new_id: 2
proc_q1 - new_id: 3
proc_q1 - new_id: 5
proc_q1 - new_id: 4
proc_q1 - new_id: 6
proc_q1 - new_id: 7
proc_q1 - new_id: 8
proc_q1 - new_id: 9

Answer 1

我认为日志输出给出了答案。你知道吗

MyT2线程在队列q2被填满之前退出。你知道吗

退出条件为：

@staticmethod
def proc_q2(thread_id, q):
    ...
    if not q.empty():
    ...

最初，队列q2是空的，因此all MyT2在队列有数据之前退出。你知道吗

因此，程序将永远在while循环中循环：

...
while not (t1_q.empty() and t2_q.empty()):
...

顺便说一句： 锁定：据我所知，队列的get（）和put（）方法是同步的。排队和解排队不需要获取锁。你知道吗

从文档https://docs.python.org/3.4/library/queue.html：

17.7. queue — A synchronized queue class