需要Python线程帮助

Question

我是雪盲从看例子，不完全符合我的情况。。。或者他们会。因此，如果有很好的例子，我还不能用大约3周的Python经验来解释它们

我有一个脚本，可以查询数据库，收集可下载电影的列表，然后将它们逐个下载到您选择的目录中。我想让它下载4或5一次，因为它需要一个年龄做。你知道吗

这里是一个简化的版本，我尝试做什么，myapp是我的数据库应用程序。不过，它似乎只是按顺序运行，尽管它说它正在启动两个线程

listOfIDs是一些容器的id，这些容器可能有电影，也可能没有电影，然后versionS返回电影文件名。你知道吗

import threading
import myapp_api

listOfIDs = (14809, 14808, 14807, 14806, 14805, 14804, 14803)
for ID in listOfIDs:
    versionS = myapp.find_one('Version', [['id', 'is', ID]], ['uploaded_movie'])

ipath = ('/Users/me/Desktop/scripts/downloads/')

exitFlag = 0

class myThread (threading.Thread):
    def __init__(self, threadID, name, counter):
        threading.Thread.__init__(self)
        self.threadID = threadID
        self.name = name
        self.counter = counter
    def run(self):
        print "Starting " + self.name
        for ID in listOfIDs:
            print "\nID= " + str(ID) + "\n"
            downLoad(ID)
        print "Exiting " + self.name

def downLoad(ID):
    versionS = myapp.find_one('Version', [['id', 'is', ID]], ['uploaded_movie'])
    path = ipath + (str(versionS).split("'")[5])
    result = myapp.download_attachment(attachment=versionS['uploaded_movie'], file_path=path)
    print "Thread Name = " + threadName


# Create new threads
thread1 = myThread(1, "Thread-1", 1)
thread2 = myThread(2, "Thread-2", 2)


# Start new Threads
thread1.start()
thread2.start()

print "Exiting Main Thread"

好的，所以我修改了代码来接受建议ShadowRanger，它仍然只是一次下载一个，我把它塞进什么地方了吗。。。代码现在是这样的。你知道吗

import threading
import myapp_api
from collections import deque

listOfIDs = (14809, 14808, 14807, 14806, 14805, 14804, 14803)
for ID in listOfIDs:
    versionS = myapp.find_one('Version', [['id', 'is', ID]], ['uploaded_movie'])

ipath = ('/Users/me/Desktop/scripts/downloads/')

def downLoad(ID):
    path = ipath + (str(versionS).split("'")[5])
    result = myapp.download_attachment(attachment=versionS['uploaded_movie'], file_path=path)

with closing(multiprocessing.Pool(4)) as pool:
    deque(pool.imap_unordered(downLoad, listOfIDs), maxlen=0)

最后，ShadowRanger给出的所有建议都是正确的，错误在于我做了一些不正确的事情（我想我很早就迭代了listOfIDs，只把最后一个传递给函数）。。。这是最后的工作版本。你知道吗

   import threading
    import myapp_api
    from collections import deque

listOfIDs = (14809, 14808, 14807, 14806, 14805, 14804, 14803)

ipath = ('/Users/me/Desktop/scripts/downloads/')

def downLoad(ID):
    versionS = myapp.find_one('Version', [['id', 'is', ID]], ['uploaded_movie'])
    path = ipath + (str(versionS).split("'")[5])
    result = myapp.download_attachment(attachment=versionS['uploaded_movie'], file_path=path)


with closing(multiprocessing.Pool(4)) as pool:
    deque(pool.imap_unordered(downLoad, listOfIDs), maxlen=0)

Answer 1

我看不出线程是如何分裂的。看起来他们下载的东西是一样的。你知道吗

如果目标是基于已知ID下载一堆文件，multiprocessing有一个.dummy模块，它可以像multiprocessing一样工作，但使用线程来实现，这为您提供了一个简单的线程池：

import multiprocessing.dummy as multiprocessing
from contextlib import closing

with closing(multiprocessing.Pool(4)) as pool: # Pick your favorite number of workers
    pool.map(downLoad, listOfIDs)