我的Python程序很慢！如何加速？我做错了吗？

Question

更新：我运行了Python的性能分析工具，发现最耗时的两个部分（这是在我决定注释掉网页浏览器和Firefox相关代码之后，因为我知道它们会是最慢的部分……），我程序中最慢的部分是 re.findall 和 re.compile，还有（len）和（添加到列表）。

我不知道是否应该一次性把我的所有代码都发上来，因为我在这个程序上花了很多心思（即使它不是很好），所以现在我只是想问……我该如何让我的Python程序更快呢？

目前我有三个怀疑的原因导致它这么慢：

可能是我的电脑本身就慢
可能是我的网络太慢（有时候我的程序需要下载网页的HTML，然后在HTML中搜索特定的文本）
我的代码本身慢（可能是循环太多了？还是其他什么？我对这些还不太了解！）

如果有人能给我一些建议，我会非常感激！

谢谢！

更新：

我觉得我的代码使用了很多循环……另外，程序要正常工作，你必须先登录这个网站：http://www.locationary.com/

from urllib import urlopen
from gzip import GzipFile
from cStringIO import StringIO
import re
import urllib
import urllib2
import webbrowser
import time
from difflib import SequenceMatcher
import os

def download(url):
    s = urlopen(url).read()
    if s[:2] == '\x1f\x8b': # assume it's gzipped data
        with GzipFile(mode='rb', fileobj=StringIO(s)) as ifh:
            s = ifh.read()
    return s

for t in range(3,39):
    print t
    s = download('http://www.locationary.com/place/en/US/Utah/Provo-page' + str(t) + '/?ACTION_TOKEN=NumericAction')
    findLoc = re.compile('http://www\.locationary\.com/place/en/US/.{1,50}/.{1,50}/.{1,100}\.jsp')
    findLocL = re.findall(findLoc,s)
    W = []
    X = []
    XA = []
    Y = []
    YA = []
    Z = []
    ZA = []

    for i in range(0,25):
        b = download(findLocL[i])        
        findYP = re.compile('http://www\.yellowpages\.com/')
        findYPL = re.findall(findYP,b)
        findTitle = re.compile('<title>(.*) \(\d{1,10}.{1,100}\)</title>')
        getTitle = re.findall(findTitle,b)        
        findAddress = re.compile('<title>.{1,100}\((.*), .{4,14}, United States\)</title>')
        getAddress = re.findall(findAddress,b)        
        if not findYPL:
            if not getTitle:
                print ""
            else:
                W.append(findLocL[i])
            b = download(findLocL[i])
            if not getTitle:
                print ""
            else:
                X.append(getAddress)
            b = download(findLocL[i])
            if not getTitle:
                print ""
            else:
                Y.append(getTitle)
    sizeWXY = len(W)

    def XReplace(text, dic):
        for i, j in dic.iteritems():
            text = text.replace(i, j)  
        XA.append(text)

    def YReplace(text2, dic2):
        for k, l in dic2.iteritems():
            text2 = text2.replace(k, l)  
        YA.append(text2)

    for d in range(0,sizeWXY):
        old = str(X[d])
        reps = {' ':'-', ',':'', '\'':'', '[':'', ']':''}
        XReplace(old, reps)
        old2 = str(Y[d])
        YReplace(old2, reps)

    count = 0    
    for e in range(0,sizeWXY):
        newYPL = "http://www.yellowpages.com/" + XA[e] + "/" + YA[e] + "?order=distance"
        v = download(newYPL)
        abc = str('<h3 class="business-name fn org">\n<a href="')
        dfe = str('" class="no-tracks url "')
        findFinal = re.compile(abc + '(.*)' + dfe)
        getFinal = re.findall(findFinal, v)
        if not getFinal:
            W.remove(W[(e-count)])
            X.remove(X[(e-count)])
            count = (count+1)
        else:
            for f in range(0,1):
                Z.append(getFinal[f])
    XA = []
    for c in range(0,(len(X))):
        aGd = re.compile('(.*), .{1,50}')
        bGd = re.findall(aGd, str(X[c]))
        XA.append(bGd)
    LenZ = len(Z)
    V = []
    for i in range(0,(len(W))):
        if i == 0:
            countTwo = 0
        gda = download(Z[i-(countTwo)])
        ab = str('"street-address">\n')
        cd = str('\n</span>')
        ZAddress = re.compile(ab + '(.*)' + cd)
        ZAddress2 = re.findall(ZAddress, gda)
        for b in range(0,(len(ZAddress2))):
            if not ZAddress2[b]:
                print ""
            else:
                V.append(str(ZAddress2[b]))
                a = str(W[i-(countTwo)])
                n = str(Z[i-(countTwo)])
                c = str(XA[i])
                d = str(V[i])
                #webbrowser.open(a)
                #webbrowser.open(n)
                m = SequenceMatcher(None, c, d)
                if m.ratio() < 0.50:
                    Z.remove(Z[i-(countTwo)])
                    W.remove(W[i-(countTwo)])
                    countTwo = (countTwo+1)

    def ZReplace(text3, dic3):
        for p, q in dic3.iteritems():
            text3 = text3.replace(p, q)  
        ZA.append(text3)

    for y in range(0,len(Z)):
        old3 = str(Z[y])
        reps2 = {':':'%3A', '/':'%2F', '?':'%3F', '=':'%3D'}
        ZReplace(old3, reps2)
    for z in range(0,len(ZA)):
        findPID = re.compile('\d{5,20}')
        getPID = re.findall(findPID,str(W[z]))
        newPID = re.sub("\D", "", str(getPID))
        finalURL = "http://www.locationary.com/access/proxy.jsp?ACTION_TOKEN=proxy_jsp$JspView$SaveAction&inPlaceID=" + str(newPID) + "&xxx_c_1_f_987=" + str(ZA[z])
        webbrowser.open(finalURL)
        time.sleep(5)

    os.system("taskkill /F /IM firefox.exe")

性能优化数据处理代码调试循环效率代码分析网络延迟性能分析工具程序加速

我的Python程序很慢！如何加速？我做错了吗？

2 个回答

撰写回答