Webcrawler循环

Question

我在我的网络爬虫中写了以下循环。在

几秒钟后就用完了。我不明白为什么。在

def crawlweb(seed):
    crawled = []
    tocrawl = [seed]
    page = tocrawl[0]
    while tocrawl:
        if page not in crawled:
            tocrawl = tocrawl[1:] + (get_links(get_page(page)))
            crawled.append(page)
    return crawled, tocrawl

Answer 1

def crawl_web(seed)
tocrawl = [seed]
crawled = []
while tocrawl:
    page = tocrawl.pop()
    if page not in crawled:
        union(tocrawl, get_all_links(get_page(page)))
        crawled.append(page)
return crawled