使用urllib3下载网页

Question

我正在为一个作业编写一个程序，它使用urllib3下载一个网页并将其存储在词典中。（我用的是spyder 3.6） 程序给了我一个“AttributeError”，我不知道我做错了什么。这是我的代码和我为作业编写的一步一步的注释。在

#Downloading a webpage
import urllib3
import sys
#these import statements allow us to use 'modules' aka 'libraries' ....
#code written by others that we can use

urlToRead = 'http://www.google.com'
#This value won't actually get used, because of the way the while loop
#below is set up. But while loops often need a dummy value like this to
#work right the first time

crawledWebLinks = {}
#Initialize an empty dictionary, in which (key, value) pairs will correspond to (short, url) eg
#("Goolge" , "http://www.google.com")

#Ok, there is a while loop coming up

#Here ends the set up

while urlToRead != ' ':
#This is a condition that dictates that the while loop will keep checking
#as long as this condition is true the loop will continue, if false it will stop
    try:
        urlToRead = input("Please enter the next URL to crawl")
    #the "try" prevents the program from crashing if there is an error
    #if there is an error the program will be sent to the except block
        if urlToRead == '':
            print ("OK, exiting loop")
            break
        #if the user leaves the input blank it will break out of the loop
        shortName = input("Please enter a short name for the URL " + urlToRead)
        webFile = urllib3.urlopen(urlToRead).read()
        #This line above uses a ready a readymade function in the urllib3 module to
        #do something super - cool:
        #IT takes a url, goes to the website for the url, downloads the
        #contents (which are in the form of HTML) and returns them to be
        #stored in a string variable (here called webFile)
        crawledWebLinks[shortName] = webFile
        #this line above place a key value pair (shortname, HTML for that url)
        #in the dictionary
    except:
        #this bit of code - the indented lines following 'except:' will be
        #excecuted if the code in the try block (the indented following lines
        #the 'try:' above) throw and error
        #this is an example of something known as exeption-handling
        print ("*************\nUnexpected Error*****", sys.exc_info()[0])
        #The snip 'sys.exc_info()[0]' return information about the last
        #error that occurred - 
        #this code is made available through the sys library that we imported above
        #Quite Magical :)
        stopOrProceed = input("Hmm..stop or proceed? Enter 1 to stop, enter anything else to continue")
        if stopOrProceed ==1 :
            print ('OK...Stopping\n')
            break
        #this break will break out of the nearest loop - in this case,
        #the while loop
    else:
        print ("Cool! Let's continue\n")
        continue
        # this continue will skip out of the current iteration of this 
        #loop and move to the next i.e. the loop will reset to the start
print (crawledWebLinks.keys())

Answer 1

您的问题是您试图调用urllib3.urlopen()，而urllib3没有成员urlopen这里是一个工作片段。我所做的只是将urllib3替换为urllib.request：

import urllib.request
import sys

urlToRead = 'http://www.google.com'

crawledWebLinks = {}

while urlToRead != ' ':
    try:
        urlToRead = input("Please enter the next URL to crawl: ")
        if urlToRead == '':
            print ("OK, exiting loop")
            break
        #if the user leaves the input blank it will break out of the loop
        shortName = input("Please enter a short name for the URL " + urlToRead + ": ")
        webFile = urllib.request.urlopen(urlToRead).read()
        crawledWebLinks[shortName] = webFile
    except:
        print ("*************\nUnexpected Error*****", sys.exc_info()[0])
        stopOrProceed = input("Hmm..stop or proceed? Enter 1 to stop, enter anything else to continue")
        if stopOrProceed ==1 :
            print ('OK...Stopping\n')
            break
    else:
        print ("Cool! Let's continue\n")
        continue
print (crawledWebLinks)

另一个注意事项是，仅仅打印出except块中的错误类型并不是很有用。我可以在30秒内调试你的代码，一旦我删除了它并查看了实际的回溯。在