Python通用Feedparser因Unicode错误崩溃

Question

我正在使用OSX 10.6和Python 2.7.1，搭配BeautifulSoup 3.0和feedparser 5.01。现在我想解析《纽约时报》的RSS订阅源，这个源是有效的，而且Beautiful Soup自己可以顺利解析。

为了产生错误，我写的最简单的代码是：

import feedparser
from BeautifulSoup import BeautifulSoup


feed = feedparser.parse("http://www.nytimes.com/services/xml/rss/nyt/GlobalHome.xml")

如果我使用网址或者用urllib2.urlopen来获取内容，都会失败。
我也尝试过字符集检测器。

错误信息是：

 /Users/user/Source/python/feed/BeautifulSoup.py:1553: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  elif data[:3] == '\xef\xbb\xbf':
/Users/user/Source/python/feed/BeautifulSoup.py:1556: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  elif data[:4] == '\x00\x00\xfe\xff':
/Users/user/Source/python/feed/BeautifulSoup.py:1559: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  elif data[:4] == '\xff\xfe\x00\x00':
Traceback (most recent call last):
  File "parse.py", line 5, in <module>
    feed = feedparser.parse("http://www.nytimes.com/services/xml/rss/nyt/GlobalHome.xml")
  File "/Users/user/Source/python/feed/feedparser.py", line 3822, in parse
    feedparser.feed(data.decode('utf-8', 'replace'))
  File "/Users/user/Source/python/feed/feedparser.py", line 1851, in feed
    sgmllib.SGMLParser.feed(self, data)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sgmllib.py", line 104, in feed
    self.goahead(0)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sgmllib.py", line 143, in goahead
    k = self.parse_endtag(i)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sgmllib.py", line 320, in parse_endtag
    self.finish_endtag(tag)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sgmllib.py", line 360, in finish_endtag
    self.unknown_endtag(tag)
  File "/Users/user/Source/python/feed/feedparser.py", line 657, in unknown_endtag
    method()
  File "/Users/user/Source/python/feed/feedparser.py", line 1545, in _end_description
    value = self.popContent('description')
  File "/Users/user/Source/python/feed/feedparser.py", line 961, in popContent
    value = self.pop(tag)
  File "/Users/user/Source/python/feed/feedparser.py", line 868, in pop
    mfresults = _parseMicroformats(output, self.baseuri, self.encoding)
  File "/Users/user/Source/python/feed/feedparser.py", line 2420, in _parseMicroformats
    p = _MicroformatsParser(htmlSource, baseURI, encoding)
  File "/Users/user/Source/python/feed/feedparser.py", line 2024, in __init__
    self.document = BeautifulSoup.BeautifulSoup(data)
  File "/Users/user/Source/python/feed/BeautifulSoup.py", line 1228, in __init__
    BeautifulStoneSoup.__init__(self, *args, **kwargs)
  File "/Users/user/Source/python/feed/BeautifulSoup.py", line 892, in __init__
    self._feed()
  File "/Users/user/Source/python/feed/BeautifulSoup.py", line 917, in _feed
    SGMLParser.feed(self, markup)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sgmllib.py", line 103, in feed
    self.rawdata = self.rawdata + data
TypeError: cannot concatenate 'str' and 'NoneType' objects

我非常希望能得到一些建议。

osx unicode urllib2 beautifulsoup rss feedparser 字符集检测

Python通用Feedparser因Unicode错误崩溃

1 个回答

撰写回答