在Python中如何编码/解码Unicode字符如ö

Question

在CentOS 6.4上使用Python 2.6.6

import json
import urllib2    

url = 'http://www.google.com.hk/complete/search?output=toolbar&hl=en&q=how%20to%20pronounce%20e'
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor())
opener.addheaders = [('Accept-Charset', 'utf-8')]
response = opener.open(url)
page = response.read()
print page

结果：

...<suggestion data="how to pronounce eyjafjallaj

在这里，Python程序崩溃了，但没有任何错误信息。

我觉得崩溃的原因是下一个字符是ö：

<toplevel>
<CompleteSuggestion>
<suggestion data="how to pronounce edinburgh"/>
</CompleteSuggestion>
<CompleteSuggestion>
<suggestion data="how to pronounce elle"/>
</CompleteSuggestion>
<CompleteSuggestion>
<suggestion data="how to pronounce edith"/>
</CompleteSuggestion>
<CompleteSuggestion>
<suggestion data="how to pronounce et al"/>
</CompleteSuggestion>
<CompleteSuggestion>
<suggestion data="how to pronounce eunice"/>
</CompleteSuggestion>
<CompleteSuggestion>
<suggestion data="how to pronounce english names"/>
</CompleteSuggestion>
<CompleteSuggestion>
<suggestion data="how to pronounce edamame"/>
</CompleteSuggestion>
<CompleteSuggestion>
<suggestion data="how to pronounce erudite"/>
</CompleteSuggestion>
<CompleteSuggestion>
<suggestion data="how to pronounce eyjafjallajökull"/>
</CompleteSuggestion>
<CompleteSuggestion>
<suggestion data="how to pronounce either"/>
</CompleteSuggestion>
</toplevel>

_{http://www.google.com.hk/complete/search?output=toolbar&hl=en&q=how%20to%20pronounce%20e}

这看起来是个unicode的问题，我尝试了很多种方法使用encode('utf-8')和decode('utf-8')，但程序还是崩溃了。有没有什么建议？

附言：看起来我需要继续使用urllib2，而不是urllib，因为urllib会忽略cookies，这会导致其他问题。

Answer 1

response.read() 返回的是一个字节串。打印字节串的时候，Python 不应该出错，因为在打印时并没有进行字符转换，字节就是直接打印出来的。

你可以试着打印 Unicode 格式的内容：

text = page.decode(response.info().getparam('charset') or 'utf-8')
print text