在Android上的Kivy/Python缺少Unicode编解码器？

Question

我有一个在kivy下运行的程序，在Windows上运行得很好，但在Android上打开文件时却出错（两个平台都是1.8.0）。奇怪的是，错误信息显示它在尝试解码ASCII，而我明明是要求使用utf-8；而且我检查过文件，里面根本没有0xFF这个字符 - 我用od工具确认过了。

代码如下：

try:
    Logger.info('Mark: opening file ' + repr(filename))
    with codecs.open(filename, 'r', encoding='utf_8') as f:
        lines = [line.rstrip(u'\r\n').lstrip(codecs.BOM) for line in f]
except UnicodeDecodeError as e:
    Logger.info('Mark: utf8 load failed: ' + str(e))
    with codecs.open(filename, 'r', encoding='cp1252') as f:
        lines = [line.rstrip(u'\r\n') for line in f]

日志文件：

[INFO              ] Mark: opening file '/mnt/sdcard/My Files/Documents/DVDs.txt'
[INFO              ] Mark: utf8 load failed: 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128)
[INFO              ] Base: Leaving application in progress...
[WARNING           ] stderr: Traceback (most recent call last):
[WARNING           ] stderr:   File "main.py", line 295, in <module>
[WARNING           ] stderr:     app.run()
[WARNING           ] stderr:   File "/home/tito/code/python-for-android/build/python-install/lib/python2.7/site-packages/kivy/app.py", line 792, in run
[WARNING           ] stderr:   File "/home/tito/code/python-for-android/build/python-install/lib/python2.7/site-packages/kivy/base.py", line 481, in runTouchApp
[WARNING           ] stderr:   File "/home/tito/code/python-for-android/build/python-install/lib/python2.7/site-packages/kivy/core/window/window_pygame.py", line 381, in mainloop
[WARNING           ] stderr:   File "/home/tito/code/python-for-android/build/python-install/lib/python2.7/site-packages/kivy/core/window/window_pygame.py", line 287, in _mainloop
[WARNING           ] stderr:   File "/home/tito/code/python-for-android/build/python-install/lib/python2.7/site-packages/kivy/base.py", line 321, in idle
[WARNING           ] stderr:   File "/home/tito/code/python-for-android/build/python-install/lib/python2.7/site-packages/kivy/clock.py", line 422, in tick
[WARNING           ] stderr:   File "/home/tito/code/python-for-android/build/python-install/lib/python2.7/site-packages/kivy/clock.py", line 537, in _process_events
[WARNING           ] stderr:   File "/home/tito/code/python-for-android/build/python-install/lib/python2.7/site-packages/kivy/clock.py", line 309, in tick
[WARNING           ] stderr:   File "main.py", line 266, in select_file_part2
[WARNING           ] stderr:     with codecs.open(filename, 'r', encoding='cp1252') as f:
[WARNING           ] stderr:   File "/home/tito/code/python-for-android/build/python-install/lib/python2.7/codecs.py", line 884, in open
[WARNING           ] stderr: LookupError: unknown encoding: cp1252

编辑：我加了一些调试代码，查看了所有编码方式，发现除了ascii、latin_1和各种utf编码外，其他的都缺失了。

Answer 1

去掉那个try/except，直接看看真正的错误是什么。codecs.BOM是一个字节串，内容是\xff\xfe，它正被用默认的ascii编码转换成Unicode。

>>> import codecs
>>> codecs.BOM
'\xff\xfe'
>>> u'test'.lstrip(codecs.BOM)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128)