如何在Python中使用nltk.corpus逐行读取和标记文本文件

Question

我的问题是如何对文档进行分类，我有两个训练数据文件，分别是 good_reviews.txt 和 bad_reviews.txt。首先，我需要加载并标记我的训练数据，每一行都是一个文档，对应一个评论。所以我主要的任务是对给定的测试数据中的评论（行）进行分类。

我找到了一种加载和标记名称数据的方法，如下所示：

from nltk.corpus import names
names = ([(name, 'male') for name in names.words('male.txt')] +
         [(name, 'female') for name in names.words('female.txt')])

所以我想要的类似的功能是标记 lines 而不是 words。我希望代码看起来像这样，当然这段代码是无效的，因为 .lines 是不正确的语法：

reviews = ([(review, 'good_review') for review in reviews.lines('good_reviews.txt')] +
           [(review, 'bad_review') for review in reviews.lines('bad_reviews.txt')])

我希望得到的结果是这样的：

>>> reviews[0]
('This shampoo is very good blablabla...', 'good_review')

Answer 1

如果你是在读取自己的文本文件，那么用不到太多的NLTK，你只需要简单地使用file.readlines()就可以了：

good_reviews = """This is great!
Wow, it amazes me...
An hour of show, a lifetime of enlightment
"""
bad_reviews = """Comme si, Comme sa.
I just wasted my foo bar on this.
An hour of s**t, ****.
"""
with open('/tmp/good_reviews.txt', 'w') as fout:
    fout.write(good_reviews)
with open('/tmp/bad_reviews.txt', 'w') as fout:
    fout.write(bad_reviews)

reviews = []
with open('/tmp/good_reviews.txt', 'r') as fingood, open('/tmp/bad_reviews.txt', 'r') as finbad:
    reviews = ([(review, 'good_review') for review in fingood.readlines()] + [(review, 'bad_review') for review in finbad.readlines()])

print reviews

[输出结果]:

[('This is great!\n', 'good_review'), ('Wow, it amazes me...\n', 'good_review'), ('An hour of show, a lifetime of enlightment\n', 'good_review'), ('Comme si, Comme sa.\n', 'bad_review'), ('I just wasted my foo bar on this.\n', 'bad_review'), ('An hour of s**t, ****.\n', 'bad_review')]

如果你打算使用NLTK的电影评论语料库，可以查看这个链接：在NLTK/Python中使用电影评论语料库进行分类