polyglot 16.7.4

功能

• 标记化（165种语言）
• 语言检测（196种语言）
• 命名实体识别（40种语言）
• 词性标注（16种语言）
• 情感分析（136种语言）
• 单词嵌入（137种语言）
• 形态分析（135种语言）
• 音译（69种语言）

显影剂

• 拉米·阿尔福@rmyeid gmail com

importpolyglotfrompolyglot.textimportText,Word

语言检测

text=Text("Bonjour, Mesdames.")print("Language Detected: Code={}, Name={}\n".format(text.language.code,text.language.name))

Language Detected: Code=fr, Name=French

标记化

zen=Text("Beautiful is better than ugly. ""Explicit is better than implicit. ""Simple is better than complex.")print(zen.words)

[u'Beautiful', u'is', u'better', u'than', u'ugly', u'.', u'Explicit', u'is', u'better', u'than', u'implicit', u'.', u'Simple', u'is', u'better', u'than', u'complex', u'.']

print(zen.sentences)

[Sentence("Beautiful is better than ugly."), Sentence("Explicit is better than implicit."), Sentence("Simple is better than complex.")]

词性标注

text=Text(u"O primeiro uso de desobediência civil em massa ocorreu em setembro de 1906.")print("{:<16}{}".format("Word","POS Tag")+"\n"+"-"*30)forword,tagintext.pos_tags:print(u"{:<16}{:>2}".format(word,tag))

Word            POS Tag
------------------------------
O               DET
primeiro        ADJ
uso             NOUN
de              ADP
desobediência   NOUN
civil           ADJ
em              ADP
massa           NOUN
ocorreu         ADJ
em              ADP
setembro        NOUN
de              ADP
1906            NUM
.               PUNCT

命名实体识别

text=Text(u"In Großbritannien war Gandhi mit dem westlichen Lebensstil vertraut geworden")print(text.entities)

[I-LOC([u'Gro\xdfbritannien']), I-PER([u'Gandhi'])]

极性

print("{:<16}{}".format("Word","Polarity")+"\n"+"-"*30)forwinzen.words[:6]:print("{:<16}{:>2}".format(w,w.polarity))

Word            Polarity
------------------------------
Beautiful        0
is               0
better           1
than             0
ugly            -1
.                0

嵌入

word=Word("Obama",language="en")print("Neighbors (Synonms) of {}".format(word)+"\n"+"-"*30)forwinword.neighbors:print("{:<16}".format(w))print("\n\nThe first 10 dimensions out the {} dimensions\n".format(word.vector.shape[0]))print(word.vector[:10])

Neighbors (Synonms) of Obama
------------------------------
Bush
Reagan
Clinton
Ahmadinejad
Nixon
Karzai
McCain
Biden
Huckabee
Lula


The first 10 dimensions out the 256 dimensions

[-2.57382345  1.52175975  0.51070285  1.08678675 -0.74386948 -1.18616164
  2.92784619 -0.25694436 -1.40958667 -2.39675403]

形态

word=Text("Preprocessing is an essential step.").words[0]print(word.morphemes)

[u'Pre', u'process', u'ing']

音译

frompolyglot.transliterationimportTransliteratortransliterator=Transliterator(source_lang="en",target_lang="ru")print(transliterator.transliterate(u"preprocessing"))

препрокессинг