2024-05-16 07:57:32 发布
网友
我用nltk.POS_tag()对一些单词进行了POS标记,因此它们被赋予了treebank标记。我想用已知的POS标签把这些词引申出来,但我不知道怎么做。我在看WordNetLemmatizer,但我不确定如何将树银行POS标签转换为lemmatizer接受的标签。如何简单地执行此转换,或者是否有使用树库标记的lemmatizer?
wordnet lemmatizer只知道四个词类(ADJ、ADV、NOUN和VERB),只有NOUN和VERB规则可以做任何特别有趣的事情。treebank标记集中的名词词组都以NN开头,动词标记都以VB开头,形容词标记以JJ开头,副词标记以RB开头。因此,从一组标签转换到另一组非常容易,比如:
from nltk.corpus import wordnet morphy_tag = {'NN':wordnet.NOUN,'JJ':wordnet.ADJ,'VB':wordnet.VERB,'RB':wordnet.ADV}[penn_tag[:2]]
正如@engineercoding在对@rmalouf答案的评论中指出的那样,与WordNet相比,Treebank中有更多的标记,请参见here for details。
以下映射覆盖尽可能多的基,它还显式地定义了在WordNet中没有匹配项的POS标记:
# Create a map between Treebank and WordNet from nltk.corpus import wordnet as wn # WordNet POS tags are: NOUN = 'n', ADJ = 's', VERB = 'v', ADV = 'r', ADJ_SAT = 'a' # Descriptions (c) https://web.stanford.edu/~jurafsky/slp3/10.pdf tag_map = { 'CC':None, # coordin. conjunction (and, but, or) 'CD':wn.NOUN, # cardinal number (one, two) 'DT':None, # determiner (a, the) 'EX':wn.ADV, # existential ‘there’ (there) 'FW':None, # foreign word (mea culpa) 'IN':wn.ADV, # preposition/sub-conj (of, in, by) 'JJ':[wn.ADJ, wn.ADJ_SAT], # adjective (yellow) 'JJR':[wn.ADJ, wn.ADJ_SAT], # adj., comparative (bigger) 'JJS':[wn.ADJ, wn.ADJ_SAT], # adj., superlative (wildest) 'LS':None, # list item marker (1, 2, One) 'MD':None, # modal (can, should) 'NN':wn.NOUN, # noun, sing. or mass (llama) 'NNS':wn.NOUN, # noun, plural (llamas) 'NNP':wn.NOUN, # proper noun, sing. (IBM) 'NNPS':wn.NOUN, # proper noun, plural (Carolinas) 'PDT':[wn.ADJ, wn.ADJ_SAT], # predeterminer (all, both) 'POS':None, # possessive ending (’s ) 'PRP':None, # personal pronoun (I, you, he) 'PRP$':None, # possessive pronoun (your, one’s) 'RB':wn.ADV, # adverb (quickly, never) 'RBR':wn.ADV, # adverb, comparative (faster) 'RBS':wn.ADV, # adverb, superlative (fastest) 'RP':[wn.ADJ, wn.ADJ_SAT], # particle (up, off) 'SYM':None, # symbol (+,%, &) 'TO':None, # “to” (to) 'UH':None, # interjection (ah, oops) 'VB':wn.VERB, # verb base form (eat) 'VBD':wn.VERB, # verb past tense (ate) 'VBG':wn.VERB, # verb gerund (eating) 'VBN':wn.VERB, # verb past participle (eaten) 'VBP':wn.VERB, # verb non-3sg pres (eat) 'VBZ':wn.VERB, # verb 3sg pres (eats) 'WDT':None, # wh-determiner (which, that) 'WP':None, # wh-pronoun (what, who) 'WP$':None, # possessive (wh- whose) 'WRB':None, # wh-adverb (how, where) '$':None, # dollar sign ($) '#':None, # pound sign (#) '“':None, # left quote (‘ or “) '”':None, # right quote (’ or ”) '(':None, # left parenthesis ([, (, {, <) ')':None, # right parenthesis (], ), }, >) ',':None, # comma (,) '.':None, # sentence-final punc (. ! ?) ':':None # mid-sentence punc (: ; ... – -) }
wordnet lemmatizer只知道四个词类(ADJ、ADV、NOUN和VERB),只有NOUN和VERB规则可以做任何特别有趣的事情。treebank标记集中的名词词组都以NN开头,动词标记都以VB开头,形容词标记以JJ开头,副词标记以RB开头。因此,从一组标签转换到另一组非常容易,比如:
正如@engineercoding在对@rmalouf答案的评论中指出的那样,与WordNet相比,Treebank中有更多的标记,请参见here for details。
以下映射覆盖尽可能多的基,它还显式地定义了在WordNet中没有匹配项的POS标记:
相关问题 更多 >
编程相关推荐