livelex 0.3.0

解析

解析（更好：词法分析）文本总是从一个称为 根词典。那本词典里的规则是一个接一个地尝试的。一到那里 是一个匹配项，一个标记与匹配文本生成的位置 文本和规则中指定的操作。如果目标是 指定，解析将在另一个词典中继续。在

令牌放在树结构中。每一个活跃的词汇都创造了一个语境 包含标记和子上下文的列表。如果目标弹回到 上一个词典，上一个上下文又变成了当前的。在

所有的标记和上下文都指向它们的父对象，因此可以进行操纵 并以多种方式查询树结构。在

树的结构由TreeBuilder构建，请参见树和 treebuilder模块。词根是承载词根词汇的上下文。 根上下文包含标记和/或其他上下文。在

TreeBuilder能够在后台线程中标记文本，并且 也可以重新生成文本的一部分，巧妙地重用前面的部分 生成令牌（如果可能）。在

迭代查询

Token和Context都有很多方法来遍历树，因为 获取父节点、子节点或同级节点。上下文有各种find（） 方法快速查找文本中某个位置的标记或上下文。在

使用上下文.查询属性可以构建类似XPath的筛选链 奎因根据文本、动作或词典快速查找标记或上下文。 这在查询模块中进行了描述。在

示例

下面是一些如何创建语言类然后使用它的示例：

importlivelexfromliveleximport(Language,lexicon,words,bygroup,bymatch,bytext,default_action,default_target,skip,tomatch,)classMyLang(Language):"""A Language represents a set of Lexicons comprising a specific language.

    A Language is never instantiated. The class itself serves as a namespace
    and can be inherited from.



    """@lexicon(re_flags=0)defroot(cls):yieldr'"',"string",cls.stringyieldr'\(',"paren",cls.parenthesizedyieldr'\d+',"number"yieldr'%',"comment",cls.commentyieldr'[,.!?]',"punctuation"yieldr'\w+',"word"@lexicondefstring(cls):yieldr'\\[\\"]','string escape'yieldr'"',"string",-1yielddefault_action,"string"@lexicon(re_flags=re.MULTILINE)defcomment(cls):yieldr'$',"comment",-1yieldr'XXX|TODO',"todo"yielddefault_action,"comment"@lexicondefparenthesized(cls):yieldr'\)',"paren",-1yield fromcls.root()s=r"""
This is (an example) text with 12 numbers
and "a string with \" escaped characters",
and a % comment that TODO lasts until the end
of the line.
""">>>importlivelex>>>tree=livelex.root(MyLang.root,s)>>>tree.dump()<ContextMyLang.rootat1-144(20children)>├╴<Token'This'at1(word)>├╴<Token'is'at6(word)>├╴<Token'('at9(paren)>├╴<ContextMyLang.parenthesizedat10-21(3children)>│├╴<Token'an'at10(word)>│├╴<Token'example'at13(word)>│╰╴<Token')'at20(paren)>├╴<Token'text'at22(word)>├╴<Token'with'at27(word)>├╴<Token'12'at32(number)>├╴<Token'numbers'at35(word)>├╴<Token'and'at43(word)>├╴<Token'"'at47(string)>├╴<ContextMyLang.stringat48-84(4children)>│├╴<Token'a string with 'at48(string)>│├╴<Token'\\"'at62(stringescape)>│├╴<Token' escaped characters'at64(string)>│╰╴<Token'"'at83(string)>├╴<Token','at84(punctuation)>├╴<Token'and'at86(word)>├╴<Token'a'at90(word)>├╴<Token'%'at92(comment)>├╴<ContextMyLang.commentat93-131(3children)>│├╴<Token' comment that 'at93(comment)>│├╴<Token'TODO'at107(todo)>│╰╴<Token' lasts until the end'at111(comment)>├╴<Token'of'at132(word)>├╴<Token'the'at135(word)>├╴<Token'line'at139(word)>╰╴<Token'.'at143(punctuation)>>>>tree.find_token(50)<Token'a string with 'at48(string)>>>>tree.find_token(50).parent<ContextMyLang.stringat48-84(4children)>>>>d=livelex.Document(MyLang.root,s)>>>d<Document'\nThis is (an example) text w...'>>>>print(d.text())Thisis(anexample)textwith12numbersand"a string with \" escaped characters",anda%commentthatTODOlastsuntiltheendoftheline.>>>d[50:56]'string'>>>withd:...d[9:12]='(a "much longer'...d[20:20]='"'...>>>print(d.text())Thisis(a"much longer example")textwith12numbersand"a string with \" escaped characters",anda%commentthatTODOlastsuntiltheendoftheline.>>>d.get_root()[3].dump()<ContextMyLang.parenthesizedat10-34(4children)>├╴<Token'a'at10(word)>├╴<Token'"'at12(string)>├╴<ContextMyLang.stringat13-33(2children)>│├╴<Token'much longer example'at13(string)>│╰╴<Token'"'at32(string)>╰╴<Token')'at33(paren)>