拆分可以包含引号字符串的括号分隔文本

Question

我正在试着拆分一些文本。基本上，我想分开一级括号，比如"('1','a',NULL),(2,'b')"=>；["('1','a',NULL)", "(2,'b')]"，但是我需要知道里面可能有引号的字符串。它至少需要满足以下要求py.测试公司名称：

from splitter import split_text


def test_normal():
    assert split_text("('1'),('2')") == ["('1')", "('2')"]
    assert split_text("(1),(2),(3)") == ["(1)", "(2)", "(3)"]


def test_complex():
    assert split_text("('1','a'),('2','b')") == ["('1','a')", "('2','b')"]
    assert split_text("('1','a',NULL),(2,'b')") == ["('1','a',NULL)", "(2,'b')"]


def test_apostrophe():
    assert split_text("('\\'1','a'),('2','b')") == ["('\\'1','a')", "('2','b')"]


def test_coma_in_string():
    assert split_text("('1','a,c'),('2','b')") == ["('1','a,c')", "('2','b')"]


def test_bracket_in_string():
    assert split_text("('1','a)c'),('2','b')") == ["('1','a)c')", "('2','b')"]


def test_bracket_and_coma_in_string():
    assert split_text("('1','a),(c'),('2','b')") == ["('1','a),(c')", "('2','b')"]


def test_bracket_and_coma_in_string_apostrophe():
    assert split_text("('1','a\\'),(c'),('2','b')") == ["('1','a\\'),(c')", "('2','b')"]

我试过以下方法：

1）正则表达式

这看起来是最好的解决方案，但不幸的是，我没有找到任何能满足所有测试要求的东西。在

我最好的办法是：

^{pr2}$

但显然，这是相当简单的，失败了test_bracket_and_coma_in_string和{}。在

2）有限状态机类解

我试着自己编写FSM代码：

OUTSIDE, IN_BRACKETS, IN_STRING, AFTER_BACKSLASH = range(4)


def split_text(text):
    state = OUTSIDE
    read = []
    result = []

    for character in text:
        if state == OUTSIDE:
            if character == ',':
                result.append(''.join(read))
                read = []
            elif character == '(':
                read.append(character)
                state = IN_BRACKETS
            else:
                read.append(character)

        elif state == IN_BRACKETS:
            read.append(character)
            if character == ')':
                state = OUTSIDE
            elif character == "'":
                state = IN_STRING

        elif state == IN_STRING:
            read.append(character)
            if character == "'":
                state = IN_BRACKETS
            elif character == '\\':
                state = AFTER_BACKSLASH

        elif state == AFTER_BACKSLASH:
            read.append(character)
            state = IN_STRING

    result.append(''.join(read))  # The rest of string
    return result

它工作正常，通过所有测试，但速度很慢。在

3）py解析

from pyparsing import QuotedString, ZeroOrMore, Literal, Group, Suppress, Word, nums

null_value = Literal('NULL')
number_value = Word(nums)
string_value = QuotedString("'", escChar='\\', unquoteResults=False)
value = null_value | number_value | string_value
one_bracket = Group(Literal('(') + value + ZeroOrMore(Literal(',') + value) + Literal(')'))
all_brackets = one_bracket + ZeroOrMore(Suppress(',') + one_bracket)


def split_text(text):
    parse_result = all_brackets.parseString(text)
    return [''.join(a) for a in parse_result]

也通过了所有测试，但令人惊讶的是，它比溶液2慢得多。在

有什么办法使解决方案快速而可靠吗？我有一种感觉，我错过了一些显而易见的东西。在

Answer 1

一种方法是使用较新的^{}模块，该模块支持(*SKIP)(*FAIL)功能：

import regex as re

def split_text(text):
    rx = r"""'.*?(?<!\\)'(*SKIP)(*FAIL)|(?<=\)),(?=\()"""
    return re.split(rx, text)

细细地说：

^{pr2}$

这个succeeds on all your examples。在

Answer 2

我做了这个，它在给定的测试中起作用。在

tests = ["('1'),('2')",
"(1),(2),(3)",
"('1','a'),('2','b')",
"('1','a',NULL),(2,'b')",
"('\\'1','a'),('2','b')",
"('1','a,c'),('2','b')",
"('1','a)c'),('2','b')",
"('1','a),(c'),('2','b')",
"('1','a\\'),(c'),('2','b')"]

for text in tests:
    tmp = ''
    res = []
    bracket = 0
    quote = False

    for idx,i in enumerate(text):
        if i=="'":
            if text[idx-1]!='\\':
                quote = not quote
            tmp += i
        elif quote:
            tmp += i
        elif i==',':
            if bracket: tmp += i
            else:   pass
        else:
            if i=='(':      bracket += 1
            elif i==')':    bracket -= 1

            if bracket:   tmp += i
            else:
                tmp += i
                res.append(tmp)
                tmp = ''

    print res

输出：

^{pr2}$

代码还有改进的余地，欢迎编辑。：）

Answer 3

这是一个正则表达式，它似乎可以工作并通过所有测试。在实际数据上运行它比用Python实现的有限状态机快6倍。在

PATTERN = re.compile(
    r"""
        \(  # Opening bracket

            (?:

            # String
            (?:'(?:
               (?:\\')|[^']  # Either escaped apostrophe, or other character
               )*'
            )
            |
            # or other literal not containing right bracket
            [^')]

            )

            (?:, # Zero or more of them separated with comma following the first one

            # String
            (?:'(?:
               (?:\\')|[^']  # Either escaped apostrophe, or other character
               )*'
            )
            |
            # or other literal
            [^')]

            )*

        \)  # Closing bracket
    """,
    re.VERBOSE)


def split_text(text):
    return PATTERN.findall(text)