在一个poin之前的列表中看到一个bigram的次数

Question

假设我有这样一段文字：

'he is hdajs asdas da he is not asd as da s i am a da daas you am a'

我已经根据这段文字创建了所有的大字：

>>> bigrams_
[('he', 'is'), ('is', 'hdajs'), ('hdajs', 'asdas'), ('asdas', 'da'), ('da', 'he'), ('he', 'is'), ('is', 'not'), ('not', 'asd'), ('asd', 'as'), ('as', 'da'), ('da', 's'), ('s', 'i'), ('i', 'am'), ('am', 'a'), ('a', 'da'), ('da', 'daas'), ('daas', 'you'), ('you', 'am'), ('am', 'a')]

现在我想创建一个新的bigram列表，其中每个bigram的第一个元素将是一个位置索引，它显示从上述格式到文本中某个点的某个bigram被看到了多少次，第二个元素将是来自初始列表的单词的bigram。例如，在上面的列表中，最后一个元素('am', 'a')已经出现了2次，因此在新列表中，它将对应于这个bigram:(2, ('am', 'a'))。 这是一个简洁的python方法。你知道吗

Answer 1

我喜欢基于@JonClements itertoolscount的解决方案（+1），但我不认为defaultdict是必要的：

from itertools import count

text = 'he is hdajs asdas da he is not asd as da s i am a da daas you am a'

words = text.split()

bigrams = zip(words, words[1:])

seen = dict()

result = [(next(seen.setdefault(bigram, count(1))), bigram) for bigram in bigrams]

print(*result, sep='\n')

输出

(1, ('he', 'is'))
(1, ('is', 'hdajs'))
(1, ('hdajs', 'asdas'))
(1, ('asdas', 'da'))
(1, ('da', 'he'))
(2, ('he', 'is'))
(1, ('is', 'not'))
(1, ('not', 'asd'))
(1, ('asd', 'as'))
(1, ('as', 'da'))
(1, ('da', 's'))
(1, ('s', 'i'))
(1, ('i', 'am'))
(1, ('am', 'a'))
(1, ('a', 'da'))
(1, ('da', 'daas'))
(1, ('daas', 'you'))
(1, ('you', 'am'))
(2, ('am', 'a'))

Answer 2

你可以试试这个：

s = 'he is hdajs asdas da he is not asd as da s i am a da daas you am a'
s1 = s.split()
new_data = list(set([(s.count(' '.join(b)), b) for b in [(s1[i], s1[i+1]) for i in range(len(s1)-1)]]))

输出：

[(2, ('am', 'a')), (1, ('da', 'daas')), (1, ('not', 'asd')), (1, ('s', 'i')), (1, ('da', 'he')), (1, ('you', 'am')), (2, ('he', 'is')), (1, ('is', 'not')), (1, ('asdas', 'da')), (1, ('asd', 'as')), (1, ('hdajs', 'asdas')), (1, ('a', 'da')), (1, ('daas', 'you')), (2, ('as', 'da')), (1, ('da', 's')), (1, ('is', 'hdajs')), (1, ('i', 'am'))]

Answer 3

您可以使用默认值为count的defaultdict对象，并逐步获取该键计数器的next值，例如：

from collections import defaultdict
from itertools import count

dd = defaultdict(lambda: count(1))
bigrams = [('he', 'is'), ('is', 'hdajs'), ('hdajs', 'asdas'), ('asdas', 'da'), ('da', 'he'), ('he', 'is'), ('is', 'not'), ('not', 'asd'), ('asd', 'as'), ('as', 'da'), ('da', 's'), ('s', 'i'), ('i', 'am'), ('am', 'a'), ('a', 'da'), ('da', 'daas'), ('daas', 'you'), ('you', 'am'), ('am', 'a')]
with_count = [(next(dd[bigram]), bigram) for bigram in bigrams]

给你：

[(1, ('he', 'is')),
 (1, ('is', 'hdajs')),
 (1, ('hdajs', 'asdas')),
 (1, ('asdas', 'da')),
 (1, ('da', 'he')),
 (2, ('he', 'is')),
 (1, ('is', 'not')),
 (1, ('not', 'asd')),
 (1, ('asd', 'as')),
 (1, ('as', 'da')),
 (1, ('da', 's')),
 (1, ('s', 'i')),
 (1, ('i', 'am')),
 (1, ('am', 'a')),
 (1, ('a', 'da')),
 (1, ('da', 'daas')),
 (1, ('daas', 'you')),
 (1, ('you', 'am')),
 (2, ('am', 'a'))]