获取匹配记录的索引

Question

我有一个函数，它将字符串拆分为单词，然后在数据帧中查找单词，如果找到了，则使用for循环搜索该行，我不想这样做，因为它会使大型数据集的速度太慢。我想使用row[value]，不想为每个匹配的单词循环整个df

我是python新手，我搜索了很多，但是可以得到我想要的，我找到了index.tolist（），但是我不想做列表，我只需要第一个匹配值的索引

任何帮助或解决方法都将不胜感激

def cal_nega_mean(my_string):
  mean = 0.00
  mean_tot = 0
  mean_sum = 0.00
  for word in my_string.split():
    if word in df.values: #at this point if it founds then get index, so that i dont have to use  for loop in next line
      for index, row in df.iterrows(): #want to change 
        if word == row.word:   # this part
          if row['value'] < -0.40:
            mean_tot += 1
            mean += row['value']
            break
  if mean_tot == 0:
    return 0
  mean = mean_sum / mean_tot
  return round(mean,2)

示例字符串输入，有超过300k个字符串

my_string = "i have a problem with my python code" 
cal_nega_mean(my_string)
# and i am using this to get return for all records
df_tweets['intensity'] = df_tweets['tweets'].apply(lambda row: cal_nega_mean(row))

要从中搜索的数据帧

df 

index   word      value  ...

  1     python    -0.56

  2     problem   -0.78

  3     alpha     -0.91

   . . .

 9000   last    -0.41

Answer 1

Pandas有一些有用的文本处理功能，应该可以帮助您。我建议你使用pd.Series.str.contains()

def cal_nega_mean(my_string):
    words = '|'.join(my_string.split())
    matches = df['word'].str.contains(words, regex=True)
    mask = (df['value'] >= -0.40) & matches # don't need value >= -0.40 if you just drop those rows
    mean_tot = mask.sum()
    mean_sum = df[mask]['value'].sum()
    mean = mean_sum / mean_tot
    return round(mean, 2)

不相关，但我也建议您删除带有“value”的行<-0.40，因为你忽略了它们

我还没有机会测试这个，但它应该可以完成任务，而且它已经矢量化了

Answer 2

您可以尝试使用i = df[df.word == word].index[0]获取满足条件df.word == word的第一行的索引。一旦有了索引，就可以用df.loc切掉行

def cal_nega_mean(my_string):
    mean = 0.00
    mean_tot = 0
    mean_sum = 0.00
    for word in my_string.split():
        try:
            i = df[df.word == word].index[0]
        except:
            continue
        row = df.loc[i]
        if row['value'] < -0.40:
            mean_tot += 1
            mean += row['value']
            break
    if mean_tot == 0:
        return 0
    mean = mean_sum / mean_tot
    return round(mean,2)

Answer 3

下面是一种使用字典的方法，您可以将word: value转换为键、值存储并将其用作查找：

word_look_up = dict(zip(df['word'], df['value']))


def cal_nega_mean(my_string): 
    mean = 0.0
    mean_tot = 0
    mean_sum = 0.00
    words = [word for word in my_string.split() if word in word_look_up]

    if not any(words): # if no word found
        return 0
    else:
        for word in words:
            value = word_look_up[word]
            if value < -0.40:
                mean_tot += 1
                mean += value
                break

    mean = mean / mean_tot
    return round(mean, 2)


df['intensity'] = df['word'].apply(cal_nega_mean)