朴素贝叶斯实现 - 准确性问题

Question

编辑：可以在以下链接找到正确的代码版本： https://github.com/a7x/NaiveBayes-Classifier

我使用了来自openClassroom的数据，开始在Python中做一个简单的朴素贝叶斯分类器。步骤就是常规的训练和预测。我有几个问题，想知道为什么准确率这么差。

在训练时，我通过以下公式计算了对数似然：

log( P ( word | spam ) +1 ) /( spamSize + vocabSize .)

我的问题是：为什么在这种情况下要加上vocabSize呢？这是正确的做法吗？下面是使用的代码：

#This is for training.     Calculate all probabilities and store them in a vector. Better to store it in a file  for easier access 
from __future__ import division
import sys,os
''' 
1. The spam and non-spam is already 50%  . So they by default are 0.5
2. Now we need to calculate probability of each word    , in spam and non-spam separately
  2.1  we can make two dictionaries, defaultdicts basically,  for spam and non-spam 
  2.2 When time comes to calculate probabilities, we just need to substitute values
'''
from collections import *
from math import *

spamDict = defaultdict(int)
nonspamDict = defaultdict(int)
spamFolders = ["spam-train"]
nonspamFolders = ["nonspam-train"]
path = sys.argv[1] #Base path
spamVector = open(sys.argv[2],'w') #WRite all spam values into this 
nonspamVector = open(sys.argv[3],'w') #Non-spam values

#Go through all files in spam and  iteratively add values
spamSize = 0
nonspamSize = 0
vocabSize = 264821
for f in os.listdir(os.path.join(path,spamFolders[0])):
    data = open(os.path.join(path,spamFolders[0],f),'r')

    for line in data:
        words = line.split(" ")
        spamSize = spamSize + len(words)
        for w in words:
            spamDict[w]+=1

for f in os.listdir(os.path.join(path,nonspamFolders[0])):
    data = open(os.path.join(path,nonspamFolders[0],f),'r')
    for line in data:
        words = line.split(" ")
        nonspamSize = nonspamSize + len(words)
        for w in words:

            nonspamDict[w]+=1
logProbspam = {}
logProbnonSpam = {} #This is to store the log probabilities
for k in spamDict.keys():
    #Need to calculate P(x | y = 1)

    numerator =  spamDict[k] + 1  # Frequency
    print 'Word',k,' frequency',spamDict[k]
    denominator = spamSize + vocabSize
    p = log(numerator/denominator)
    logProbspam[k] = p
for k in nonspamDict.keys():
    numerator = nonspamDict[k] + 1 #frequency
    denominator = nonspamSize + vocabSize
    p = log(numerator/denominator)
    logProbnonSpam[k] = p

for k in logProbnonSpam.keys():
    nonspamVector.write(k+" "+str(logProbnonSpam[k])+"\n")
for k in logProbspam.keys():
    spamVector.write(k+" "+str(logProbspam[k])+"\n")

在预测时，我只是把一封邮件拆分成单词，分别计算垃圾邮件和非垃圾邮件的概率，然后把它们相乘0.5。哪个概率更高就标记为那个类别。下面是代码：

http://pastebin.com/8Y6Gm2my（由于某种原因，Stackoverflow又出问题了 :-/)

编辑：我已经去掉了 spam = spam + 1 的部分。现在我只是忽略那些单词

问题：我的准确率非常差。如下面所述。

    No of files in spam is 130
    No. of spam in  ../NaiveBayes/spam-test  is  53  no. of non-spam 77
    No of files in non-spam is 130
    No. of spam in  ../NaiveBayes/nonspam-test/  is  6  no. of non-spam 124

请告诉我我哪里出错了。我觉得准确率低于50%说明实现中一定有明显的错误。

机器学习概率计算数据集准确率分类器训练过程朴素贝叶斯对数似然

朴素贝叶斯实现 - 准确性问题

2 个回答

撰写回答