如何使用python在文本文件上循环（以删除标记和规范化）

Question

我试图在一个文本文件上循环以删除标记、标点符号和停止词。我已经使用Python3.8.3 Beautiful Soup从站点上刮取了文本（报纸文章）。它返回一个我保存为文件的列表。但是，我不知道如何使用循环来处理文本文件中的所有项

在下面的代码中，我使用了listfilereduced.text（包含来自一篇新闻文章的数据，link to listfilereduced.txt here），但是我希望在listfile.text（包含来自多篇文章的数据，link to listfile.text）上运行此代码。任何帮助都将不胜感激

另外，文本为非英语脚本，但标签均为英语

#This text file contains just one news item

with open('listfilereduced.txt', 'r', encoding='utf8') as my_file:
    rawData = my_file.read()
    print(rawData)

#Separating body text from other data

articleStart = rawData.find("<div class=\"story-element story-element-text\">")
articleData = rawData[:articleStart]
articleBody = rawData[articleStart:]
print(articleData)
print("*******")
print(articleBody)
print("*******")


#First, I define a function to strip tags from the body text

def stripTags(pageContents):
    insideTag = 0
    text = ''

    for char in pageContents:
        if char == '<':
            insideTag = 1
        elif (insideTag == 1 and char == '>'):
            insideTag = 0
        elif insideTag == 1:
            continue
        else:
            text += char
    return text

#Calling the function

articleBodyText = stripTags(articleBody)
print(articleBodyText)

 
#Isolating article title and publication date

TitleEndLoc = articleData.find("</h1>")
dateStartLoc = articleData.find("<div class=\"storyPageMetaData-m__publish-time__19bdV\">")
dateEndLoc=articleData.find("<div class=\"meta-data-icons storyPageMetaDataIcons-m__icons__3E4Xg\">")
titleString = articleData[:TitleEndLoc]
dateString = articleData[dateStartLoc:dateEndLoc]


#Call stripTags function to clean
articleTitle= stripTags(titleString)
articleDate = stripTags(dateString)

print(articleTitle)
print(articleDate)

#Cleaning the date a bit more

startLocDate = articleDate.find(":")
endLocDate = articleDate.find(",")
articleDateClean = articleDate[startLocDate+2:endLocDate]
print(articleDateClean)


#save all this data to a dictionary that saves the title, data and the body text 
PAloTextDict = {"Title": articleTitle, "Date": articleDateClean, "Text": articleBodyText}
print(PAloTextDict)

#Normalize text by: 

#1. Splitting paragraphs of text into lists of words
articleBodyWordList = articleBodyText.split()
print(articleBodyWordList)


#2.Removing punctuation and stopwords

#https://bnlp.readthedocs.io/en/latest/

from bnlp.corpus import stopwords, punctuations

#A. Remove punctuation first

listNoPunct = []

for word in articleBodyWordList:
    for mark in punctuations:
        word=word.replace(mark, '')
    listNoPunct.append(word)
print(listNoPunct)



#B. removing stopwords
banglastopwords = stopwords()
print(banglastopwords)
 
cleanList=[]
for word in listNoPunct:
    if word in banglastopwords:
        continue
    else:
        cleanList.append(word)
print(cleanList)