如何从HTML内容中移除"&nbsp"？

Question

我有一个HTML页面，内容如下：

<div class="theater">
    <div class="desc" id="theater_16109207495969942346">
        <h2 class="name"><a href="/movies?near=pune&amp;tid=df8f66de0a592b4a" id="link_1_theater_16109207495969942346">Esquare Victory Camp</a></h2>
        <div class="info">site no 2429,general thimayya road, camp contonment,oppositekayani bakery, Pune - 020 2613 2975
            <a class="fl" href="" target="_top"></a>
        </div>
    </div>
    <div class="showtimes">
        <div class="show_left">
            <div class="movie">
                <div class="name"><a href="/movies?near=pune&amp;mid=1cdcf90092189400">Hawaa Hawaai</a>
                </div><span class="info">Drama - Hindi</span>
                <div class="times"><span style="color:#666"><span style="padding:0 "></span>
                    <!-- -->10:30am</span><span style="color:#666"><span style="padding:0 "> &amp;nbsp</span>
                    <!-- -->3:45</span><span style="color:#666"><span style="padding:0 "> &amp;nbsp</span>
                    <!-- -->6:00</span><span style="color:"><span style="padding:0 "> &amp;nbsp</span>
                    <!-- -->8:30pm</span>
                </div>
            </div>
        </div>
        <div class="show_right">
            <div class="movie">
                <div class="name"><a href="/movies?near=pune&amp;mid=6b59ad39004d895b">The Amazing Spider Man 2</a>
                </div><span class="info">Action/Adventure/Thriller - English - <a class="fl" href="/url?q=http://www.youtube.com/watch%3Fv%3DSCjCk59PIzw&amp;sa=X&amp;oi=movies&amp;ii=0&amp;usg=AFQjCNGpVM5U04h0acABA7eApb6EIO4Ejw">Trailer</a></span>
                <div class="times"><span style="color:#666"><span style="padding:0 "></span>
                    <!-- -->1:00</span><span style="color:"><span style="padding:0 "> &amp;nbsp</span>
                    <!-- -->10:45pm</span>
                </div>
            </div>
        </div>
        <p class="clear"></p>
    </div>
</div>

在这个页面中，我们可以看到很多地方都有 &nbsp。还有很多其他的unicode字符。我想提取这个页面的内容。

我现在的做法是：

def removeNonAscii(s): return "".join(i for i in s if ord(i)<128)

myName = soup.findAll("div", {"class" : "theater"})
for x in myName:
   xt = str(x)
   print removeNonAscii(xt)
   print "<br>"

结果是：

Esquare Victory Camp
site no 2429,general thimayya road, camp contonment,oppositekayani bakery, Pune - 020 2613 2975
Hawaa Hawaai
Drama - Hindi
10:30am &nbsp3:45 &nbsp6:00 &nbsp8:30pm
The Amazing Spider Man 2
Action/Adventure/Thriller - English - Trailer
1:00 &nbsp10:45pm

一切看起来都不错，除了 &nbsp。我尝试过替换 &nbsp，也搜索过其他解决方案，但还是没有找到办法。我觉得没有 ; 的 &nbsp 可能是个问题。请问怎么才能去掉 &nbsp 呢？

Answer 1

lxml.html 可能是一个更合适的库，它可以把 "&nbsp" 和其他 HTML 标签转换成正确的字符。

import lxml.html
import lxml.html.clean
html = """your HTML"""
doc = lxml.html.fromstring(html)
cleaner = lxml.html.clean.Cleaner(style=True)
doc = cleaner.clean_html(doc)
text = doc.text_content()

Answer 2

根据你想要去掉不换行空格的处理阶段，这个过程可以很简单。比如，当你处理你提供的HTML片段时，你只需要从文本元素中去掉字符串“&nbsp”就可以了：

s = """your HTML"""
soup = BeautifulSoup(s)
texts = soup.find_all(text=True)
for t in texts:
   newtext = t.replace("&nbsp", "")
   t.replace_with(newtext)