漂亮的汤找到（）并没有找到所有的结果的类

Question

我有一些代码试图在tracklist容器中提取所有html内容，它应该有88首歌曲。信息肯定在那里（我打印了汤来检查），所以我不确定为什么前30react-contextmenu-wrapper之后的所有东西都丢失了。你知道吗

from bs4 import BeautifulSoup
from urllib.request import urlopen
import re


spotify = 'https://open.spotify.com/playlist/3vSFv2hZICtgyBYYK6zqrP'
html = urlopen(spotify)
soup = BeautifulSoup(html, "html5lib")

main = soup.find(class_ = 'tracklist-container')
print(main)

谢谢你的帮助。 打印输出电流如下：

                  1.
              </div></div><div class="tracklist-col name"><div class="top-align track-name-wrapper"><span class="track-name" dir="auto">Move On - Teen Daze Remix</span><span class="artists-albums"><a href="/artist/3HrczLBDJXJu6dJWEMbKHa" tabindex="-1"><span dir="auto">Garden City Movement</span></a>     • <a href="/album/4p8FxnuYzykCcN7xbjA9jq" tabindex="-1"><span dir="auto">Entertainment</span></a></span></div></div><div class="tracklist-col explicit"></div><div class="tracklist-col duration"><div class="top-align"><span class="total-duration">5:11</span><span class="preview-duration">0:30</span></div></div><div class="progress-bar-outer"><div class="progress-bar"></div></div></li><li class="tracklist-row js-track-row tracklist-row--track track-has-preview" data-position="2" role="button" tabindex="0"><div class="tracklist-col position-outer"><div class="play-pause top-align"><svg aria-label="Play" class="svg-play" role="button"><use xlink:href="#icon-play" xmlns:xlink="http://www.w3.org/1999/xlink"></use></svg><svg aria-label="Pause" class="svg-pause" role="button"><use xlink:href="#icon-pause" xmlns:xlink="http://www.w3.org/1999/xlink"></use></svg></div><div class="tracklist-col__track-number position top-align">
                  2.
              </div></div><div class="tracklist-col name"><div class="top-align track-name-wrapper"><span class="track-name" dir="auto">Flicker</span><span class="artists-albums"><a href="/artist/4qpWUfUAeI34HzvCORn1ze" tabindex="-1"><span dir="auto">Forhill</span></a>     • <a href="/album/0gfz1Tbst40swwL357cRqG" tabindex="-1"><span dir="auto">Flicker</span></a></span></div></div><div class="tracklist-col explicit"></div><div class="tracklist-col duration"><div class="top-align"><span class="total-duration">3:45</span><span class="preview-duration">0:30</span></div></div><div class="progress-bar-outer"><div class="progress-bar"></div></div></li><li class="tracklist-row js-track-row tracklist-row--track track-has-preview" data-position="3" role="button" tabindex="0"><div class="tracklist-col position-outer"><div class="play-pause top-align"><svg aria-label="Play" class="svg-play" role="button"><use xlink:href="#icon-play" xmlns:xlink="http://www.w3.org/1999/xlink"></use></svg><svg aria-label="Pause" class="svg-pause" role="button"><use xlink:href="#icon-pause" xmlns:xlink="http://www.w3.org/1999/xlink"></use></svg></div><div class="tracklist-col__track-number position top-align">

。。。你知道吗

                  30.
              </div></div><div class="tracklist-col name"><div class="top-align track-name-wrapper"><span class="track-name" dir="auto">Trapdoor</span><span class="artists-albums"><a href="/artist/3nqTFzjmi1LLM6pn0TRMv8" tabindex="-1"><span dir="auto">Eagle Eyed Tiger</span></a>     • <a href="/album/48Q8Jgk1x4wiHWecV4nlz6" tabindex="-1"><span dir="auto">Future or Past</span></a></span></div></div><div class="tracklist-col explicit"></div><div class="tracklist-col duration"><div class="top-align"><span class="total-duration">4:14</span><span class="preview-duration">0:30</span></div></div><div class="progress-bar-outer"><div class="progress-bar"></div></div></li></ol><button class="link js-action-button" data-track-type="view-all-button">View all on Spotify</button></div>

最后一项应该是第88项。感觉我的搜索结果被截短了。你知道吗

Answer 1

既然你的思路似乎是对的，我并没有试图解决全部问题，而是试图给你一个有用的提示：做动态webscraping。你知道吗

“为什么是硒？靓汤还不够吗？

用Python刮网通常只需要使用漂亮的汤就可以达到目的。BeautifulSoup是一个非常强大的库，它通过遍历DOM（文档对象模型）使web抓取更容易实现。但它只做静态刮削。静态刮取忽略JavaScript。它从服务器获取网页而不需要浏览器的帮助。你得到的正是你在“查看页面源”中看到的，然后你把它切块。如果您要查找的数据仅在“查看页面源”中可用，则无需再进一步。但是，如果您需要组件中存在的数据，而这些数据是通过单击JavaScript链接呈现的，那么动态刮取就可以解决问题。靓汤与硒的结合将起到动态刮削的作用。Selenium通过python自动化web浏览器交互。因此，JavaScript链接呈现的数据可以通过Selenium自动点击按钮获得，然后可以通过Beautiful Soup提取。” https://medium.com/ymedialabs-innovation/web-scraping-using-beautiful-soup-and-selenium-for-dynamic-page-2f8ad15efe25

下面是我在DOM中的30首歌曲结尾看到的内容，它指向一个按钮：

    </li>
   </ol>
   <button class="link js-action-button" data-track-type="view-all-button">
    View all on Spotify
   </button>
  </div>