<p>下面的脚本根据给定的搜索关键字从网页中提取所有链接。但它不会超出第一页。尽管下面的代码可以通过操作URL中的页码(如<a href="https://stackoverflow.com/questions/44968594/beautiful-soup-blank-screen-for-a-long-time-without-any-output/44976576#44976576">other answer</a>中的<em>Rutger de Knijf</em>所述)轻松地修改为从多个页面获得所有结果。在</p>
<pre><code>from pprint import pprint
import requests
from BeautifulSoup import BeautifulSoup
def get_url_for_search_key(search_key):
base_url = 'http://www.marketing-interactive.com/'
response = requests.get(base_url + '?s=' + search_key)
soup = BeautifulSoup(response.content)
return [url['href'] for url in soup.findAll('a', {'rel': 'bookmark'})]
</code></pre>
<p>用法:</p>
^{pr2}$
<p>输出:</p>
<pre><code>[u'http://www.marketing-interactive.com/astro-launches-digital-marketing-arm-blaze-digital/',
u'http://www.marketing-interactive.com/singapore-polytechnic-on-the-hunt-for-digital-marketing-agency/',
u'http://www.marketing-interactive.com/how-to-get-your-bosses-on-board-your-digital-marketing-plan/',
u'http://www.marketing-interactive.com/digital-marketing-institute-launches-brand-refresh/',
u'http://www.marketing-interactive.com/entropia-highlights-the-7-original-sins-of-digital-marketing/',
u'http://www.marketing-interactive.com/features/futurist-right-mindset-digital-marketing/',
u'http://www.marketing-interactive.com/lenovo-brings-board-new-digital-marketing-head/',
u'http://www.marketing-interactive.com/video/discussing-digital-marketing-indonesia-video/',
u'http://www.marketing-interactive.com/ubs-melvin-kwek-joins-credit-suisse-as-apac-digital-marketing-lead/',
u'http://www.marketing-interactive.com/linkedins-top-10-digital-marketing-predictions-2017/']
</code></pre>
<p>希望这是你想要的作为你项目的第一步。在</p>