我来自beautifulsoup的爬虫根本不写网页的内容

Question

我写了一个网络爬虫代码来收集来自Waze live map的数据。它曾经工作过，并成功地将响应的内容写到一个文本文件中。但是，现在它将保存在文本文件中的唯一输出是“'b'”值。它似乎无法访问该网页的内容。我之前添加了标题，因为我使用的链接是页面的RSS提要。这是我的代码：

# Extract the object from Waze live map
from bs4 import BeautifulSoup
import urllib.request
import requests


class AppURLOpener(urllib.request.FancyURLopener):
  version = "Mozilla/5.0"

def event (left, right, bottom, up):

link = "https://www.waze.com/rtserver/web/TGeoRSS?ma=600&mj=100&mu=100&left="+ left + "&right=" + right + "&bottom=" + bottom + "&top=" + \
       up + "&_=1504116242156"
print (link)
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)     Chrome/37.0.2049.0 Safari/537.36'}
r = requests.get(link)
rs = r.content
print (rs)
soup = BeautifulSoup(rs, 'html.parser')
soup2 = soup.encode('UTF-8')
content = str(soup2)
with open("I-5 LA.txt", "a") as myfile:
    myfile.write(content)

我没有收到任何错误，问题是JSON响应的内容没有写入文本文件再也不会了。可以有人帮我吗？在

Answer 1

我相信原因是这个来自waze的网址不再公开了。见下文。我也在尝试访问，但将讨论，看看是否可以通过合作伙伴计划获得提要访问。在

https://support.google.com/waze/partners/answer/7239095?hl=enhttps://www.waze.com/forum/viewtopic.php?f=12&t=249944

Postby bedo2991 » Sun Feb 11, 2018 7:45 pm

Waze disabled all requests to GeoRSS feed, with the exception of interface tokens. Get in contact with your Global Champs in case you need access to it for something useful to the whole community.