每次我运行这个代码的时候，蜘蛛核都会死掉

Question

我正在用BeautifulSoup从NBA网站上搜集数据。想要创建一个包含姓名，球员简历链接，身高，体重，出生日期的列表。 名字和球员的生物链接正在被成功地刮，但其他人没有。 链接：https://in.global.nba.com/playerindex/。 而且，每次我试图访问variables explorer中的变量时，我都注意到spyder内核正在消亡。你知道吗

names = []
tr = soup.find_all("tr",class_="ng-scope")
for i in tr:
    td = i.find("td",class_="left player")
    anchor = td.find("a",class_="player-name ng-isolate-scope")
    href = td.find("a")["data-ng-href"]
    span = anchor.find("span",class_="ng-binding")
    spans = anchor.find("span",class_="ng- 
    binding").findNextSibling().findNextSibling()
    name = span.text + " " + spans.text
    linktoplayer = 'https://in.global.nba.com'+href
    driver.get(linktoplayer)
    html_docs = driver.page_source
    soups = BeautifulSoup(html_docs,'lxml')
    div = soups.find("div",class_="player-info-right hidden-sm")
    p = div.find("p",class_="ng-binding")
    upperspan = p.find("span",class_="ng-binding")
    innerspan = upperspan.find("span",class_="ng-binding")
    height = innerspan.text
    print(height)
    weight = innerspan.next_sibling.next_sibling.next_sibling
    dob = upperspan.next_sibling.next_sibling.next_sibling
    dob = dob.split(" ")[1]
    bio ={
            "name":name,
            "href":href,
            "height":height,
            "weight":weight,
            "dob":dob
        }
    names.append(bio)

Answer 1

请参见浏览器网络选项卡中的网站请求API获取JSON数据。你知道吗

例如

import requests

jsonData = requests.get("https://in.global.nba.com/stats2/league/playerlist.json?locale=en").json()

for x in jsonData['payload']['players']:
    #print player profile data
    print(x['playerProfile'])
    #print team profile data
    print(x['teamProfile'])

O/p:

玩家资料数据

 {'code': 'ivica_zubac', 'country': 'Croatia', 'countryEn': 'Croatia', 'displayAffiliation': 'Croatia', 'displayName': 'Ivica Zubac', 'displayNameEn': 'Ivica Zubac', 'dob': '858661200000', 'draftYear': '2016', 'experience': '3', 'firstInitial': 'I', 'firstName': 'Ivica', 'firstNameEn': 'Ivica', 'height': '7-1', 'jerseyNo': '40', 'lastName': 'Zubac', 'lastNameEn': 'Zubac', 'leagueId': '00', 'playerId': '1627826', 'position': 'C', 'schoolType': '', 'weight': '240 lbs'}
 ...

团队简介数据

{'abbr': 'LAC', 'city': 'LA', 'cityEn': 'LA', 'code': 'clippers', 'conference': 'Western', 'displayAbbr': 'LAC', 'displayConference': 'Western', 'division': 'Pacific', 'id': '1610612746', 'isAllStarTeam': False, 'isLeagueTeam': True, 'leagueId': '00', 'name': 'Clippers', 'nameEn': 'Clippers'}
 ....

将毫秒转换为日期：

例如

import datetime
ms = '858661200000'
dob = datetime.datetime.fromtimestamp(int(ms)/1000.0).date()
print(dob)

O/p:

1997-03-18