如何获取web图形的javascript标记中的数据，以便生成自己的图形？

2024-06-02 09:01:52 发布

您现在位置：Python中文网/ 问答频道 /正文
626 3

到目前为止，通过使用python库twill和beautifulsoup，我可以登录到我的公用事业公司的网站，并分别获得我的煤气和电力使用数据。实际上，他们用javascript绘制了一个很好的绘图，原始数据可以在其中隐藏。你知道吗

twill库使用showforms()、show()等很好地显示了web表单#1、#2、#3等的组成，但据我所知，不是javascript元素。beautifulsoup似乎能够显示html元素，比如网页的标题，例如：title1 = soup1.html.head.title，但是我似乎无法让它列出javascript元素（页面上有4到5个不同的javascript，其中一个包含下面的数据）。你知道吗

有没有什么方法可以分别使用twill或beautifulsoup中的showjavascript()或soup1.html.body.javascript之类的东西来访问这个javascript数据，并解析它以便导入matplotlib或其他python绘图包？你知道吗

javascript元素中的数据如下所示（您可以看到我、邻居和高效邻居）：

<script type="text/javascript">
(function() {
window.seriesDTO = {
  "series" : [ {
    "id" : "YOU",
    "data" : [ {
      "startDate" : "Fri, 30 Aug 2013 00:00:00",
      "endDate" : "Mon, 23 Sep 2013 23:59:59",
      "value" : 66,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Sun, 01 Sep 2013 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Fri, 30 Aug 2013 00:00:00",
      "upperDate" : "Mon, 23 Sep 2013 23:59:59"
    }, {
      "startDate" : "Tue, 24 Sep 2013 00:00:00",
      "endDate" : "Tue, 22 Oct 2013 23:59:59",
      "value" : 85,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Tue, 01 Oct 2013 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Tue, 24 Sep 2013 00:00:00",
      "upperDate" : "Tue, 22 Oct 2013 23:59:59"
    }, {
      "startDate" : "Wed, 23 Oct 2013 00:00:00",
      "endDate" : "Thu, 21 Nov 2013 23:59:59",
      "value" : 135,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Fri, 01 Nov 2013 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Wed, 23 Oct 2013 00:00:00",
      "upperDate" : "Thu, 21 Nov 2013 23:59:59"
    }, {
      "startDate" : "Fri, 22 Nov 2013 00:00:00",
      "endDate" : "Mon, 23 Dec 2013 23:59:59",
      "value" : 134,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Sun, 01 Dec 2013 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Fri, 22 Nov 2013 00:00:00",
      "upperDate" : "Mon, 23 Dec 2013 23:59:59"
    }, {
      "startDate" : "Tue, 24 Dec 2013 00:00:00",
      "endDate" : "Thu, 23 Jan 2014 23:59:59",
      "value" : 72,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Wed, 01 Jan 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Tue, 24 Dec 2013 00:00:00",
      "upperDate" : "Thu, 23 Jan 2014 23:59:59"
    }, {
      "startDate" : "Fri, 24 Jan 2014 00:00:00",
      "endDate" : "Mon, 24 Feb 2014 23:59:59",
      "value" : 109,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Sat, 01 Feb 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Fri, 24 Jan 2014 00:00:00",
      "upperDate" : "Mon, 24 Feb 2014 23:59:59"
    }, {
      "startDate" : "Tue, 25 Feb 2014 00:00:00",
      "endDate" : "Tue, 25 Mar 2014 23:59:59",
      "value" : 84,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Sat, 01 Mar 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Tue, 25 Feb 2014 00:00:00",
      "upperDate" : "Tue, 25 Mar 2014 23:59:59"
    }, {
      "startDate" : "Wed, 26 Mar 2014 00:00:00",
      "endDate" : "Thu, 24 Apr 2014 23:59:59",
      "value" : 86,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Tue, 01 Apr 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Wed, 26 Mar 2014 00:00:00",
      "upperDate" : "Thu, 24 Apr 2014 23:59:59"
    }, {
      "startDate" : "Fri, 25 Apr 2014 00:00:00",
      "endDate" : "Fri, 23 May 2014 23:59:59",
      "value" : 56,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Thu, 01 May 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Fri, 25 Apr 2014 00:00:00",
      "upperDate" : "Fri, 23 May 2014 23:59:59"
    }, {
      "startDate" : "Sat, 24 May 2014 00:00:00",
      "endDate" : "Tue, 24 Jun 2014 23:59:59",
      "value" : 76,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Sun, 01 Jun 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Sat, 24 May 2014 00:00:00",
      "upperDate" : "Tue, 24 Jun 2014 23:59:59"
    }, {
      "startDate" : "Wed, 25 Jun 2014 00:00:00",
      "endDate" : "Thu, 24 Jul 2014 23:59:59",
      "value" : 72,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Tue, 01 Jul 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Wed, 25 Jun 2014 00:00:00",
      "upperDate" : "Thu, 24 Jul 2014 23:59:59"
    }, {
      "startDate" : "Fri, 25 Jul 2014 00:00:00",
      "endDate" : "Mon, 25 Aug 2014 23:59:59",
      "value" : 81,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Fri, 01 Aug 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Fri, 25 Jul 2014 00:00:00",
      "upperDate" : "Mon, 25 Aug 2014 23:59:59"
    } ],
    "unit" : "KWH",
    "precision" : 0
  }, {
    "id" : "NEIGHBORS",
    "data" : [ {
      "startDate" : "Fri, 30 Aug 2013 00:00:00",
      "endDate" : "Mon, 23 Sep 2013 23:59:59",
      "value" : 257.958526611328125,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Sun, 01 Sep 2013 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Fri, 30 Aug 2013 00:00:00",
      "upperDate" : "Mon, 23 Sep 2013 23:59:59"
    }, {
      "startDate" : "Tue, 24 Sep 2013 00:00:00",
      "endDate" : "Tue, 22 Oct 2013 23:59:59",
      "value" : 316.710968017578125,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Tue, 01 Oct 2013 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Tue, 24 Sep 2013 00:00:00",
      "upperDate" : "Tue, 22 Oct 2013 23:59:59"
    }, {
      "startDate" : "Wed, 23 Oct 2013 00:00:00",
      "endDate" : "Thu, 21 Nov 2013 23:59:59",
      "value" : 373.00909423828125,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Fri, 01 Nov 2013 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Wed, 23 Oct 2013 00:00:00",
      "upperDate" : "Thu, 21 Nov 2013 23:59:59"
    }, {
      "startDate" : "Fri, 22 Nov 2013 00:00:00",
      "endDate" : "Mon, 23 Dec 2013 23:59:59",
      "value" : 442.25927734375,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Sun, 01 Dec 2013 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Fri, 22 Nov 2013 00:00:00",
      "upperDate" : "Mon, 23 Dec 2013 23:59:59"
    }, {
      "startDate" : "Tue, 24 Dec 2013 00:00:00",
      "endDate" : "Thu, 23 Jan 2014 23:59:59",
      "value" : 389.46453857421875,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Wed, 01 Jan 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Tue, 24 Dec 2013 00:00:00",
      "upperDate" : "Thu, 23 Jan 2014 23:59:59"
    }, {
      "startDate" : "Fri, 24 Jan 2014 00:00:00",
      "endDate" : "Mon, 24 Feb 2014 23:59:59",
      "value" : 402.319488525390625,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Sat, 01 Feb 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Fri, 24 Jan 2014 00:00:00",
      "upperDate" : "Mon, 24 Feb 2014 23:59:59"
    }, {
      "startDate" : "Tue, 25 Feb 2014 00:00:00",
      "endDate" : "Tue, 25 Mar 2014 23:59:59",
      "value" : 319.855438232421875,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Sat, 01 Mar 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Tue, 25 Feb 2014 00:00:00",
      "upperDate" : "Tue, 25 Mar 2014 23:59:59"
    }, {
      "startDate" : "Wed, 26 Mar 2014 00:00:00",
      "endDate" : "Thu, 24 Apr 2014 23:59:59",
      "value" : 328.7225341796875,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Tue, 01 Apr 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Wed, 26 Mar 2014 00:00:00",
      "upperDate" : "Thu, 24 Apr 2014 23:59:59"
    }, {
      "startDate" : "Fri, 25 Apr 2014 00:00:00",
      "endDate" : "Fri, 23 May 2014 23:59:59",
      "value" : 301.809600830078125,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Thu, 01 May 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Fri, 25 Apr 2014 00:00:00",
      "upperDate" : "Fri, 23 May 2014 23:59:59"
    }, {
      "startDate" : "Sat, 24 May 2014 00:00:00",
      "endDate" : "Tue, 24 Jun 2014 23:59:59",
      "value" : 324.0477294921875,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Sun, 01 Jun 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Sat, 24 May 2014 00:00:00",
      "upperDate" : "Tue, 24 Jun 2014 23:59:59"
    }, {
      "startDate" : "Wed, 25 Jun 2014 00:00:00",
      "endDate" : "Thu, 24 Jul 2014 23:59:59",
      "value" : 291.14556884765625,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Tue, 01 Jul 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Wed, 25 Jun 2014 00:00:00",
      "upperDate" : "Thu, 24 Jul 2014 23:59:59"
    }, {
      "startDate" : "Fri, 25 Jul 2014 00:00:00",
      "endDate" : "Mon, 25 Aug 2014 23:59:59",
      "value" : 326.228668212890625,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Fri, 01 Aug 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Fri, 25 Jul 2014 00:00:00",
      "upperDate" : "Mon, 25 Aug 2014 23:59:59"
    } ],
    "unit" : "KWH",
    "precision" : 0
  }, {
    "id" : "EFFICIENT_NEIGHBORS",
    "data" : [ {
      "startDate" : "Fri, 30 Aug 2013 00:00:00",
      "endDate" : "Mon, 23 Sep 2013 23:59:59",
      "value" : 87.5,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Sun, 01 Sep 2013 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Fri, 30 Aug 2013 00:00:00",
      "upperDate" : "Mon, 23 Sep 2013 23:59:59"
    }, {
      "startDate" : "Tue, 24 Sep 2013 00:00:00",
      "endDate" : "Tue, 22 Oct 2013 23:59:59",
      "value" : 112.07543182373046875,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Tue, 01 Oct 2013 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Tue, 24 Sep 2013 00:00:00",
      "upperDate" : "Tue, 22 Oct 2013 23:59:59"
    }, {
      "startDate" : "Wed, 23 Oct 2013 00:00:00",
      "endDate" : "Thu, 21 Nov 2013 23:59:59",
      "value" : 123,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Fri, 01 Nov 2013 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Wed, 23 Oct 2013 00:00:00",
      "upperDate" : "Thu, 21 Nov 2013 23:59:59"
    }, {
      "startDate" : "Fri, 22 Nov 2013 00:00:00",
      "endDate" : "Mon, 23 Dec 2013 23:59:59",
      "value" : 143,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Sun, 01 Dec 2013 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Fri, 22 Nov 2013 00:00:00",
      "upperDate" : "Mon, 23 Dec 2013 23:59:59"
    }, {
      "startDate" : "Tue, 24 Dec 2013 00:00:00",
      "endDate" : "Thu, 23 Jan 2014 23:59:59",
      "value" : 116,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Wed, 01 Jan 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Tue, 24 Dec 2013 00:00:00",
      "upperDate" : "Thu, 23 Jan 2014 23:59:59"
    }, {
      "startDate" : "Fri, 24 Jan 2014 00:00:00",
      "endDate" : "Mon, 24 Feb 2014 23:59:59",
      "value" : 131,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Sat, 01 Feb 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Fri, 24 Jan 2014 00:00:00",
      "upperDate" : "Mon, 24 Feb 2014 23:59:59"
    }, {
      "startDate" : "Tue, 25 Feb 2014 00:00:00",
      "endDate" : "Tue, 25 Mar 2014 23:59:59",
      "value" : 109,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Sat, 01 Mar 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Tue, 25 Feb 2014 00:00:00",
      "upperDate" : "Tue, 25 Mar 2014 23:59:59"
    }, {
      "startDate" : "Wed, 26 Mar 2014 00:00:00",
      "endDate" : "Thu, 24 Apr 2014 23:59:59",
      "value" : 105,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Tue, 01 Apr 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Wed, 26 Mar 2014 00:00:00",
      "upperDate" : "Thu, 24 Apr 2014 23:59:59"
    }, {
      "startDate" : "Fri, 25 Apr 2014 00:00:00",
      "endDate" : "Fri, 23 May 2014 23:59:59",
      "value" : 103,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Thu, 01 May 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Fri, 25 Apr 2014 00:00:00",
      "upperDate" : "Fri, 23 May 2014 23:59:59"
    }, {
      "startDate" : "Sat, 24 May 2014 00:00:00",
      "endDate" : "Tue, 24 Jun 2014 23:59:59",
      "value" : 119,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Sun, 01 Jun 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Sat, 24 May 2014 00:00:00",
      "upperDate" : "Tue, 24 Jun 2014 23:59:59"
    }, {
      "startDate" : "Wed, 25 Jun 2014 00:00:00",
      "endDate" : "Thu, 24 Jul 2014 23:59:59",
      "value" : 102,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Tue, 01 Jul 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Wed, 25 Jun 2014 00:00:00",
      "upperDate" : "Thu, 24 Jul 2014 23:59:59"
    }, {
      "startDate" : "Fri, 25 Jul 2014 00:00:00",
      "endDate" : "Mon, 25 Aug 2014 23:59:59",
      "value" : 104,
      "estimated" : false,
      "shouldUseMajorityMonthForBilling" : true,
      "startOfMajorityMonth" : "Fri, 01 Aug 2014 00:00:00",
      "firstMajorityMonth" : null,
      "lastMajorityMonth" : null,
      "lowerDate" : "Fri, 25 Jul 2014 00:00:00",
      "upperDate" : "Mon, 25 Aug 2014 23:59:59"
    } ],
    "unit" : "KWH",
    "precision" : 0
  }, {
    "id" : "BILL_IDENTIFIERS",
    "data" : [ {
      "value" : 1,
      "year" : 2013,
      "startDate" : "Fri, 30 Aug 2013 00:00:00",
      "endDate" : "Mon, 23 Sep 2013 23:59:59",
      "lowerDate" : "Fri, 30 Aug 2013 00:00:00",
      "upperDate" : "Mon, 23 Sep 2013 23:59:59"
    }, {
      "value" : 2,
      "year" : 2013,
      "startDate" : "Tue, 24 Sep 2013 00:00:00",
      "endDate" : "Tue, 22 Oct 2013 23:59:59",
      "lowerDate" : "Tue, 24 Sep 2013 00:00:00",
      "upperDate" : "Tue, 22 Oct 2013 23:59:59"
    }, {
      "value" : 3,
      "year" : 2013,
      "startDate" : "Wed, 23 Oct 2013 00:00:00",
      "endDate" : "Thu, 21 Nov 2013 23:59:59",
      "lowerDate" : "Wed, 23 Oct 2013 00:00:00",
      "upperDate" : "Thu, 21 Nov 2013 23:59:59"
    }, {
      "value" : 4,
      "year" : 2013,
      "startDate" : "Fri, 22 Nov 2013 00:00:00",
      "endDate" : "Mon, 23 Dec 2013 23:59:59",
      "lowerDate" : "Fri, 22 Nov 2013 00:00:00",
      "upperDate" : "Mon, 23 Dec 2013 23:59:59"
    }, {
      "value" : 1,
      "year" : 2014,
      "startDate" : "Tue, 24 Dec 2013 00:00:00",
      "endDate" : "Thu, 23 Jan 2014 23:59:59",
      "lowerDate" : "Tue, 24 Dec 2013 00:00:00",
      "upperDate" : "Thu, 23 Jan 2014 23:59:59"
    }, {
      "value" : 2,
      "year" : 2014,
      "startDate" : "Fri, 24 Jan 2014 00:00:00",
      "endDate" : "Mon, 24 Feb 2014 23:59:59",
      "lowerDate" : "Fri, 24 Jan 2014 00:00:00",
      "upperDate" : "Mon, 24 Feb 2014 23:59:59"
    }, {
      "value" : 3,
      "year" : 2014,
      "startDate" : "Tue, 25 Feb 2014 00:00:00",
      "endDate" : "Tue, 25 Mar 2014 23:59:59",
      "lowerDate" : "Tue, 25 Feb 2014 00:00:00",
      "upperDate" : "Tue, 25 Mar 2014 23:59:59"
    }, {
      "value" : 4,
      "year" : 2014,
      "startDate" : "Wed, 26 Mar 2014 00:00:00",
      "endDate" : "Thu, 24 Apr 2014 23:59:59",
      "lowerDate" : "Wed, 26 Mar 2014 00:00:00",
      "upperDate" : "Thu, 24 Apr 2014 23:59:59"
    }, {
      "value" : 5,
      "year" : 2014,
      "startDate" : "Fri, 25 Apr 2014 00:00:00",
      "endDate" : "Fri, 23 May 2014 23:59:59",
      "lowerDate" : "Fri, 25 Apr 2014 00:00:00",
      "upperDate" : "Fri, 23 May 2014 23:59:59"
    }, {
      "value" : 6,
      "year" : 2014,
      "startDate" : "Sat, 24 May 2014 00:00:00",
      "endDate" : "Tue, 24 Jun 2014 23:59:59",
      "lowerDate" : "Sat, 24 May 2014 00:00:00",
      "upperDate" : "Tue, 24 Jun 2014 23:59:59"
    }, {
      "value" : 7,
      "year" : 2014,
      "startDate" : "Wed, 25 Jun 2014 00:00:00",
      "endDate" : "Thu, 24 Jul 2014 23:59:59",
      "lowerDate" : "Wed, 25 Jun 2014 00:00:00",
      "upperDate" : "Thu, 24 Jul 2014 23:59:59"
    }, {
      "value" : 8,
      "year" : 2014,
      "startDate" : "Fri, 25 Jul 2014 00:00:00",
      "endDate" : "Mon, 25 Aug 2014 23:59:59",
      "lowerDate" : "Fri, 25 Jul 2014 00:00:00",
      "upperDate" : "Mon, 25 Aug 2014 23:59:59"
    } ]
  } ],
  "sufficientData" : true
};
})();
</script>

Tags： falsetruevaluenullestimatedfrituestartdate
1条回答
网友
1楼 · 发布于 2024-06-02 09:01:52

您可以使用^{}包解析javascript代码：

SlimIt is a JavaScript minifier written in Python. It compiles JavaScript into more compact code so that it downloads and runs faster.

SlimIt also provides a library that includes a JavaScript parser, lexer, pretty printer and a tree visitor.

在您的情况下，可以通过查找对象的第一个赋值来应用它：

import json

from bs4 import BeautifulSoup
from slimit import ast
from slimit.parser import Parser
from slimit.visitors import nodevisitor


data = """
<script type="text/javascript">
(function() {
window.seriesDTO = {
...
};
})();
</script>
"""

soup = BeautifulSoup(data)

parser = Parser()
tree = parser.parse(soup.script.text)

json_data = next(node.right.to_ecma() for node in nodevisitor.visit(tree)
                 if isinstance(node, ast.Assign) and isinstance(node.right, ast.Object))

print json.loads(json_data)

打印Python词典：

{u'series': [{u'data': [{u'endDate': u'Mon, 23 Sep 2013 23:59:59',
                         u'estimated': False,
                         u'firstMajorityMonth': None,
                         u'lastMajorityMonth': None,
                         u'lowerDate': u'Fri, 30 Aug 2013 00:00:00',
                         u'shouldUseMajorityMonthForBilling': True,
                         u'startDate': u'Fri, 30 Aug 2013 00:00:00',
                         u'startOfMajorityMonth': u'Sun, 01 Sep 2013 00:00:00',
                         u'upperDate': u'Mon, 23 Sep 2013 23:59:59',
                         u'value': 66},
...
{u'endDate': u'Mon, 25 Aug 2014 23:59:59',
                         u'lowerDate': u'Fri, 25 Jul 2014 00:00:00',
                         u'startDate': u'Fri, 25 Jul 2014 00:00:00',
                         u'upperDate': u'Mon, 25 Aug 2014 23:59:59',
                         u'value': 8,
                         u'year': 2014}],
              u'id': u'BILL_IDENTIFIERS'}],
 u'sufficientData': True}

相关问题 更多 >

    编程相关推荐