如何从类似Python语法的JavaScript函数调用中提取数据

Question

我正在用Python中的scrapy从一个网站上抓取数据。你知道吗

所需数据位于脚本标记中，如下所示：

<script type="text/javascript">
getDetailsfrmBean("storePg","564","Berwyn, IL","7180 W CERMAK RD.","SPACE A1","","BERWYN","IL","US","60402","(708) 788-5097","{Monday-Saturday=10-9,sunday=11-6}","41.8507029","-87.8033709");
</script>

我可以使用xpath获得以下内容：

item['lat'] = tree.xpath('//script[@type="text/javascript"]/text()'.extract()[0].encode('utf-8')
item['long'] = tree.xpath('//script[@type="text/javascript"]/text()'.extract()[0].encode('utf-8')

那么

item['lat'] = 'getDetailsfrmBean("storePg","564","Berwyn, IL","7180 W CERMAK RD.","SPACE A1","","BERWYN","IL","US","60402","(708) 788-5097","{Monday-Saturday=10-9,sunday=11-6}","41.8507029","-87.8033709");'

item['long'] = 'getDetailsfrmBean("storePg","564","Berwyn, IL","7180 W CERMAK RD.","SPACE A1","","BERWYN","IL","US","60402","(708) 788-5097","{Monday-Saturday=10-9,sunday=11-6}","41.8507029","-87.8033709");'

但是我怎样才能解析这些内容呢

item['lat'] is equal to "41.8507029"
item['long'] is equal to "-87.8033709"
item['city'] is equal to "BERWYN"
item['state'] is equal to "IL"

我能得到解决这个问题的建议吗。你知道吗

Answer 1

用re试试这个

import re
temp_string = 'getDetailsfrmBean("storePg","564","Berwyn, IL","7180 W CERMAK RD.","SPACE A1","","BERWYN","IL","US","60402","(708) 788-5097","{Monday-Saturday=10-9,sunday=11-6}","41.8507029","-87.8033709");'
split_list = filter(None, re.split("[, \-!?:\"]+",temp_string))
print split_list

应产生以下输出：

['getDetailsfrmBean(', 'storePg', '564', 'Berwyn', 'IL', '7180', 'W', 'CERMAK', 'RD.', 'SPACE', 'A1', 'BERWYN', 'IL', 'US', '60402', '(708)', '788', '5097', '{Monday', 'Saturday=10', '9', 'sunday=11', '6}', '41.8507029', '87.8033709', ');']

从这里的答案中得到：https://stackoverflow.com/a/23720594/5907969

Answer 2

因为这个调用也是有效的Python语法，所以我们可以使用ast模块。加上参数都是字符串文字，这使事情更简单。你知道吗

import ast

line = 'getDetailsfrmBean("storePg","564","Berwyn, IL","7180 W CERMAK RD.","SPACE A1","","BERWYN","IL","US","60402","(708) 788-5097","{Monday-Saturday=10-9,sunday=11-6}","41.8507029","-87.8033709");'

print([arg.s for arg in ast.parse(line).body[0].value.args])

输出：

['storePg', '564', 'Berwyn, IL', '7180 W CERMAK RD.', 'SPACE A1', '', 'BERWYN', 'IL', 'US', '60402', '(708) 788-5097', '{Monday-Saturday=10-9,sunday=11-6}', '41.8507029', '-87.8033709']

说明：

print([arg.s           # value of string literal
       for arg in
       ast.parse(line)
      .body            # module (list of statements)
       [0]             # first statement (an Expr node)
      .value           # expression (a Call)
      .args            # arguments to function call
       ])

Answer 3

您可以使用简单的正则表达式来提取逗号分隔的引号字符串部分：

import re

line = 'getDetailsfrmBean("storePg","564","Berwyn, IL","7180 W CERMAK RD.","SPACE A1","","BERWYN","IL","US","60402","(708) 788-5097","{Monday-Saturday=10-9,sunday=11-6}","41.8507029","-87.8033709");'

args_string = re.match(r'getDetailsfrmBean\((.+)\);$', line.strip()).group(1)
print(args_string)

输出：

"storePg","564","Berwyn, IL","7180 W CERMAK RD.","SPACE A1","","BERWYN","IL","US","60402","(708) 788-5097","{Monday-Saturday=10-9,sunday=11-6}","41.8507029","-87.8033709"

然后有多种方法可以从此类数据中解析字符串列表：

import ast
import json
import csv

args_array = '[%s]' % args_string

assert (json.loads(args_array)
        == ast.literal_eval(args_array)
        == next(csv.reader([args_string]))
        == ['storePg', '564', 'Berwyn, IL', '7180 W CERMAK RD.', 'SPACE A1', '', 'BERWYN', 'IL', 'US', '60402',
            '(708) 788-5097', '{Monday-Saturday=10-9,sunday=11-6}', '41.8507029', '-87.8033709'])