基于python3和lxml的新型抓取爬虫模块
AISTLAB_novel_grab的Python项目详细描述
novel grab crawler module using python3 and lxml
multiprocesssing with multithread version
winxos, AISTLAB Since 2017-02-19
安装:
pip3 install aistlab_novel_grab
一。用法:
在控制台中运行命令:
novel_grab http://the_url_of_novel_chapters_page
示例:
novel_grab http://book.zongheng.com/showchapter/654086.html
SUPPORTED SITES: * http://book.zongheng.com * http://www.aoyuge.com * http://www.quanshu.net
2.用作python模块:
fromnovel_grab.novel_grabimportDownloaderd=Downloader()print(d.get_info())ifd.set_url('http://book.zongheng.com/showchapter/221579.html'):d.start()**TIPS** \*Whend=Downloader(),d.get\_info()cangetsupportedsitesinfo. \*Onced.set\_url(url)willreturntheurlisvalidornot. \*Ofcourseyoucanused.get\_info()toaccessthestateofdatanytime. \*Whilefinished,willcreate:math:`novel_name`.zipfileinyourcurrentpath,defaultzipmethodusingzipfile.ZIP\_DEFLATED
出于教育目的,请照顾好自己。