使用asyncio的dht爬虫框架。
Maga的Python项目详细描述
使用asyncio的dht爬虫框架。
用法
frommagaimportMagaimportlogginglogging.basicConfig(level=logging.INFO)classCrawler(Maga):asyncdefhandler(self,infohash,addr):logging.info(addr)logging.info(infohash)# Or, if you want to have more controlclassCrawler(Maga):asyncdefhandle_get_peers(self,infohash,addr):logging.info("Receive get peers message from DHT {}. Infohash: {}.".format(addr,infohash))asyncdefhandle_announce_peer(self,infohash,addr,peer_addr):logging.info("Receive announce peer message from DHT {}. Infohash: {}. Peer address:{}".format(addr,infohash,peer_addr))crawler=Crawler()# Set port to 0 will use a random available portcrawler.run(port=0)
更改日志
版本3.0.0
- 将peer_addrparam添加到handle_announce_peer方法。
- 不要在处理程序上引发notimplementederror
版本2.0.1
- 不执行信号时不要失败。(#3)
版本2.0.0
- 添加handle_get_peer,handle_announce_peer方法。
- 将addrparam添加到处理程序