python http“requests”的扩展，使用“logging enforded”进行详细日志记录。
requests-fortified的Python项目详细描述
强化请求
正在工作
用verbose扩展python httprequests 使用logging-fortified进行日志记录，并对 下载包含json、csv和xml数据格式的文件以及流媒体。
重要提示
这个python项目是requests-mv-integration的重构 出于一般使用目的。
徽章
| info | |
|---|---|
| package |
安装
pip install requests_fortified
架构
requests-fortified是Python package requests的扩展 它用于处理所有的http请求，包括rest和soap中的api， 屏幕报废，并处理json、xml和csv格式的响应下载。
用法
URL_REST_COUNTRIES= \ 'https://restcountries.eu/rest/v2/all'fromrequests_fortifiedimport(RequestsFortifiedDownload,)request_download=RequestsFortifiedDownload(logger_level=logging.DEBUG)result= \ request_download.request(request_method='GET',request_url=URL_REST_COUNTRIES,request_params=None,request_retry=None,request_headers=HEADER_CONTENT_TYPE_APP_JSON,request_label="REST Countries")json_rest_countries=result.json()pprint(json_rest_countries)
示例
$ python3 examples/example_request.py ====================================================== run-examples requests-fortified ====================================================== rm -fR _tmp/*.json ======================================================'Logger file path: ./tmp/log_1526043820.json'['{"asctime": "2018-05-11 06:03:26 -0700", "levelname": "NOTE", "name": "requests_fortified", "version": "0.1.0", "message": "debug"}\n', '{"asctime": "2018-05-11 06:03:26 -0700", "levelname": "INFO", "name": "requests_fortified", "version": "0.1.0", "message": "START"}\n', '{"asctime": "2018-05-11 06:03:26 -0700", "levelname": "DEBUG", "name": "requests_fortified", "version": "0.1.0", "message": "REST Countries: Start"}\n', '{"asctime": "2018-05-11 06:03:26 -0700", "levelname": "DEBUG", "name": "requests_fortified", "version": "0.1.0", "message": "REST Countries: Start: Details", "Disk:": {"path": "/", "total": "232.62 GB", "used": "134.44 GB", "free": "97.94 GB", "percent": 57}, "Mem": {"total": "16.00 GB", "used": "12.50 GB", "free": "147.59 KB", "shared": "0 B", "buffers": "0 B", "cached": "0 B"}, "allow_redirects": true, "build_request_curl": true, "cookie_payload": null, "request_auth": null, "request_cert": null, "request_data": null, "request_headers": {"Content-Type": "application/json", "User-Agent": "(requests-fortified/0.1.0, Python/3.6.5)"}, "request_json": null, "request_label": "REST Countries", "request_method": "GET", "request_params": null, "request_retry": {"timeout": 60, "tries": 3, "delay": 10}, "request_url": "https://restcountries.eu/rest/v2/all", "stream": false, "timeout": null, "verify": true}\n', '{"asctime": "2018-05-11 06:03:26 -0700", "levelname": "DEBUG", "name": "requests_fortified", "version": "0.1.0", "message": "REST Countries: Start", "request_label": "REST Countries", "request_retry_excps": ["ConnectTimeout", "ReadTimeout", "Timeout"], "request_retry_http_status_codes": [500, 502, 503, 504, 429], "timeout": null}\n', '{"asctime": "2018-05-11 06:03:26 -0700", "levelname": "DEBUG", "name": "requests_fortified", "version": "0.1.0", "message": "REST Countries: Attempt", "attempts": 1, "delay": 10, "request_label": "REST Countries", "request_url": "https://restcountries.eu/rest/v2/all", "timeout": 60, "tries": 3}\n', '{"asctime": "2018-05-11 06:03:26 -0700", "levelname": "DEBUG", "name": "requests_fortified", "version": "0.1.0", "message": "REST Countries: Session: Details", "cookie_payload": {}, "request_label": "REST Countries"}\n', '{"asctime": "2018-05-11 06:03:26 -0700", "levelname": "DEBUG", "name": "requests_fortified", "version": "0.1.0", "message": "REST Countries: Details", "request_data": "", "request_headers": {"Content-Type": "application/json", "User-Agent": "(requests-fortified/0.1.0, Python/3.6.5)"}, "request_label": "REST Countries", "request_method": "GET", "request_params": {}, "request_url": "https://restcountries.eu/rest/v2/all", "timeout": 60}\n', '{"asctime": "2018-05-11 06:03:26 -0700", "levelname": "NOTE", "name": "requests_fortified", "version": "0.1.0", "message": "REST Countries: Curl", "request_curl": "curl --verbose -X GET -H \'Content-Type: application/json\' -H \'User-Agent: (requests-fortified/0.1.0, Python/3.6.5)\' --connect-timeout 60 -L \'https://restcountries.eu/rest/v2/all\'", "request_label": "REST Countries", "request_method": "GET"}\n', '{"asctime": "2018-05-11 06:03:26 -0700", "levelname": "DEBUG", "name": "requests_fortified.support.requests_session_client", "version": "0.1.0", "message": "Session Request: Details", "allow_redirects": true, "headers": {"Content-Type": "application/json", "User-Agent": "(requests-fortified/0.1.0, Python/3.6.5)"}, "method": "GET", "timeout": 60, "url": "https://restcountries.eu/rest/v2/all", "verify": true}\n', '{"asctime": "2018-05-11 06:03:27 -0700", "levelname": "DEBUG", "name": "requests_fortified", "version": "0.1.0", "message": "REST Countries: Response: Details", "http_status_code": 200, "http_status_desc": "Request fulfilled, document follows", "http_status_type": "Successful", "response_headers": {"Date": "Fri, 11 May 201813:03:27 GMT", "Content-Type": "application/json;charset=utf-8", "Transfer-Encoding": "chunked", "Connection": "keep-alive", "Set-Cookie": "__cfduid=d46030914d4b01044c167b67bbca43b951526043807;expires=Sat, 11-May-19 13:03:27 GMT;path=/;domain=.restcountries.eu; HttpOnly", "Access-Control-Allow-Origin": "*", "Access-Control-Allow-Methods": "GET", "Access-Control-Allow-Headers": "Accept, X-Requested-With", "Cache-Control": "public, max-age=86400", "Expect-CT": "max-age=604800, report-uri=\\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\\"", "Server": "cloudflare", "CF-RAY": "4194d881fc146bd4-SJC", "Content-Encoding": "gzip"}}\n', '{"asctime": "2018-05-11 06:03:27 -0700", "levelname": "DEBUG", "name": "requests_fortified", "version": "0.1.0", "message": "REST Countries: Cookie Payload", "cookie_payload": {"__cfduid": "d46030914d4b01044c167b67bbca43b951526043807"}, "request_label": "REST Countries"}\n', '{"asctime": "2018-05-11 06:03:27 -0700", "levelname": "DEBUG", "name": "requests_fortified", "version": "0.1.0", "message": "REST Countries: Try Send Request: Is Return Response: Checking", "request_url": "https://restcountries.eu/rest/v2/all"}\n', '{"asctime": "2018-05-11 06:03:27 -0700", "levelname": "DEBUG", "name": "requests_fortified", "version": "0.1.0", "message": "REST Countries: Try Send Request: Is Return Response: Valid", "request_url": "https://restcountries.eu/rest/v2/all"}\n', '{"asctime": "2018-05-11 06:03:27 -0700", "levelname": "INFO", "name": "requests_fortified", "version": "0.1.0", "message": "REST Countries: Finished", "request_time_msecs": 569}\n', '{"asctime": "2018-05-11 06:03:27 -0700", "levelname": "DEBUG", "name": "requests_fortified", "version": "0.1.0", "message": "REST Countries: Usage", "Disk:": {"path": "/", "total": "232.62 GB", "used": "134.44 GB", "free": "97.94 GB", "percent": 57}, "Mem": {"total": "16.00 GB", "used": "12.50 GB", "free": "145.89 KB", "shared": "0 B", "buffers": "0 B", "cached": "0 B"}}\n', '{"asctime": "2018-05-11 06:03:27 -0700", "levelname": "INFO", "name": "requests_fortified", "version": "0.1.0", "message": "COMPLETED", "connection": "<requests.adapters.HTTPAdapter object at 0x1043b11d0>", "cookies": "<RequestsCookieJar[<Cookie __cfduid=d46030914d4b01044c167b67bbca43b951526043807 for .restcountries.eu/>]>", "elapsed": "0:00:00.286022", "encoding": "utf-8", "headers": "{\'Date\': \'Fri, 11 May 201813:03:27 GMT\', \'Content-Type\': \'application/json;charset=utf-8\', \'Transfer-Encoding\': \'chunked\', \'Connection\': \'keep-alive\', \'Set-Cookie\': \'__cfduid=d46030914d4b01044c167b67bbca43b951526043807;expires=Sat, 11-May-19 13:03:27 GMT;path=/;domain=.restcountries.eu; HttpOnly\', \'Access-Control-Allow-Origin\': \'*\', \'Access-Control-Allow-Methods\': \'GET\', \'Access-Control-Allow-Headers\': \'Accept, X-Requested-With\', \'Cache-Control\': \'public, max-age=86400\', \'Expect-CT\': \'max-age=604800, report-uri=\\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\\"\', \'Server\': \'cloudflare\', \'CF-RAY\': \'4194d881fc146bd4-SJC\', \'Content-Encoding\': \'gzip\'}", "history": [], "raw": "<urllib3.response.HTTPResponse object at 0x1043b1dd8>", "reason": "", "request": "<PreparedRequest [GET]>", "status_code": 200, "url": "https://restcountries.eu/rest/v2/all"}\n', ] 'DEBUG' [ { 'alpha2Code': 'AF', 'alpha3Code': 'AFG', 'altSpellings': ['AF', 'Afġānistān'], 'area': 652230.0, 'borders': ['IRN', 'PAK', 'TKM', 'UZB', 'TJK', 'CHN'], 'callingCodes': ['93'], 'capital': 'Kabul', 'cioc': 'AFG', 'currencies': [ {'code': 'AFN', 'name': 'Afghan afghani', 'symbol': '؋'}, ], 'demonym': 'Afghan', 'flag': 'https://restcountries.eu/data/afg.svg', 'gini': 27.8, 'languages': [ { 'iso639_1': 'ps', 'iso639_2': 'pus', 'name': 'Pashto', 'nativeName': 'پښتو', }, { 'iso639_1': 'uz', 'iso639_2': 'uzb', 'name': 'Uzbek', 'nativeName': 'O\u02bbzbek', }, { 'iso639_1': 'tk', 'iso639_2': 'tuk', 'name': 'Turkmen', 'nativeName': 'Türkmen', }, ], 'latlng': [33.0, 65.0], 'name': 'Afghanistan', 'nativeName': 'افغانستان', 'numericCode': '004', 'population': 27657145, 'region': 'Asia', 'regionalBlocs': [ { 'acronym': 'SAARC', 'name': 'South Asian Association for Regional Cooperation', 'otherAcronyms': [], 'otherNames': [], }, ], 'subregion': 'Southern Asia', 'timezones': ['UTC+04:30'], 'topLevelDomain': ['.af'], 'translations': { 'br': 'Afeganistão', 'de': 'Afghanistan', 'es': 'Afganistán', 'fa': 'افغانستان', 'fr': 'Afghanistan', 'hr': 'Afganistan', 'it': 'Afghanistan', 'ja': 'アフガニスタン', 'nl': 'Afghanistan', 'pt': 'Afeganistão', }, },
课程
- class RequestsFortified–基类使用带有重试功能和详细日志记录的requests。
- class RequestsFortifiedDownload–下载文件处理。
- class RequestsFortifiedUpload–上传文件处理。
要求
requests-fortified模块基于python 3构建，并且依赖于 Python Package Index PyPI中提供了几个python模块。
make install
或
python3 -m pip uninstall --yes --no-input -r requirements.txt python3 -m pip install --upgrade -r requirements.txt
套餐
- beautifulsoup4：https://pypi.python.org/pypi/beautifulsoup4
- deepdiff：https://pypi.python.org/pypi/deepdiff
- 日志强化：*tbd*
- pyhttpstatus实用程序：https://pypi.python.org/pypi/pyhttpstatus-utils
- 请求：https://pypi.python.org/pypi/requests
- safe cast：https://pypi.python.org/pypi/safe-cast
欢迎加入QQ群-->： 979659372
