invokust 0.52

调用

从python中运行Locust负载测试的工具，无需使用蝗虫命令行。这为自动化提供了更大的灵活性，例如QA/CI/CD测试，也使得可以在蝗虫上运行蝗虫以获得最终的可伸缩性。

安装

通过PIP安装：

pip3 install invokust

示例

使用蝗虫文件运行负载测试：

importinvokustsettings=invokust.create_settings(locustfile='locustfile_example.py',host='http://example.com',num_clients=1,hatch_rate=1,run_time='3m')loadtest=invokust.LocustLoadTest(settings)loadtest.run()loadtest.stats()'{"num_requests_fail": 0, "num_requests": 10, "success": {"/": {"median_response_time": 210, "total_rpm": 48.13724156297892, "request_type": "GET", "min_response_time": 210, "response_times": {"720": 1, "210": 7, "820": 1, "350": 1}, "num_requests": 10, "response_time_percentiles": {"65": 210, "95": 820, "75": 350, "85": 720, "55": 210}, "total_rps": 0.802287359382982, "max_response_time": 824, "avg_response_time": 337.2}}, "locust_host": "http://example.com", "fail": {}, "num_requests_success": 10}'

在没有蝗虫文件的情况下运行负载测试：

importinvokustfromlocustimportHttpLocust,TaskSet,taskclassTask(TaskSet):@task()defget_home_page(self):'''        Gets /        '''self.client.get("/")classWebsiteUser(HttpLocust):task_set=Tasksettings=invokust.create_settings(classes=[WebsiteUser],host='http://example.com',num_clients=1,hatch_rate=1,run_time='3m')loadtest=invokust.LocustLoadTest(settings)loadtest.run()loadtest.stats()'{"num_requests_fail": 0, "num_requests": 10, "success": {"/": {"median_response_time": 330, "total_rpm": 40.53552561950598, "request_type": "GET", "min_response_time": 208, "response_times": {"230": 1, "1000": 1, "780": 1, "330": 1, "1100": 1, "210": 3, "790": 1, "670": 1}, "num_requests": 10, "response_time_percentiles": {"65": 780, "95": 1100, "75": 790, "85": 1000, "55": 670}, "total_rps": 0.6755920936584331, "max_response_time": 1111, "avg_response_time": 552.8}}, "locust_host": "http://example.com", "fail": {}, "num_requests_success": 10}'

在aws lambda上运行蝗虫

AWS Lambda是一个很好的负载测试工具，因为它非常便宜（或免费）并且高度可伸缩。

有许多负载测试工具，如ab和wrk。还有其他基于云的负载测试选项，如BlazeMeter或Loader和一些使用aws lambda的diy解决方案，如Goad或serverless-artillery。但这些都有同样的缺点：过于简单化。它们可以执行简单的get或post请求，但无法准确模拟更复杂的行为。例如，浏览网站、随机选择商品、填写购物车和结帐。但有了Locust这是可能的。

其中包括一个在aws lambda上运行蝗虫的示例函数，lambda_locust.py。

创建lambda函数

在lambda上运行蝗虫测试的过程涉及蝗虫负载测试的creating a zip file，创建lambda函数，然后触发该函数。

在本地安装invokust（及其依赖项）python包：

pip3 install invokust --target=python-packages

或者如果在Mac上运行（需要为64位Linux编译Python包），则可以使用Docker:

docker run -it --volume=$PWD/python-packages:/python-packages python:3.6 bash -c "pip install invokust --target=/python-packages"

创建zip文件：

zip -q -r lambda_locust.zip lambda_locust.py locustfile_example.py python-packages

然后使用aws cli创建lambda函数：

aws lambda create-function --function-name lambda_locust --timeout 300 --runtime python3.6 --role arn:aws:iam::9999999999:role/lambda_basic_execution --handler lambda_locust.handler --zip-file fileb://lambda_locust.zip

或者Terraform和示例main.tf文件：

terraform apply
...

调用函数

蝗虫设置可以传递给lambda函数，也可以从环境变量设置。环境变量为：

• 蝗虫文件：用于负载测试的蝗虫文件
• 蝗虫类：用于负载测试（而不是蝗虫文件）的蝗虫类的名称。如果不止一个，用逗号分隔。
• 蝗虫宿主：对其运行负载测试的宿主
• 蝗虫数量客户端：要模拟的客户端数量
• 蝗虫孵化率：每秒要启动的客户端数
• 蝗虫运行时间：测试应该运行的时间。

AWS CLI有效载荷中蝗虫设置的示例：

aws lambda invoke --function-name lambda_locust --invocation-type RequestResponse --payload '{"locustfile": "locustfile_example.py", "host":"https://example.com", "num_requests":"20", "num_clients": "1", "hatch_rate": "1", "run_time":"3m"}' output.txt
{
    "StatusCode": 200
}
cat output.txt
"{\"success\": {\"GET_/\": {\"request_type\": \"GET\", \"num_requests\": 20, \"min_response_time\": 86, \"median_response_time\": 93 ...

python boto3示例：

importjsonfromboto3.sessionimportSessionfrombotocore.clientimportConfigsession=Session()config=Config(connect_timeout=10,read_timeout=310)client=session.client('lambda',config=config)lambda_payload={'locustfile':'locustfile_example.py','host':'https://example.com','num_clients':'1','hatch_rate':1,'run_time':'3m'}response=client.invoke(FunctionName='lambda_locust',Payload=json.dumps(lambda_payload))json.loads(response['Payload'].read())'{"success": {"GET_/": {"request_type": "GET", "num_requests": 20, "min_response_time": 87, "median_response_time": 99, "avg_response_time": 97.35 ...

运行实际负载测试

lambda函数执行时间限制在5分钟以内。要运行实际负载测试，需要重复调用该函数，并可能并行调用以生成足够的负载。为了管理这一点，有一个名为LambdaLoadTest的类可以管理在并行循环中调用函数并收集统计信息。

importloggingfrominvokust.aws_lambdaimportLambdaLoadTestlogging.basicConfig(level=logging.INFO)lambda_payload={'locustfile':'locustfile_example.py','host':'https://example.com','num_clients':'1','hatch_rate':1,'run_time':'3m'}load_test=LambdaLoadTest(lambda_function_name='lambda_locust',threads=2,ramp_time=0,time_limit=30,lambda_payload=lambda_payload)load_test.run()INFO:root:StartingloadtestFunction:lambda_locustRamptime:0Threads:2Lambdapayload:{'locustfile':'locustfile_example.py','host':'https://example.com','run_time':'30s','num_clients':'1','hatch_rate':1,'run_time':'3m'}INFO:root:threadstartedINFO:root:threads:1,rpm:0,run_time:0,requests_total:0,request_fail_ratio:0,invocation_error_ratio:0INFO:root:threads:1,rpm:0,run_time:3,requests_total:0,request_fail_ratio:0,invocation_error_ratio:0INFO:root:threadstartedINFO:invokust.aws_lambda.lambda_load_test:Invocationcomplete.Requests(errors):20(0),executiontime:2982,sleeping:0INFO:root:threads:2,rpm:368,run_time:6,requests_total:20,request_fail_ratio:0.0,invocation_error_ratio:0.0INFO:invokust.aws_lambda.lambda_load_test:Invocationcomplete.Requests(errors):20(0),executiontime:2896,sleeping:0INFO:invokust.aws_lambda.lambda_load_test:Invocationcomplete.Requests(errors):20(0),executiontime:3023,sleeping:0INFO:root:threads:2,rpm:770,run_time:9,requests_total:60,request_fail_ratio:0.0,invocation_error_ratio:0.0INFO:invokust.aws_lambda.lambda_load_test:Invocationcomplete.Requests(errors):20(0),executiontime:3041,sleeping:0INFO:invokust.aws_lambda.lambda_load_test:Invocationcomplete.Requests(errors):20(0),executiontime:3071,sleeping:0INFO:root:threads:2,rpm:768,run_time:12,requests_total:100,request_fail_ratio:0.0,invocation_error_ratio:0.0INFO:invokust.aws_lambda.lambda_load_test:Invocationcomplete.Requests(errors):20(0),executiontime:3031,sleeping:0INFO:invokust.aws_lambda.lambda_load_test:Invocationcomplete.Requests(errors):20(0),executiontime:2965,sleeping:0INFO:root:threads:2,rpm:782,run_time:15,requests_total:140,request_fail_ratio:0.0,invocation_error_ratio:0.0INFO:invokust.aws_lambda.lambda_load_test:Invocationcomplete.Requests(errors):20(0),executiontime:2957,sleeping:0INFO:invokust.aws_lambda.lambda_load_test:Invocationcomplete.Requests(errors):20(0),executiontime:3069,sleeping:0INFO:root:threads:2,rpm:779,run_time:18,requests_total:180,request_fail_ratio:0.0,invocation_error_ratio:0.0INFO:invokust.aws_lambda.lambda_load_test:Invocationcomplete.Requests(errors):20(0),executiontime:3015,sleeping:0INFO:invokust.aws_lambda.lambda_load_test:Invocationcomplete.Requests(errors):20(0),executiontime:3018,sleeping:0INFO:root:threads:2,rpm:771,run_time:21,requests_total:220,request_fail_ratio:0.0,invocation_error_ratio:0.0INFO:invokust.aws_lambda.lambda_load_test:Invocationcomplete.Requests(errors):20(0),executiontime:3055,sleeping:0INFO:invokust.aws_lambda.lambda_load_test:Invocationcomplete.Requests(errors):20(0),executiontime:3015,sleeping:0INFO:root:threads:2,rpm:771,run_time:24,requests_total:260,request_fail_ratio:0.0,invocation_error_ratio:0.0INFO:invokust.aws_lambda.lambda_load_test:Invocationcomplete.Requests(errors):20(0),executiontime:3007,sleeping:0INFO:invokust.aws_lambda.lambda_load_test:Invocationcomplete.Requests(errors):20(0),executiontime:3012,sleeping:0INFO:root:threads:2,rpm:779,run_time:27,requests_total:300,request_fail_ratio:0.0,invocation_error_ratio:0.0INFO:invokust.aws_lambda.lambda_load_test:Invocationcomplete.Requests(errors):20(0),executiontime:3029,sleeping:0INFO:invokust.aws_lambda.lambda_load_test:Invocationcomplete.Requests(errors):20(0),executiontime:3037,sleeping:0INFO:root:threads:2,rpm:773,run_time:30,requests_total:340,request_fail_ratio:0.0,invocation_error_ratio:0.0INFO:invokust.aws_lambda.lambda_load_test:Invocationcomplete.Requests(errors):20(0),executiontime:3078,sleeping:0INFO:invokust.aws_lambda.lambda_load_test:Invocationcomplete.Requests(errors):20(0),executiontime:3100,sleeping:0INFO:root:threads:2,rpm:758,run_time:33,requests_total:380,request_fail_ratio:0.0,invocation_error_ratio:0.0INFO:root:TimelimitreachedINFO:root:Waitingforthreadstoexit...INFO:invokust.aws_lambda.lambda_load_test:Invocationcomplete.Requests(errors):20(0),executiontime:2965,sleeping:0INFO:root:threadfinishedINFO:invokust.aws_lambda.lambda_load_test:Invocationcomplete.Requests(errors):20(0),executiontime:3033,sleeping:0INFO:root:threadfinishedload_test.get_summary_stats(){'lambda_invocation_count':21,'total_lambda_execution_time':63399,'requests_total':420,'request_fail_ratio':0.0,'invocation_error_ratio':0.0}

还有一个用于运行负载测试的示例cli工具，invokr.py：

$ ./invokr.py --function_name=lambda_locust --locust_file=locustfile_example.py --locust_host=https://example.com --threads=1 --time_limit=30 --locust_requests=20
2017-05-22 20:16:22,432 INFO   MainThread
Starting load test
Function: lambda_locust
Ramp time: 0
Threads: 1
Lambda payload: {'locustfile': 'locustfile_example.py', 'host': 'https://example.com', 'num_requests': 20, 'num_clients': 20, 'hatch_rate': 10, 'run_time':'3m'}

2017-05-22 20:16:22,432 INFO   thread_1    thread started
2017-05-22 20:16:22,436 INFO   MainThread  threads: 1, rpm: 0, run_time: 0, requests_total: 0, request_fail_ratio: 0, invocation_error_ratio: 0
2017-05-22 20:16:25,440 INFO   MainThread  threads: 1, rpm: 0, run_time: 3, requests_total: 0, request_fail_ratio: 0, invocation_error_ratio: 0
2017-05-22 20:16:27,983 INFO   thread_1    Invocation complete. Requests (errors): 20 (0), execution time: 5186, sleeping: 0
2017-05-22 20:16:28,446 INFO   MainThread  threads: 1, rpm: 216, run_time: 6, requests_total: 20, request_fail_ratio: 0.0, invocation_error_ratio: 0.0
2017-05-22 20:16:31,448 INFO   MainThread  threads: 1, rpm: 216, run_time: 9, requests_total: 20, request_fail_ratio: 0.0, invocation_error_ratio: 0.0
2017-05-22 20:16:32,860 INFO   thread_1    Invocation complete. Requests (errors): 20 (0), execution time: 4798, sleeping: 0
2017-05-22 20:16:34,453 INFO   MainThread  threads: 1, rpm: 246, run_time: 12, requests_total: 40, request_fail_ratio: 0.0, invocation_error_ratio: 0.0
2017-05-22 20:16:37,454 INFO   MainThread  threads: 1, rpm: 246, run_time: 15, requests_total: 40, request_fail_ratio: 0.0, invocation_error_ratio: 0.0
2017-05-22 20:16:38,110 INFO   thread_1    Invocation complete. Requests (errors): 20 (0), execution time: 5176, sleeping: 0
2017-05-22 20:16:40,458 INFO   MainThread  threads: 1, rpm: 229, run_time: 18, requests_total: 60, request_fail_ratio: 0.0, invocation_error_ratio: 0.0
2017-05-22 20:16:43,072 INFO   thread_1    Invocation complete. Requests (errors): 20 (0), execution time: 4882, sleeping: 0
2017-05-22 20:16:43,464 INFO   MainThread  threads: 1, rpm: 242, run_time: 21, requests_total: 80, request_fail_ratio: 0.0, invocation_error_ratio: 0.0
2017-05-22 20:16:46,466 INFO   MainThread  threads: 1, rpm: 242, run_time: 24, requests_total: 80, request_fail_ratio: 0.0, invocation_error_ratio: 0.0
2017-05-22 20:16:48,237 INFO   thread_1    Invocation complete. Requests (errors): 20 (0), execution time: 5041, sleeping: 0
2017-05-22 20:16:49,467 INFO   MainThread  threads: 1, rpm: 232, run_time: 27, requests_total: 100, request_fail_ratio: 0.0, invocation_error_ratio: 0.0
2017-05-22 20:16:52,468 INFO   MainThread  threads: 1, rpm: 232, run_time: 30, requests_total: 100, request_fail_ratio: 0.0, invocation_error_ratio: 0.0
2017-05-22 20:16:53,347 INFO   thread_1    Invocation complete. Requests (errors): 20 (0), execution time: 5041, sleeping: 0
2017-05-22 20:16:55,468 INFO   MainThread  threads: 1, rpm: 235, run_time: 33, requests_total: 120, request_fail_ratio: 0.0, invocation_error_ratio: 0.0
2017-05-22 20:16:55,468 INFO   MainThread  Time limit reached
2017-05-22 20:16:55,468 INFO   MainThread  Waiting for threads to exit...
2017-05-22 20:16:58,666 INFO   thread_1    Invocation complete. Requests (errors): 20 (0), execution time: 5209, sleeping: 0
2017-05-22 20:16:58,667 INFO   thread_1    thread finished
2017-05-22 20:16:59,484 INFO   MainThread  Load test finished. lambda_invocation_count: 7, total_lambda_execution_time: 35333, requests_total: 140, request_fail_ratio: 0.0, invocation_error_ratio: 0.0
2017-05-22 20:16:59,484 INFO   MainThread  Aggregated results: {'success': {'GET_/': {'median_response_time': 285.7142857142857, 'total_rps': 19.753860147907517, 'avg_response_time': 615.4214285714286, 'max_response_time': 1703, 'min_response_time': 100, 'response_times': {'1100': 6, '1400': 9, '120': 2, '140': 14, '960': 3, '180': 9, '160': 11, '260': 6, '300': 3, '1200': 6, '340': 3, '200': 12, '1700': 2, '220': 9, '1300': 10, '380': 3, '440': 5, '240': 5, '1000': 2, '480': 2, '1600': 2, '1500': 9, '100': 1, '360': 1, '540': 1, '940': 1, '280': 1, '320': 1, '400': 1}, 'total_rpm': 1185.231608874451}}, 'fail': {}, 'num_requests': 140, 'num_requests_fail': 0, 'num_requests_success': 140, 'total_time': 35333, 'invocations': 7, 'approximate_cost': 7.4824e-05}
2017-05-22 20:16:59,484 INFO   MainThread  Exiting...

标签：

• lambda
• run
• info
• aws
• time
• requests
• 小包装
• locust
• ratio

欢迎加入QQ群-->： 979659372

推荐PyPI第三方库

• arangopipe

  机器学习元数据管理与分析包

• TQapis

  Python包提供了一个简单的接口来支持美国国债量化网的http协议

• hlymode

  这是第一个对外发布的模块，测试哦

• NetworkSim

  未提供项目说明

• myBestTool

  我的工具

• choicelib

  从给定列表中选择最佳相似接口库

• googlecloudscheduler

  防止攻击的包

• pyobjcframeworkmetal

  防止攻击的包

• delpywidget

  Jupyter笔记本上的Python块

• arkho-jsonquer

  基于JSON文件的查询路径原型

• sanic-dropzone

  用Sanic上传文件Dropzone.js.

• aoeapi

  用于的Python 3 APIageofempires.com网站

• lsst-alert-packet

  与Vera C.Rubin天文台警报包交互的代码

• core-service

  异步服务微框架

• featureengine

  防止攻击的包