我想使用像pytest-benchmark
这样的插件来显示并行调用的运行时统计信息test_valid_submission(0)
,…,test_valid_submission(EXPERIMENT_SIZE)
为此,我有以下代码,它试图使用pytest-benchmark
(类似于Grouping Parametrized Benchmarks with pytest)实现这一点:
@pytest.mark.parametrize("counter", range(EXPERIMENT_SIZE))
def test_performance_under_load(benchmark, counter):
benchmark(test_valid_submission, counter)
当我打电话时
pytest --workers auto --tests-per-worker auto -vv --benchmark-only --benchmark-verbose --group-by=func
我希望最后得到a benchmark summary table,
我的EXPERIMENT_SIZE
并行test_valid_submission()
调用的每个运行时的最小值、最大值、平均值和标准偏差。遗憾的是,没有打印基准汇总表(请参见下面的详细信息)
@hoefling评论说pytest-benchmark
不支持并行运行和收集基准数据
是否有其他pytest插件(或其他解决方案)可以
EXPERIMENT_SIZE
个并行test_valid_submission(x)
调用并将它们分组在一起test_valid_submission(x)
,另一个用于test_invalid_submission(x)
通过pytest-benchmark
、EXPERIMENT_SIZE=3
、iterations=1
和rounds=1
,我得到以下输出(但即使在EXPERIMENT_SIZE>=5
时,它也显示rounds=5
,并且没有统计数据)
但是我得到了以下输出(删除了EXPERIMENT_SIZE=3
和不相关的行):
============================== test session starts ========================================== platform linux -- Python 3.6.10, pytest-5.3.5, py-1.8.1, pluggy-0.13.1 -- /anaconda3/envs/reg/bin/python benchmark: 3.2.3 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000) plugins: repeat-0.8.0, cov-2.8.1, bdd-3.2.1, clarity-0.3.0a0, benchmark-3.2.3, parallel-0.1.0 collected 22 items pytest-parallel: 8 workers (processes), 3 tests per worker (threads) ... test/test_x.py::test_valid_submission SKIPPED Computing precision for time.perf_counter ... 50.99ns. Calibrating to target round 5.00us; will estimate when reaching 2.50us (using: time.perf_counter, precision: 50.99ns). Computing precision for time.perf_counter ... 48.98ns. Calibrating to target round 5.00us; will estimate when reaching 2.50us (using: time.perf_counter, precision: 48.98ns). Computing precision for time.perf_counter ... 49.01ns. Calibrating to target round 5.00us; will estimate when reaching 2.50us (using: time.perf_counter, precision: 49.01ns). Measured 1 iterations: 105.72s. Running 5 rounds x 1 iterations ... Measured 1 iterations: 105.73s. Running 5 rounds x 1 iterations ... Measured 1 iterations: 117.20s. Running 5 rounds x 1 iterations ... Ran for 339.53s. Ran for 350.11s. Ran for 461.53s. test/test_x.py::test_performance_under_load[2] PASSED test/test_x.py::test_performance_under_load[1] PASSED test/test_x.py::test_performance_under_load[0] PASSED ========================== 3 passed, 19 skipped in 714.05s (0:11:54) ========================
或者使用benchmark.pedantic(test_valid_submission, args=[counter], iterations=1, rounds=1)
也不会导致打印统计数据
目前没有回答
相关问题 更多 >
编程相关推荐