使用BigQuery测试BigQuery
bqqtest的Python项目详细描述
测试BigQuery查询的工具
可以测试查询逻辑到BigQuery
Basic Usage
Simple
frombqqtestimportQueryTestfromgoogle.cloudimportbigquery# expectedexpected_schema=[{"name":"name","type":"STRING","mode":"NULLABLE"},{"name":"value","type":"INT64","mode":"NULLABLE"},]expected_datum=[["abc",100],["bbb",333]]expected={"schema":expected_schema,"datum":expected_datum}# actualtarget_schema=[{"name":"name","type":"STRING","mode":"NULLABLE"},{"name":"value","type":"INT64","mode":"NULLABLE"},]target_datum=[["abc",100],["bbb",333]]tables={"test.target_table":{"schema":target_schema,"datum":target_datum}}eval_query={"query":"SELECT * FROM test.target_table","params":[]}qt=QueryTest(bigquery.Client(),expected,tables,eval_query)success,diff=qt.run()success# True
Group By
frombqqtestimportQueryTestfromgoogle.cloudimportbigquery# expectedexpected_schema=[{"name":"item","type":"STRING","mode":"NULLABLE"},{"name":"total","type":"INT64","mode":"NULLABLE"},]expected_datum=[["abc",300],["bbb",333]]expected={"schema":expected_schema,"datum":expected_datum}# actualtarget_schema=[{"name":"item","type":"STRING","mode":"NULLABLE"},{"name":"value","type":"INT64","mode":"NULLABLE"},]target_datum=[["abc",100],["bbb",333],["abc",200]]tables={"test.target_table":{"schema":target_schema,"datum":target_datum}}eval_query={"query":"SELECT item, SUM(value) AS total FROM test.target_table GROUP BY item","params":[],}qt=QueryTest(bigquery.Client(),expected,tables,eval_query)success,diff=qt.run()success# True
Multi Table
frombqqtestimportQueryTestfromgoogle.cloudimportbigquery# expectedexpected_schema=[{"name":"item","type":"STRING","mode":"NULLABLE"},{"name":"value","type":"INT64","mode":"NULLABLE"},]expected_datum=[["abc",100],["bbb",333],["xxxx",888],["zzzz",999]]expected={"schema":expected_schema,"datum":expected_datum}# actualtarget_schema=[{"name":"item","type":"STRING","mode":"NULLABLE"},{"name":"value","type":"INT64","mode":"NULLABLE"},]target_datum1=[["abc",100],["bbb",333]]target_datum2=[["xxxx",888],["zzzz",999]]tables={"test.table1":{"schema":target_schema,"datum":target_datum1},"test.table2":{"schema":target_schema,"datum":target_datum2},}eval_query={"query":"SELECT * FROM `test.table1` UNION ALL SELECT * FROM `test.table2`","params":[],}qt=QueryTest(bigquery.Client(),expected,tables,eval_query)success,diff=qt.run()success# True
特征
see also https://qiita.com/tamanobi/items/9434ca0dbd5f0d3018d9
- 使用WITH临时生成BigQuery中未保存的测试数据。
- BigQuery是以保存的数据扫描量和API请求数收费的，所以可以抑制费用进行单元测试。
- 费用详情请参阅BigQuery的官方文档
- 测试不需要重写查询
- 在库内部，重写目标表格的Identifier以替换表格
注意
直接向BigQuery发行查询。- 项目
标签：
欢迎加入QQ群-->： 979659372
