欧洲xfel元数据目录web应用程序的python客户端,可在https://in.xfel.eu/metadata上找到
metadata-client的Python项目详细描述
mymdc是欧洲xfel数据管理的web应用程序设计。
此库(metadata_client)是所公开的restful api的客户端 欧洲xfel元数据目录web应用程序mymdc (https://in.xfel.eu/metadata)。
存储库:
依赖项:
- OAuthlib(https://pypi.python.org/pypi/oauthlib)
- 请求(https://github.com/kennethreitz/requests)
- 请求oauthlib(https://github.com/requests/requests-oauthlib)
- OAuth2 xfel_客户端(https://git.xfel.eu/gitlab/ITDM/oauth2_xfel_client)
安装
python项目
- 安装要求,如果以前从未安装过
1.1. For OS X distributions:
sudo port install python35 sudo port sudo port select --set python3 python35 sudo port install py35-pip sudo port select --set pip pip35 sudo port install py35-nose sudo port select --set nosetests nosetests-3.5 pip install pycodestyle1.2. For Linux distributions:
sudo apt-get update sudo apt-get install python3.5
- 在python环境中提供metadata_客户端库
2.1. Install it via pip:
# Install dependencies from local wheels files pip install --no-index --upgrade --find-links ./external_dependencies/* # Install dependencies from the pypi pip install -r requirements.txtOr as a normal python project (via .egg file):
python setup.py install python setup.py install --userRunning this command the “compiled” metadata_client-3.0.3-py3.4.egg file is generated under the current Python installation site-packages folder.
or. Install it as a normal python project (via Wheel):
python setup.py bdist_wheelRunning this command 2 folders are generated under the current Python installation site-packages folder:
- metadata_client with the sources;
- metadata_client-3.0.3.dist-info/ with Wheels configuration files.
要识别python站点包文件夹,请运行:
python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())"
用法
要使用此项目,您需要导入它。
如果您想直接与api方法交互,那么应该导入metadataclientapi类:
from metadata_client.metadata_client_api import MetadataClientApi
如果要与模型类交互,则应导入metadataclient
from metadata_client.metadata_client import MetadataClient
或导入所有内容:
import metadata_client
连接到MDC(元数据目录):
from metadata_client.metadata_client import MetadataClient # Necessary configuration variables to establish a connection user_id = '201ed15ff071a63e76cb0b91a1ab17b36d5f92d24b6df4497aa646e39c46a324' user_secret = 'a8ae80f5e96531f19bf2d2b6102f5a537196aca44a673ad36533310e07529757' user_email = 'luis.maia@xfel.eu' # metadata_web_app_url = 'https://in.xfel.eu/dev_metadata' token_url = 'https://in.xfel.eu/dev_metadata/oauth/token' refresh_url = 'https://in.xfel.eu/dev_metadata/oauth/token' auth_url = 'https://in.xfel.eu/dev_metadata/oauth/authorize' scope = '' base_api_url = 'https://in.xfel.eu/dev_metadata/api/' # Generate the connection client_conn = MetadataClient(client_id=user_id, client_secret=user_secret, user_email=user_email, token_url=token_url, refresh_url=refresh_url, auth_url=auth_url, scope=scope, base_api_url=base_api_url)
与MDC(元数据目录)的交互:
2.1 Example data_group_types:
all_group_types = MetadataClient.get_all_data_group_types(client_conn) all_group_types # >>> {'success': True, # 'data': [{'description': '', 'identifier': 'RAW', 'name': 'Raw', 'flg_available': True, 'id': 1}, # {'description': '', 'identifier': 'CAL', 'name': 'Calibration', 'flg_available': True, 'id': 2}, # {'description': '', 'identifier': 'PROC', 'name': 'Processed', 'flg_available': True, 'id': 3}, # {'description': '', 'identifier': 'REDU', 'name': 'Reduced', 'flg_available': True, 'id': 4}, # {'description': '', 'identifier': 'SIM', 'name': 'Simulation', 'flg_available': True, 'id': 5}, # {'description': '', 'identifier': 'UNK', 'name': 'Unknown', 'flg_available': True, 'id': 6}], # 'app_info': {}, # 'info': 'Got data_group_type successfully'} all_group_types['success'] # >>> True all_group_types['data'][0] # >>> {'description': '', 'identifier': 'RAW', 'name': 'Raw', 'flg_available': True, 'id': 1} all_group_types['data'][0]['name'] # >>> 'Raw'2.2 Example instruments:
all_xfel_instruments = MetadataClient.get_all_xfel_instruments(client_conn) >>> for instrument in all_xfel_instruments['data']: ... print('id = {0} | name = {1}'.format(instrument['id'], instrument['name'])) ... # id = -1 | name = test-instrument # id = 1 | name = SPB/SFX SASE1 # id = 2 | name = FXE SASE1 # id = 3 | name = SQS SASE3 # id = 4 | name = SCS SASE3 # id = 5 | name = MID SASE2 # id = 6 | name = HED SASE2 # id = 7 | name = Hera South Detector Test Stand # id = 8 | name = SASE1 Test Stand # id = 9 | name = SASE2 Test Stand # id = 10 | name = SASE3 Test Stand2.3 Get instrument active proposal:
active_proposal = MetadataClient.get_active_proposal_by_instrument(client_conn, 1)2.4 Register Run replica:
# (e.g. proposal_number == 1234) # (e.g. proposal_number == 12) # (e.g. repository_identifier == 'XFEL_GPFS_OFFLINE_RAW_CC') resp = MetadataClient.register_run_replica(client_conn, proposal_number, run_number, repository_identifier) # resp = {'success': True, # 'info': 'Run replica registered successfully', # 'data': {'experiment_id': '-1', # 'sample_id': '-1', # 'run_id': '1588', # 'data_group_id': '777'}, # 'app_info': {}}2.5 Unregister Run replica:
# (e.g. proposal_number == 1234) # (e.g. proposal_number == 12) # (e.g. repository_identifier == 'XFEL_GPFS_OFFLINE_RAW_CC') resp = MetadataClient.unregister_run_replica(client_conn, proposal_number, run_number, repository_identifier) # resp = {'success': True, # 'info': 'Run replica unregistered successfully', # 'data': {'data_group_id': '-1', # 'repository_id': '1', # 'flg_available': 'false'}, # 'app_info': {}}
有关其他示例,请查看tests/文件夹。
开发和测试
开发时,在提交更改之前,请验证:
所有测试继续成功通过(以验证运行nosetests):
# Go to the source code directory cd metadata_client # Run all tests nosetests . # Run all tests and get information about coverage for all files inside metadata_client package pip install python-dateutil pip install nose-cov nosetests --with-cov --cover-erase --cover-inclusive --cov-report term-missing --cov metadata_client # Run all tests with xunit nosetests --where=./metadata_client/ --with-xunit --xunit-file=pythonTest.xml # If you don't want use nosetests you can simply run the test class python metadata_client/tests/metadata_client_test.py
代码始终遵守pycodesyle代码约定(以验证运行pycodesyle):
pycodestyle .
要为依赖项生成所有wheels文件,请执行:
# Generate Wheels to its dependencies pip wheel --wheel-dir=./external_dependencies -r requirements.txt pip wheel --wheel-dir=./external_dependencies --find-links=./external_dependencies -r requirements.txt # Generate Wheels to itself and dependencies pip wheel --wheel-dir=./external_dependencies . pip wheel --wheel-dir=./external_dependencies --find-links=./external_dependencies .
请确保在requirements.txt和setup.py文件中具有所需的版本。
正在https://pypi.org上注册库
要注册此python库,需要执行以下步骤:
# Install twine python -m pip install --upgrade twine # Generates egg file in the dist/ folder python setup.py install # Upload new version .egg and .whl files twine upload dist/* twine upload metadata_client-3.0.3-py3-none-any.whl # In case a teste is necessary, it is possible to test it against test.pypi.org twine upload --repository-url https://test.pypi.org/legacy/ dist/* --verbose