非官方工作流API客户端
tdworkflow的Python项目详细描述
非官方珍宝工作流API客户端。在
安装
pip install tdworkflow
如果要使用开发版本，请按如下方式运行：
^{pr2}$使用
importosfromtdworkflow.clientimportClientapikey=os.getenv("TD_API_KEY")client=Client(site="us",apikey=apikey)# Or, write endpoint explicitly# client = Client(endpoint="api-workflow.treasuredata.com", apikey=apikey)projects=client.projects("pandas-df")secrets={"td.apikey":apikey,"td.apiserver":"https://api.treasuredata.com","test":"secret-foo"}client.set_secrets(projects[0],secrets)client.secrets(projects[0])# ['td.apikey', 'td.apiserver', "test"]client.delete_secrets(projects[0],["test","td.apiserver"])
GitHub的Uplaod项目
在执行示例代码之前，必须安装gitpython
pip install gitpython
用git-python克隆示例库并上传digdag项目。在
importtempfileimportosimportshutilimporttdworkflowfromgitimportGit# Download example GitHub repositoorytempdir=tempfile.gettempdir()git_repo="https://github.com/treasure-data/treasure-boxes/"shutil.rmtree(os.path.join(tempdir,"treasure-boxes"))try:Git(tempdir).clone(git_repo)print("Clone repository succeeded")exceptException:print("Repository clone failed")raise# Upload specific Workflow projectapikey=os.getenv("TD_API_KEY")site="us"target_box=os.path.join("integration-box","python")target_path=os.path.join(tempdir,"treasure-boxes",target_box)client=tdworkflow.client.Client(site=site,apikey=apikey)project=client.create_project("my-project",target_path)
如果您想在浏览器上打开宝工作流控制台，可以获得如下工作流URL：
CONSOLE_URL={"us":"https://console.treasuredata.com/app/workflows","eu01":"https://console.eu01.treasuredata.com/app/workflows","jp":"https://console.treasuredata.co.jp/app/workflows",}workflows=client.project_workflows(project)workflows=list(filter(lambdaw:w.name!="test",workflows))ifworkflows:print(f"Project created! Open {CONSOLE_URL[site]}/{workflows[0].id}/info on your browser and clieck 'New Run' button.")else:print("Project creation failed.")
启动工作流会话
您可以使用Client.start_attempt启动工作流会话。在
attempt=client.start_attempt(workflows[0])# Wait attempt until finish. This may require few minutes.attempt=client.wait_attempt(attempt)
连接到开源digdag
由于珍宝工作流托管在digdag上，tdworkflow与开源digdag兼容。在
注意
开源digdagapi可能与珍宝工作流API不同，因此tdworkflow可能无法与开源digdag的某些API一起工作。在
下面是连接本地digdag服务器的示例代码。在
>>>importtdworkflow>>>importrequests>>>session=requests.Session()>>>client=tdworkflow.client.Client(...endpoint="localhost:65432",apikey="",_session=session,scheme="http")>>>client.projects()[Project(id=1,name='python-tdworkflow',revision='134fe2f9-ded3-4e7c-af8e-8a82d55d688b',archiveType='db',archiveMd5='5Lc6F6m3DtmBN4DA5MzK8A==',createdAt='2019-11-01T13:03:26Z',deletedAt=None,updatedAt='2019-11-01T13:03:26Z')]
- 项目
标签：
欢迎加入QQ群-->： 979659372
