python 3.6+库,用于交互提交给aws批处理。
reticle的Python项目详细描述
十字线
python 3.6+库,用于交互提交给aws批处理。
❯ pip install reticle
功能:
- 提交批处理作业
端到端示例
将作业部署到aws批处理的主要对象是批处理作业定义。 每个批处理作业定义都有名称、参数和某种形式的可选参数验证。
fromreticle.aws.batchimportJobDefinitionfromreticle.aws.s3importS3UriclassDemoJobDefinition(JobDefinition):"""A Batch job definition for demonstrating our API. Args: input_object: The S3 URI to the input object. """def__init__(self,input_object:S3Uri):self.input_object=input_object@propertydefname(self)->str:"""Return the job definition name."""return'demo-job'defvalidate(self)->None:"""Validate this parameterized job definition."""ifnotself.input_object.object_exists():raisef'S3 object does not exist: {self.input_object}'
现在,我们可以将参数化作业定义包装到批处理作业中,并设置特定的修订。
fromreticle.aws.batchimportBatchJobdefinition=DemoJobDefinition(input_object='s3://bucket/object')definition.at_revision('6')job=BatchJob(definition)
提交此批处理作业很容易,并且可以立即执行自省:
response=job.submit(queue='prod')
当作业处于RUNNING
状态时,我们可以访问作业的cloudwatch日志:
forlog_eventinjob.log_stream_events():print(log_event)"""LogEvent(timestamp="1543809952329", message="You have started up this demo job", ingestion_time="1543809957080")LogEvent(timestamp="1543809955437", message="Configuration, we are loading from...", ingestion_time="1543809957080")LogEvent(timestamp="1543809955437", message="Defaulting to approximate values", ingestion_time="1543809957080")LogEvent(timestamp="1543809955437", message="Setting up logger, nothing to see here", ingestion_time="1543809957080")"""
如果必须,我们可以取消工作,只要我们提供理由:
job.terminate(reason='I was just testing!')