计划启动EC2实例并在i中运行python脚本

Question

我试图在AWS中调度python脚本，但是我不希望实例一直运行。因此，尝试自动化以下过程：

1. 在特定时间启动EC2实例
2. 在其中运行python脚本
3. 作业完成后停止EC2实例。

我无法将此脚本直接作为Lambda函数运行，因为该脚本执行一些需要更多RAM的并行处理，因此选择一个更大的AWS实例，而不是将其作为Lambda函数编写。另外，不要让这个实例一直运行，因为它很昂贵。

到目前为止，我遵循Automatic starting and stopping of AWS EC2 instances with Lambda and CloudWatch · matoski.com并创建了一个Lambda函数来在特定时间启动和停止实例，但是在实例启动后，我找不到运行python脚本的方法。

有人能指点我正确的方向吗？

Answer 1

对于提出这个问题的未来开发人员来说，一种新的解决方法是：

1. 使用包含AmazonEC2RoleforSSM策略的角色创建EC2
2. 创建lambda来执行唤醒、运行命令、关机
3. 使用Cloudwatch事件触发lambda

所以：

1. 按以下步骤操作：https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html

2. 使用以下lambda骨架：

import time
import boto3

REGION_NAME = 'us-east-1'

WORKING_DIRECTORY = '<YOUR WORKING DIRECTORY, IF ANY>'

COMMAND = """
    echo "Hello, world!"
    """

INSTANCE_ID = '<YOUR INSTANCE ID>'


def start_ec2():
    ec2 = boto3.client('ec2', region_name=REGION_NAME)
    ec2.start_instances(InstanceIds=[INSTANCE_ID])

    while True:
        response = ec2.describe_instance_status(InstanceIds=[INSTANCE_ID], IncludeAllInstances=True)
        state = response['InstanceStatuses'][0]['InstanceState']

        print(f"Status: {state['Code']} - {state['Name']}")

        # If status is 16 ('running'), then proceed, else, wait 5 seconds and try again
        if state['Code'] == 16:
            break
        else:
            time.sleep(5)

    print('EC2 started')


def stop_ec2():
    ec2 = boto3.client('ec2', region_name=REGION_NAME)
    ec2.stop_instances(InstanceIds=[INSTANCE_ID])

    while True:
        response = ec2.describe_instance_status(InstanceIds=[INSTANCE_ID], IncludeAllInstances=True)
        state = response['InstanceStatuses'][0]['InstanceState']

        print(f"Status: {state['Code']} - {state['Name']}")

        # If status is 80 ('stopped'), then proceed, else wait 5 seconds and try again
        if state['Code'] == 80:
            break
        else:
            time.sleep(5)

    print('Instance stopped')


def run_command():
    client = boto3.client('ssm', region_name=REGION_NAME)

    time.sleep(10)  # I had to wait 10 seconds to "send_command" find my instance 

    cmd_response = client.send_command(
        InstanceIds=[INSTANCE_ID],
        DocumentName='AWS-RunShellScript',
        DocumentVersion="1",
        TimeoutSeconds=300,
        MaxConcurrency="1",
        CloudWatchOutputConfig={'CloudWatchOutputEnabled': True},
        Parameters={
            'commands': [COMMAND],
            'executionTimeout': ["300"],
            'workingDirectory': [WORKING_DIRECTORY],
        },
    )

    command_id = cmd_response['Command']['CommandId']
    time.sleep(1)  # Again, I had to wait 1s to get_command_invocation recognises my command_id

    retcode = -1
    while True:
        output = client.get_command_invocation(
            CommandId=command_id,
            InstanceId=INSTANCE_ID,
        )

        # If the ResponseCode is -1, the command is still running, so wait 5 seconds and try again
        retcode = output['ResponseCode']
        if retcode != -1:
            print('Status: ', output['Status'])
            print('StdOut: ', output['StandardOutputContent'])
            print('StdErr: ', output['StandardErrorContent'])
            break

        print('Status: ', retcode)
        time.sleep(5)

    print('Command finished successfully') # Actually, 0 means success, anything else means a fail, but it didn't matter to me
    return retcode


def lambda_handler(event, context):
    retcode = -1
    try:
        start_ec2()
        retcode = run_command()
    finally:  # Independently of what happens, try to shutdown the EC2
        stop_ec2()

    return retcode

1. 按以下步骤操作：https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/RunLambdaSchedule.html