Utilizing Python Boto3 library and other AWS Services to stop ec2 instances

8 min readJul 30, 2022

For this project, we were provided with the following scenario:
Our DevOps engineering team often uses a development lab to test releases of our application. The Managers are complaining about the rising cost of our development lab and need to save money by stopping our (for this example) 3 ec2 instances after all engineers are clocked out.

Initial Task: Create a Python script that you can run that will stop all instances.

Advanced Task: We want to ensure that only our Development instances are stopped to make sure nothing in Production is accidentally stopped. Add logic to your script that only stops running instances that have the Environment: Dev tag.

COMPLEX Task:

Use your script in a Lambda function using Python 3.7 or higher runtime. Make sure the lambda runs on a set schedule daily. No one should be working in the Dev environment past 7pm. (Note: to test you may need to modify the time accordingly, so you aren’t waiting for 7pm)

Required items:

a. AWS IAM user account with admin Access (Personal or business account)
b. IAM permissions
c. Boto3 and AWS CLI to be installed on Cloud9 using the following commands:
1. pip3 install awscli
2. pip3 install boto3 {For reference please see https://boto3.amazonaws.com/v1/documentation/api/latest/guide/quickstart.html}

Below is a screen shot, of the Python script, that can be ran manually to stop 3x designated ec2 instances, when instructed to do so.

When executed from the AWS CLI prompt, the script will search through the ec2 instances that are currently running, after locating the 3x instanceIds, the stop_instances() function will be executed and the script will print out the below data, please note the InstanceIds that are in bold text, they are the same Ids that were included in the python script.

{‘StoppingInstances’: [{‘CurrentState’: {‘Code’: 64, ‘Name’: ‘stopping’}, ‘InstanceId’: ‘i-00cb99b282fd43430’, ‘PreviousState’: {‘Code’: 16, ‘Name’: ‘running’}}, {‘CurrentState’: {‘Code’: 64, ‘Name’: ‘stopping’}, ‘InstanceId’: ‘i-0a20507d47d28b35b’, ‘PreviousState’: {‘Code’: 16, ‘Name’: ‘running’}}, {‘CurrentState’: {‘Code’: 64, ‘Name’: ‘stopping’}, ‘InstanceId’: ‘i-0db75a618e9eb8d64’, ‘PreviousState’: {‘Code’: 16, ‘Name’: ‘running’}}], ‘ResponseMetadata’: {‘RequestId’: ‘778e244e-64e8–4e6f-aecb-6b68f895387e’, ‘HTTPStatusCode’: 200, ‘HTTPHeaders’: {‘x-amzn-requestid’: ‘778e244e-64e8–4e6f-aecb-6b68f895387e’, ‘cache-control’: ‘no-cache, no-store’, ‘strict-transport-security’: ‘max-age=31536000; includeSubDomains’, ‘content-type’: ‘text/xml;charset=UTF-8’, ‘content-length’: ‘1257’, ‘date’: ‘Sat, 30 Jul 2022 03:41:16 GMT’, ‘server’: ‘AmazonEC2’}, ‘RetryAttempts’: 0}}

So, that accomplishes the initial task (Create a Python script that you can run that will stop all instances.), however, I feel like I should do a little extra and attempt the advanced task: We want to ensure that only our Development instances are stopped to make sure nothing in Production is accidentally stopped. Add logic to your script that only stops running instances that have the Environment: Dev tag.

I built the below python script, with a few additional lines of code to include the logic to make sure that only the running instances with a name of Environment and a tag named Dev will be stopped.

When executed from the AWS CLI prompt, the script will search through the ec2 instances that are currently running, then it checks for any tags that may have a name of “Environment” and a Values of Dev”. For any instances that meet the criteria, those will be shut down or stopped and will need to be started manually when they are needed again.

Prior to running the script, I took the below screenshot from within the AWS Console to show how many ec2 instances were running and what they were named. There were 4x named Prod and another 3x named Dev, which should be stopped after a successful execution of the script.

Below is the output from the successfully completed stop_instances():

[‘i-0a20507d47d28b35b’, ‘i-00cb99b282fd43430’, ‘i-0db75a618e9eb8d64’]

{‘StoppingInstances’: [{‘CurrentState’: {‘Code’: 64, ‘Name’: ‘stopping’}, ‘InstanceId’: ‘i-00cb99b282fd43430’, ‘PreviousState’: {‘Code’: 16, ‘Name’: ‘running’}}, {‘CurrentState’: {‘Code’: 64, ‘Name’: ‘stopping’}, ‘InstanceId’: ‘i-0a20507d47d28b35b’, ‘PreviousState’: {‘Code’: 16, ‘Name’: ‘running’}}, {‘CurrentState’: {‘Code’: 64, ‘Name’: ‘stopping’}, ‘InstanceId’: ‘i-0db75a618e9eb8d64’, ‘PreviousState’: {‘Code’: 16, ‘Name’: ‘running’}}], ‘ResponseMetadata’: {‘RequestId’: ‘efecc35a-1f78–40d3-a953–281cd7aacafe’, ‘HTTPStatusCode’: 200, ‘HTTPHeaders’: {‘x-amzn-requestid’: ‘efecc35a-1f78–40d3-a953–281cd7aacafe’, ‘cache-control’: ‘no-cache, no-store’, ‘strict-transport-security’: ‘max-age=31536000; includeSubDomains’, ‘content-type’: ‘text/xml;charset=UTF-8’, ‘content-length’: ‘1257’, ‘date’: ‘Sat, 30 Jul 2022 03:58:28 GMT’, ‘server’: ‘AmazonEC2’}, ‘RetryAttempts’: 0}}

Looking in at the previous 7x ec2 instances in the AWS console, we find that the 3x instances that were tagged as Dev, were stopped and the 4x instances that are tagged with Prod are still running.

So… That takes care of the Advanced task but I still feel adventurous, so I am going for the COMPLEX task: Use your script in a Lambda function using Python 3.7 or higher runtime. Make sure the lambda runs on a set schedule daily. No one should be working in the Dev environment past 7pm. (Note: to test you may need to modify the time accordingly, so you aren’t waiting for 7pm) I have to admit, I have definitely “tweaked” the previous python code a bit more.

There are a few items that will be needed to implement this complex task:
1. Lambda function (2x, I’ll share with you later what the second one is for.)
2. IAM Role
3. IAM Permissions
4. CloudWatch Logs
5. Amazon EventBridge

1. Log into the AWS Console and select the IAM service
2. Click Roles
3. Click Create role
4. Keep default: AWS service
5. Select Lambda for the use case
6. Click Next
7. Click Create Policy
8. Click the tab labeled JSON
9. Enter the following code

10. Click Next: Tags button
11. Click Next: Review
12. Provide a new for the new policy and click Create Policy button
13. Open Lambda services
14. Create Function
15. Select Author from scratch
16. Enter a name for the new function
17. Select Python 3.7 or higher for the Runtime
18. Select Use an existing role
19. Click the dropdown arrow and select the name of the role that you just made
20. Click Create function

21. Scroll down and select the Code button
22. Clear out the info that is currently placed in the lambda_function.py script to stop ec2 instances with Dev for the tag
23. Enter the following code:

24. Click Deploy to save the changes
25. Click Test and configure, then save the test event
26. Now click Test the test the new code
27. If it was executed correctly, you should see this result:

28. Create a new lambda function for starting the Dev servers on a predetermined schedule (Surprise!!!)
29. Create Function
30. Select Author from scratch
31. Enter a name for the new function
32. Select Python 3.7 or higher for the Runtime
33. Select Use an existing role
34. Click the dropdown arrow and select the name of the role that you just made
35. Click Create function
36. Scroll down and select the Code button
37. Clear out the info that is currently placed in the lambda_function.py script to stop ec2 instances with Dev for the tag
38. Enter the following code:

39. Click Deploy to save the changes
40. Click Test and configure, then save the test event
41. Now click Test the test the new code
42. If it was executed correctly, you should see this result:

43. Open CloudWatch and click on Log Groups
44. You should see a log group for the ec2 instance(s) that you created earlier

45. Click on one of the entries labeled /aws/lambda/instance name
46. Click on the Log stream

47. Log events give a detailed account of what happened with the Lambda function

48. While in CloudWatch, locate and expand Events, then select Rules
49. Click on Go To Amazon EventBridge

50. Click on Create Rule
51. Enter a name
52. Select Schedule for the Rule type
53. Click Next

54. Configure the Schedule pattern, select the default utilizes a Cron expression (A cron expression is a combination of 6 values that tell EventBridge when to run the rule. For example cron(0 12 * * ? *) runs the rule every day at 12:00pm UTC+0.) or select the option that allows a setting using a Rate expression (A rate expression starts when the rule is created and runs the rule on a defined interval. For example, rate(1 minute) runs the rule every minute and rate(1 hour) runs the rule every hour.