Managing cloud platforms

We can use network automation techniques through Python to work on various cloud providers. From working on cloud instances, to spinning up new VMs, controlling full access like ACLs, and creating specific network layer tasks like VPNs, and network configurations of each instance, we can automate just about anything using available connectors or APIs in Python. Let's see some basic configuration and connections on the most popular cloud platform, Amazon Web Services (AWS) using Python.

AWS provides an extensive API through its SDK called Boto 3. Boto 3 provides two types of APIs to be used, a low-level API set that is used to interact with direct AWS services, and a high-layer Python friendly API set for quick interactions with AWS. Along with Boto 3, we also would need to have the AWS CLI that is used as a command-line interface (CLI) to interact with AWS from the local machine. Think of this as a CLI based tool that is equally like DOS is to Windows from a CLI perspective.

The installation of both the AWS CLI and Boto 3 is done using pip:

• To install from AWS CLI, use the following command:

pip install awscli

• To install from Boto 3, use the following command:

pip install boto3

Once installed, the packages are ready to use. However, we need to configure an access key in the AWS Web Management Console which will have a certain level of restrictions (that we will define while creating the access key).

Let's quickly set up a new access key to manage the AWS in Python from our local machine:

1. Log in to the AWS web console and select IAM as the option:

1. Click on Add user to create a username and password pair shown as follows:

1. Select username and ensure to check Programmatic access to get the access ID and secret key to be used in our Python calls:

1. We also need the user to be part of a certain group (for security restrictions). In our case we make it part of the admin group which has full rights on the AWS instance:

1. If we made our selections correctly, a user is created with the username we selected (booktest) with an access key and a secret access key:

1. Once we have this key, we go back to our Python installation and on the Command Prompt, call the AWS CLI command aws configure:

1. As per the questions asked, we fetch the values from the AWS web console and paste them in the CLI. The final question of Default output format can be text or json. However, for our purpose of automation and working with Python, we would select json instead of text.

Once we are done with this backend configuration, we are ready to test our scripts by calling the Boto 3 API in Python.

Let's see an example of getting all running instances on the current AWS account for which we have the key:

import boto3
ec2 = boto3.resource('ec2')
for instance in ec2.instances.all():
    print (instance)
    print (instance.id, instance.state)

Since we have already configured the backend credentials and key with the aws configure CLI command, we do not need to specify any credentials in our scripts. 

The output of the preceding code is as follows:

As we see in the preceding output, we get back two instances which are EC2 instances with their instance IDs. Additionally, we also get some other parameters for the currently configured instances. In some cases, if we do not want to use the current preconfigured keys, we can call the Python program by passing the values directly into Boto 3 functions as follows:

import boto3
 
aws_access_key_id = 'accesskey'
aws_secret_access_key = 'secretaccesskey'
region_name = 'us-east-2'
 
ec2 = boto3.client('ec2',aws_access_key_id=aws_access_key_id,aws_secret_access_key=aws_secret_access_key,region_name=region_name)

Let's see another example of fetching the private IP address and instance ID for each of the instances:

import boto3

ec2 = boto3.client('ec2')
response = ec2.describe_instances()
for item in response['Reservations']:
    for eachinstance in item['Instances']:
        print (eachinstance['InstanceId'],eachinstance['PrivateIpAddress'])

The preceding code gives the following output:

Using the Boto 3 API, we can also spin up new instances in our subscription. Let's see a final example of spinning up a new Virtual Machine(VM) with EC2 using Boto 3.

Before we call the Python to spin a new VM, we need to select which Amazon Machine Image (AMI) image to use for the instance. To find out the AMI image value, we need to open AMI in the AWS web console shown as follows:

Once we have finalized the AMI, we call the easy part, spinning the new VM:

import boto3
ec2 = boto3.resource('ec2')
ec2.create_instances(ImageId='amid-imageid', MinCount=1, MaxCount=5)

It will take some time for the script to execute, and the result value would be the instance with all its configured parameters based upon the AMI image ID selected. Similarly, we can spin up various type of instances or even new security filters using Boto 3 and ensure we have cloud controlling automation in place.