Terraform – Security Groups, Provisioners and Ansible

My previous blog posts looked at what Terraform is and also how to deploy an AWS EC2 instance with Terraform. In this blog post I am going to alter my main.tf file so that Terraform also creates a new AWS security group with some ingress rules and also runs an Ansible playbook against the new EC2 instance.

Note: As with my previous posts I am aiming to use the free-tier on AWS for as much as possible. If you are following along please make sure to check your AWS billing and to terminate (or terraform destroy) instances that are no longer required.

First up, a brief introduction to Security Groups.

AWS Security Groups

AWS Security Groups allow access to AWS instances (e.g. EC2 instances) and can be used to allow only certain ports (e.g. 22 for SSH), certain protocols (e.g. TCP or UDP), or certain IP CIDR ranges for ingress (inbound) or egress (outbound).

Security Groups can be set up in the AWS Management Console (a.k.a. AWS website) and also via tools like Terraform, which is what this blog post is aiming to do.

It is important to set up Security Groups to secure and gain access to your AWS resources, it is not recommended to leave resources’ ports/services open to IP addresses or CIDR ranges that are not needed (e.g. 0.0.0.0/0 will leave it open to all IP addresses).

During this blog post I am setting up a Security group to allow ingress (inbound traffic) to port 8080 where BusyBox is going to display a basic web page, and port 22 for SSH so that I can connect. I am limiting access to these ports to my IP address (in CIDR format, its octet.octet.octet.octet/32, replacing the octets with the IP address). If you need your IP address and are not sure what it is then a quick “what is my IP address” on Google will give you an answer.

main.tf

My main.tf file for this blog posts reads as follows:

provider "aws" {
   profile = "default"
   region = "eu-west-2"
}

variable "web_port" {
   description = "Port to handle HTTP requests"
   type = number
   default = 8080
}

variable "ssh_port" {
   description = "Port to handle SSH"
   type = number
   default = 22
}

resource "aws_instance" "geektechstuff_tf_example_ec2" {
   ami = "ami-00f6a0c18edb19300"
   instance_type = "t2.micro"
   key_name = "SSH_KEY_FILE_NAME"
   vpc_security_group_ids = [aws_security_group.geektechstuff_tf_example_sg.id]
   
   user_data = <<-EOF
               #!/bin/bash
               echo "GeekTechStuff: You've connected to web page! Web Security Group worked." > index.html
               nohup busybox httpd -f -p ${var.web_port} &
               EOF
   
   tags = {
     Name = "geektechstuff_ec2_test"
     Purpose = "terraform testing"
     }
     
    provisioner "remote-exec" {
      connection {
          type = "ssh"
          host = aws_instance.geektechstuff_tf_example_ec2.public_ip
          user = "ubuntu"
          private_key = file("SSH_KEY_FILE_NAME.pem")
          timeout = "40s"
          }
      inline = [
      "sudo apt-get -y install python3" 
    ]
  }
  
   provisioner "local-exec" {
      command = "ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -u ubuntu --private-key ~/.ssh/SSH_KEY_FILE_NAME.pem -i '${aws_instance.geektechstuff_tf_example_ec2.public_ip},' ansible_playbook.yml"
}
     
}

resource "aws_security_group" "geektechstuff_tf_example_sg" {
    name = "geek_tf_sg_instance"
    ingress {
        from_port = var.web_port
        to_port = var.web_port
        protocol = "tcp"
        cidr_blocks = ["IP_ADDRESS_IN_CIDR_FORMAT"]
    }
    ingress {
        from_port = var.ssh_port
        to_port = var.ssh_port
        protocol = "tcp"
        cidr_blocks = ["IP_ADDRESS_IN_CIDR_FORMAT"]
    }
    tags = {
      Name = "geektechstuff_sg_test"
      Purpose = "terraform testing"
      }
}

output "public_ip" {
    value = aws_instance.geektechstuff_tf_example_ec2.public_ip
    description = "Public IP of the EC2 instance"
}

I have introduced a few new concepts since my previous blog post so will go through them now.

Variables:

Terraform supports variables and in concept of Dev Ops’ DRY (Don’t Repeat Yourself) I am trying to use them so that if I have to replace a value I can do it in one place rather than several.

variable "web_port" {
description = "Port to handle HTTP requests"
type = number
default = 8080
}

variable "ssh_port" {
description = "Port to handle SSH"
type = number
default = 22
}

Each variable is given a name (e.g. “ssh_port”), a description, a type and I have included a default value. If needed a value can be passed to Terraform via the command line when running terraform apply which would take priority over the default value.

EC2 with SSH Key, Security Group and User Data:

resource "aws_instance" "geektechstuff_tf_example_ec2" {
ami = "ami-00f6a0c18edb19300"
instance_type = "t2.micro"
key_name = "SSH_KEY_FILE_NAME"
vpc_security_group_ids = [aws_security_group.geektechstuff_tf_example_sg.id]

user_data = <<-EOF
#!/bin/bash
echo "GeekTechStuff: You've connected to web page! Web Security Group worked." > index.html
nohup busybox httpd -f -p ${var.web_port} &
EOF

tags = {
Name = "geektechstuff_ec2_test"
Purpose = "terraform testing"
}

I already have a key pair in AWS and the “SSH_KEY_FILE_NAME” should match the key pair name that AWS already has.

“vpc_security_group_ids ” references the AWS Security Group that I am going to ask Terraform to create and is detailed later in the main.tf file. Don’t worry, Terraform is smart enough to know about this dependency and will create the objects in the correct order.

“user_data” is passed to the AWS EC2 instance at creation. As I want to test that my AWS Security Group is created and is working I’m asking that some text is written to index.html (in the root directory), and that BusyBox (a web server) is started up and providing access via the port defined in the variable web_port.

Provisioner

I using two provisioners during this project.

provisioner "remote-exec" {
connection {
type = "ssh"
host = aws_instance.geektechstuff_tf_example_ec2.public_ip
user = "ubuntu"
private_key = file("SSH_KEY_FILE_NAME.pem")
timeout = "40s"
}
inline = [
"sudo apt-get -y install python3" 
]
}

provisioner "local-exec" {
command = "ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -u ubuntu --private-key ~/.ssh/SSH_KEY_FILE_NAME.pem -i '${aws_instance.geektechstuff_tf_example_ec2.public_ip},' ansible_playbook.yml"
}

}

“remote-exec” runs commands remotely (e.g. on the EC2 instance being created) and needs connection details so that it can connect. The “inline”  details are the commands that I want to run. Ubuntu already comes with Python3 installed so you may wonder why I’ve asked it to run this command and the simple answer is that I needed Terraform to wait until the EC2 instance was created, running and allowing SSH connections before it attempts anything with Ansible otherwise Ansible will fail. The remote-exec provisioner allows me that work-around as it keeps trying SSH until its either available or a timeout (40 seconds in this instance) is reached.

 

“local-exec” runs commands on the device running Terraform, in this case my Raspberry Pi 4 which also has Ansible installed. The command is telling ansible-playbook to run, with host checking turned off, as the user “ubuntu” (default for EC2 Ubuntu instances), with the private SSH key that matches the public key AWS has, to an inventory (-i) that is the public IP address of the EC2 instance and that the playbook “ansible_playbook.yml” (stored in the same directory as my Terraform files) should run.

For this example I used my Ansible playbook that detects the operating system (OS) and creates a folder depending on the OS detected.

Security Group

resource "aws_security_group" "geektechstuff_tf_example_sg" {
    name = "geek_tf_sg_instance"
    ingress {
        from_port = var.web_port
        to_port = var.web_port
        protocol = "tcp"
        cidr_blocks = ["IP_ADDRESS_IN_CIDR_FORMAT"]
    }
    ingress {
        from_port = var.ssh_port
        to_port = var.ssh_port
        protocol = "tcp"
        cidr_blocks = ["IP_ADDRESS_IN_CIDR_FORMAT"]
    }
    tags = {
      Name = "geektechstuff_sg_test"
      Purpose = "terraform testing"
      }
}

The security group allows ingress on the ports defined in the variables web_port and ssh_port.

Output

Terraform allows an output option which in this example outputs the public IP address of the EC2 instance that has been created, allowing me to easily see the IP address I need to enter in my browser or for my SSH test.

output "public_ip" {
    value = aws_instance.geektechstuff_tf_example_ec2.public_ip
    description = "Public IP of the EC2 instance"
}

Testing

Terraform reported that it had created two new resources (the EC2 instance and the Security Group) and on testing:

The web_port and user_data options worked.

The ssh_port and Ansible (local-exec) options worked.

Terraform Destroy

Remember to terraform destroy if you no longer need the resources that Terraform has created.