Using Ansible Roles, aka ELK Stack Ansible Playbook (V2)

I have a working Ansible Playbook that installs the ELK stack (Elasticsearch, Logstash, Kibana and for this example, Filebeat) across three different computers. I have tested the playbook several times and it works each time which is a great sign, however the playbook itself is a little messy.

Note: All the files for my ELK Ansible playbook can be found on my GitHub at:

https://github.com/geektechdude/ELK_Ansible_Playbook

As noted in my previous blog post, I have partially been inspired by Jeff Geerling’s Ansible Live Sessions on YouTube for this series of Ansible Playbook blogposts and partially inspired by my recent learnings around ELK.

Currently the playbook clocks in at 181 lines long which makes reading or editing it a little harder than it needs to be. Thankfully Ansible has a solution for this called roles.

Ansible Roles

Ansible roles can be called on from within a playbook. The role has all the tasks that you want Ansible to carry out for a particular sequence, e.g. all the tasks to install Kibana.  I am going to define 4 roles with a basic structure, these 4 roles are:

• elastic
• logstash
• kibana
• filebeat

Having these four roles will allow me to reduce the number of lines in my playbook and should allow me to edit the sequence of each role more easily as they will be in their own shorter defined area rather than somewhere in a long sequence of playbook commands.

Structure Of A Role

For roles to work in Ansible they need to be placed in a folder called roles located in the same directory as the playbook calling them. The roles folder then contains a subfolder named the same as the role name being called in the playbook. This “role name” subfolder then contains two subfolders called “meta” and “tasks”, both of which contain a YAML file called main.yml. It may read a little complicated but hopefully an image of the folder directory will help:

So the folder structure of the roles folder is:

roles/
     elastic/
            meta/
                main.yml
            tasks/
                 main.yml
     kibana/
           meta/
               main.yml
           tasks/
               main.yml
     logstash/
             meta/
                 main.yml
             tasks/
                 main.yml
     filebeat/
             meta/
                 main.yml
             tasks/
                 main.yml

If you would like to interact with the folder structure then please download a copy of the files/folder for this project from my GitHub.

There can be other sub-folders of roles including folders for files, templates and handlers. I am not currently using or creating these folders but may in future when building the roles. For further information on these other folders or about roles please check out the Ansible Documentation on Roles.

meta/main.yml

Currently meta/main.yml only contains one line:

dependencies: []

If the role has any dependencies they would be listed here. The metadata file can be expanded with further details of the role for use with both Ansible and Ansible Galaxy.

tasks/main.yml

tasks/main.yml is where all the tasks that were previously in the main playbook are. For example, the elastic role now carries out the download, install and configuration of elasticsearch.

---

- name: Download Elastic Deb Package

get_url:

url: https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.6.2-amd64.deb

dest: /tmp

- name: Install Elastic Deb Package

become: yes

apt:

deb: /tmp/elasticsearch-7.6.2-amd64.deb

- name: Update Elastic Config (IP Address to 0.0.0.0)

become: yes

lineinfile:

destfile: /etc/elasticsearch/elasticsearch.yml

regexp: 'network.host'

line: 'network.host: 0.0.0.0'

- name: Updating Elastic Config (Port Number)

become: yes

lineinfile:

destfile: /etc/elasticsearch/elasticsearch.yml

regexp: 'http.port'

line: 'http.port: 9200'

- name: Updating Elastic Config (Node Name)

become: yes

lineinfile:

destfile: /etc/elasticsearch/elasticsearch.yml

regexp: '#node.name: node-1'

line: 'node.name: node-1'




- name: Updating Elastic Config (Cluster Initial Master Nodes)

become: yes

lineinfile:

destfile: /etc/elasticsearch/elasticsearch.yml

line: 'cluster.initial_master_nodes: ["node-1"]'




- name: Start ElasticSearch Service

become: yes

service:

name: elasticsearch

state: started

A webpage is not always the best for showing YAML formatting, sometimes the formatting goes a little silly, so here is a picture of the same content:

With the ELK components broken off into roles their individual tasks become a little more manageable and easier to view/update. As the roles contain the tasks, the playbook can be updated to reference the roles rather than the long list of tasks. For my testing I have kept my original playbook.yml (from my first post) and created a new playbook called playbook_with_roles.yml for use with the roles.

The playbook has been reduced from 181 lines down to a more manageable 33 lines of YAML and it becomes clearer to see what the various hosts are running. A nice feature of using roles is that whilst the playbook is in play it displays the role name and then the task name, rather than just the task name e.g.

Without roles the playbook would display just the task:

Install Elastic Deb Package

When using roles this becomes:

elastic: Install Elastic Deb Package

This can make it easier to see where a playbook is up to or which tasks the play is currently using, especially if you have named a task generically (e.g. called the install of the various ELK components just “install” and not detailed which component was installing).