Getting started

Upstream package is used by default

The debops.metricbeat role depends on the debops.extrepo Ansible role to configure access to the Elastic.co APT repository. This means that usually the latest available Metricbeat release will be installed by default. If you require older releases, you can use the debops.apt_preferences role to select the desired package version.

Alternatively, you can "mask" the metricbeat__extrepo__dependent_sources variable in the Ansible inventory and configure the APT repositories yourself via the debops.apt role. You can also use the debops.elastic_co role to configure Elastic APT repositories specifically.

Default configuration

The metricbeat role includes a basic configuration for Metricbeat which will by default enable metrics in the system module.

The default configuration does not try to connect to any existing Elasticsearch cluster. You should finish the configuration in the Ansible inventory to tailor the role to your own environment. An example configuration for an Elasticsearch + Kibana environment is provided below:

---

# Example configuration for Metricbeat deployment with an Elasticsearch cluster
# as the data destination. Kibana is used as the UI layer.

---
# Configuration for all Metricbeat instances
# File: ansible/inventory/group_vars/debops_all_hosts/metricbeat.yml
metricbeat__configuration:

  - name: 'output_elasticsearch'
    config:
      output.elasticsearch:
        hosts:
          - 'https://es1.example.org:9200'
          - 'https://es2.example.org:9200'
          - 'https://es3.example.org:9200'
        username: 'remote_monitoring_user'
        password: '${REMOTE_USER_PASSWORD}'

  - name: 'setup_kibana'
    config:
      setup.kibana:
        host: "https://kibana.example.org:443"
        # Elasticsearch account which can create Kibana dashboards
        username: 'elastic'
        password: '{{ lookup("file", secret + "/elasticsearch/credentials/"
                      + "built-in/elastic/password") }}'

  - name: 'setup_ilm'
    config:
      setup.ilm.overwrite: true

  - name: 'monitoring'
    config:
      monitoring:
        enabled: true
        elasticsearch:
          username: 'remote_monitoring_user'
          password: '${REMOTE_USER_PASSWORD}'

# Store the Metricbeat user password in Metricbeat keystore. The Elasticsearch
# cluster should be deployed first to ensure that the passwords are present.
metricbeat__keys:

  - RMOTE_USER_PASSWORD: '{{ lookup("file", secret + "/elasticsearch/credentials/"
                             + "built-in/remote_monitoring_user/password") }}'


---
# Configuration for Elasticsearch nodes
# File: ansible/inventory/group_vars/debops_service_elasticsearch/metricbeat.yml
metricbeat__group_snippets:

  - name: 'modules.d/elasticsearch-xpack.yml'
    config:

      - module: 'elasticsearch'
        hosts: [ 'https://{{ ansible_fqdn }}:9200' ]
        username: 'remote_monitoring_user'
        password: '${REMOTE_USER_PASSWORD}'
        xpack.enabled: True
        period: '10s'


---
# Configuration for Kibana nodes
# File: ansible/inventory/group_vars/debops_service_kibana/metricbeat.yml
metricbeat__group_snippets:

  - name: 'modules.d/kibana-xpack.yml'
    config:

      - module: 'kibana'
        hosts: [ 'localhost:5601' ]
        username: 'remote_monitoring_user'
        password: '${REMOTE_USER_PASSWORD}'
        xpack.enabled: True
        period: '10s'

For ease of use, the examples are provided in the debops.metricbeat role documentation in the DebOps monorepo. Elasticsearch and Kibana services can be configured using the debops.elasticsearch and debops.kibana roles, respectively.

Check the Metricbeat configuration upstream documentation to learn more about configuring Metricbeat for your environment and requirements.

Role debugging tips

Most of the configuration files generated by the role are protected by the Ansible no_log keyword. To make debugging easier, you can use the debops__no_log variable in an inventory (in the development environment) or with Ansible --extra-vars parameter (one time, for production environment) to disable log protection. See the variable documentation for more details.

Example inventory

To install and configure Metricbeat on a host, it needs to be included in the [debops_service_metricbeat] Ansible inventory group:

[debops_all_hosts]
hostname

[debops_service_metricbeat]
hostname

Fully-fledged Elastic stack can be deployed like this:

[debops_all_hosts]
es1     ansible_host=es1.example.org
es2     ansible_host=es2.example.org
es3     ansible_host=es3.example.org
kibana  ansible_host=kibana.example.org

[debops_service_elasticsearch]
es1
es2
es3

[debops_service_kibana]
kibana

[debops_service_metricbeat:children]
debops_service_elasticsearch
debops_service_kibana

Example playbook

If you are using this role without DebOps, here's an example Ansible playbook that uses the debops.metricbeat role:

---

- name: Manage Metricbeat service
  collections: [ 'debops.debops', 'debops.roles01',
                 'debops.roles02', 'debops.roles03' ]
  hosts: [ 'debops_service_metricbeat' ]
  become: True

  environment: '{{ inventory__environment | d({})
                   | combine(inventory__group_environment | d({}))
                   | combine(inventory__host_environment  | d({})) }}'

  roles:

    - role: extrepo
      tags: [ 'role::extrepo', 'skip::extrepo', 'role::metricbeat' ]
      extrepo__dependent_sources:
        - '{{ metricbeat__extrepo__dependent_sources }}'

    - role: metricbeat
      tags: [ 'role::metricbeat', 'skip::metricbeat' ]

Ansible tags

You can use Ansible --tags or --skip-tags parameters to limit what tasks are performed during Ansible run. This can be used after host is first configured to speed up playbook execution, when you are sure that most of the configuration has not been changed.

Available role tags:

role::metricbeat

Main role tag, should be used in the playbook to execute all of the role tasks as well as role dependencies.