Getting started

Upstream package is used by default

The debops.elasticsearch role depends on the debops.extrepo Ansible role to configure access to the Elastic.co APT repository. This means that usually the latest available Elasticsearch release will be installed by default. If you require older releases, you can use the debops.apt_preferences role to select the desired package version.

Alternatively, you can "mask" the elasticsearch__extrepo__dependent_sources variable in the Ansible inventory and configure the APT repositories yourself via the debops.apt role.

Elasticsearch is insecure in standalone mode

The Elasticsearch service in the default configuration supports only plaintext connections between the standalone host and clients. There's no client authentication or authorization policies as well. Due to that you should take care to not expose your standalone Elasticsearch service to the outside world without proper encryption and authentication.

In cluster mode, and with debops.pki role configured on the host, the debops.elasticsearch role will automatically enable the X-Pack plugin and configure TLS encryption for HTTP clients and Transport communication between cluster nodes. This is possible due to changes in the Elastic licensing (the default installation comes with the Basic subscription which provides support for TLS and user/group management via the X-Pack plugin).

With secure cluster communication over TLS, the debops.elasticsearch role can use the Elasticsearch API to manage user accounts and role definitions in the cluster. A default set of built-in users will be created automatically; passwords of these users will be stored in the secret/elasticsearch/credentials/built-in/ directory on the Ansible Controller (managed by the debops.secret role). After that, with the base URL of the Elasticsearch API set in the elasticsearch__api_base_url variable, the role can manage Elasticsearch roles and user accounts using the elastic superuser account.

You can install additional plugins that provide encrypted connections, authentication, authorization and access control:

• X-Pack, an Elastic commercial plugin with free trial period. Supports encryption, authentication, access control, integrates with Kibana and Logstash. Since the plugin is developed by the same team, its releases are in parallel with Elasticsearch.
• Search Guard, an open source third party plugin with commercial support. Has features comparable with X-Pack, with more basic features like HTTP and transport encryption, basic authentication and access control available free of charge.
• ReadonlyREST, an open source security plugin focused on the Elasticsearch HTTP REST interface security.

Standalone deployment or cluster

With the default configuration, the debops.elasticsearch role will deploy the Elasticsearch service in a "standalone" mode without exposing the service to the outside world. This allows easy deployments for development or testing purposes.

In production environment, you are strongly advised to deploy Elasticsearch on at least three (3) nodes to ensure consistency in the cluster. To enable cluster mode, you will need to configure the firewall for Elasticsearch and use one or multiple Ansible inventory groups to design your cluster architecture.

The role currently does not support deployment of multiple Elasticsearch instances on one host. As an alternative, consider setting up an internal container environment with each Elasticsearch instance in a separate container with its own IP address or use Elastic Cloud Elasticsearch Service which does this for you and more.

See the Elasticsearch clustering for more details.

Use as a role dependency

The Elasticsearch main configuration file does not support an include statement or conf.d directory. To mitigate that and allow multiple Elasticsearch configuration sources from other Ansible roles, the debops.elasticsearch role supports operation as a dependent role. This functionality can be used by other Ansible roles to better manage Elasticsearch plugins or extend the cluster configuration without the need to implement the entire role again and with preserved idempotency.

See the Usage as a role dependency for more details.

Example inventory

To deploy Elasticsearch in a standalone mode, you can add the host to the [debops_service_elasticsearch] Ansible inventory group:

[debops_service_elasticsearch]
hostname

The default playbook supports use of different Ansible inventory groups for different types of Elasticsearch nodes. See Elasticsearch clustering for more details.

Example playbook

If you are using this role without DebOps, here's an example Ansible playbook that uses the debops.elasticsearch role:

---

- name: Manage Elasticsearch cluster
  collections: [ 'debops.debops', 'debops.roles01',
                 'debops.roles02', 'debops.roles03' ]
  hosts: [ 'debops_service_elasticsearch',
           'debops_service_elasticsearch_master',
           'debops_service_elasticsearch_data',
           'debops_service_elasticsearch_ingest',
           'debops_service_elasticsearch_lb' ]
  become: True

  environment: '{{ inventory__environment | d({})
                   | combine(inventory__group_environment | d({}))
                   | combine(inventory__host_environment  | d({})) }}'

  pre_tasks:

    - name: Prepare elasticsearch environment
      import_role:
        name: 'elasticsearch'
        tasks_from: 'main_env'
      tags: [ 'role::elasticsearch', 'role::secret', 'role::elasticsearch:config' ]

  roles:

    - role: extrepo
      tags: [ 'role::extrepo', 'skip::extrepo', 'role::elasticsearch' ]
      extrepo__dependent_sources:
        - '{{ elasticsearch__extrepo__dependent_sources }}'

    - role: secret
      tags: [ 'role::secret', 'role::elasticsearch', 'role::elasticsearch:config' ]
      secret__directories:
        - '{{ elasticsearch__secret__directories }}'

    - role: apt_preferences
      tags: [ 'role::apt_preferences', 'skip::apt_preferences' ]
      apt_preferences__dependent_list:
        - '{{ java__apt_preferences__dependent_list }}'

    - role: etc_services
      tags: [ 'role::etc_services', 'skip::etc_services' ]
      etc_services__dependent_list:
        - '{{ elasticsearch__etc_services__dependent_list }}'

    - role: sysctl
      tags: [ 'role::sysctl', 'skip::sysctl' ]
      sysctl__dependent_parameters:
        - '{{ elasticsearch__sysctl__dependent_parameters }}'

    - role: ferm
      tags: [ 'role::ferm', 'skip::ferm' ]
      ferm__dependent_rules:
        - '{{ elasticsearch__ferm__dependent_rules }}'

    - role: java
      tags: [ 'role::java', 'skip::java' ]

    - role: elasticsearch
      tags: [ 'role::elasticsearch', 'skip::elasticsearch' ]

Ansible tags

You can use Ansible --tags or --skip-tags parameters to limit what tasks are performed during Ansible run. This can be used after a host was first configured to speed up playbook execution, when you are sure that most of the configuration is already in the desired state.

Available role tags:

role::elasticsearch

Main role tag, should be used in the playbook to execute all of the role tasks as well as role dependencies.

role::elasticsearch:config

Generate the Elasticsearch configuration taking into account different configuration sources.