Ping a Specific Port

Question

Konstantin

Asked: 2024-03-22 00:50:43 +0800 CST2024-03-22 00:50:43 +0800 CST 2024-03-22 00:50:43 +0800 CST

Creating a configuration file for Prometheus

772

Inventory file (inventories/test/environments/tmp6.yml):

test:
  children:
    environments:
      children:
        tmp6:

tmp6:
  vars:
    ansible_user: superuser
    ansible_password: superuser
    ansible_become_password: "{{ ansible_password }}"
  children:
    infra:
      hosts:
        host_5:
          ansible_host: 192.168.1.72
        host_6:
          ansible_host: 192.168.1.83
    tools:
      hosts:
        host_7:
          ansible_host: 192.168.1.239
        host_8:
          ansible_host: 192.168.1.46

General playbook.yml:

- hosts: environments
  roles:
    - role: host-configuration
    - role: host-users
    - role: node-exporter
  post_tasks:
    - ansible.builtin.copy:
        content: |
          - target: 
              - {{ ansible_host }}:9100
            labels:
              environment_name: {{ inventory_file | ansible.builtin.basename | ansible.builtin.regex_replace('.yml$', '') }}
              environment_type: {{ inventory_file | ansible.builtin.dirname | ansible.builtin.dirname | ansible.builtin.basename }}
        dest: "./{{ inventory_file | ansible.builtin.basename }}"
      delegate_to: localhost

After successfully installing the node_exporter on the nodes, need to create a common configuration file for the environment:

- target:
    - 192.168.1.72:9100
    - 192.168.1.83:9100
    - 192.168.1.239:9100
    - 192.168.1.46:9100
  labels:
    environment_name: tmp6
    environment_type: test

The main problem in formation the list of targets.

If create such a template:

- target: 
    - {{ ansible_host }}:9100
  labels:
    environment_name: {{ inventory_file | ansible.builtin.basename | ansible.builtin.regex_replace('.yml$', '') }}
    environment_type: {{ inventory_file | ansible.builtin.dirname | ansible.builtin.dirname | ansible.builtin.basename }}

value of the variable ansible_host always changes. Therefore it is not a solution.

How to create a dynamic host list in a playbook?

P.S.: Playbook can be applied to multiple environments:

ansible-playbook playbook.yml -i inventories/test -l tmp1,tmp3,tmp6

General task:

install the node_exporter on the nodes of group "environments"
create a common configuration files on the localhost
copy the configuration files to the host "prometheus"

2 Answers

Voted

larsks · Answer 1 · 2024-03-22T09:15:48+08:00

If multiple environments are used, multiple files must be created

Ansible doesn't know anything about your "environments". If you want to use that information in your playbooks, you'll need to add it to your inventory. For example, we can add environment_type and environment_name variables like this:

test:
  vars:
    environment_type: test
  children:
    environments:
      children:
        tmp5:
        tmp6:

tmp5:
  vars:
    environment_name: tmp5
  children:
    tmp5_infra:
      hosts:
        host_1:
          ansible_host: 192.168.1.1
        host_2:
          ansible_host: 192.168.1.2
    tmp5_tools:
      hosts:
        host_3:
          ansible_host: 192.168.1.3
        host_4:
          ansible_host: 192.168.1.4

tmp6:
  vars:
    environment_name: tmp6
  children:
    tmp6_infra:
      hosts:
        host_5:
          ansible_host: 192.168.1.5
        host_6:
          ansible_host: 192.168.1.6
    tmp6_tools:
      hosts:
        host_7:
          ansible_host: 192.168.1.7
        host_8:
          ansible_host: 192.168.1.8

Given the above inventory file, we can write a playbook like this:

- hosts: environments
  gather_facts: false
  tasks:

    # This task groups hosts by environment_name
    - delegate_to: localhost
      run_once: true
      loop: "{{ ansible_play_hosts_all }}"
      set_fact:
        env_to_host: >
          {{ env_to_host|combine({
            hostvars[item].environment_name: env_to_host.get(hostvars[item].environment_name, []) + [item]
          })}}
      vars:
        env_to_host: {}

    - delegate_to: localhost
      run_once: true
      loop: "{{ env_to_host.keys() }}"
      copy:
        content: |
          - target: 
          {% for host in env_to_host[item] %}
              - {{ hostvars[host].ansible_host }}:9100
          {% endfor %}
            labels:
              environment_name: {{ item }}
              environment_type: {{ hostvars[env_to_host[item]|first].environment_type }}
        dest: "./config-{{item}}.yaml"

If we run:

ansible-playbook playbook.yaml

We get:

$ ls config*
config-tmp5.yaml  config-tmp6.yaml
$ cat config-tmp5.yaml
- target:
    - 192.168.1.1:9100
    - 192.168.1.2:9100
    - 192.168.1.3:9100
    - 192.168.1.4:9100
  labels:
    environment_name: tmp5
    environment_type: test
$ cat config-tmp6.yaml
- target:
    - 192.168.1.5:9100
    - 192.168.1.6:9100
    - 192.168.1.7:9100
    - 192.168.1.8:9100
  labels:
    environment_name: tmp6
    environment_type: test

If we limit by environment, like this:

ansible-playbook playbook.yaml -l tmp5

We get:

$ ls config*
config-tmp5.yaml
$ cat config-tmp5.yaml
- target:
    - 192.168.1.1:9100
    - 192.168.1.2:9100
    - 192.168.1.3:9100
    - 192.168.1.4:9100
  labels:
    environment_name: tmp5
    environment_type: test

And finally, if we limit to individual hosts, like this:

ansible-playbook playbook.yaml -l host_1,host_5

We get:

$ ls config*
config-tmp5.yaml  config-tmp6.yaml
$ cat config-tmp5.yaml
- target:
    - 192.168.1.1:9100
  labels:
    environment_name: tmp5
    environment_type: test
$ cat config-tmp6.yaml
- target:
    - 192.168.1.5:9100
  labels:
    environment_name: tmp6
    environment_type: test

Konstantin · Answer 2 · 2024-03-23T00:42:23+08:00

The problem can be solved with intermediate temporary files.

Creating temporary files:

- ansible.builtin.file:
    path: "./.configs_prometheus/{{
           inventory_file | ansible.builtin.dirname | ansible.builtin.dirname | ansible.builtin.basename }}_--_{{
           ansible_host }}_--_{{ inventory_file | ansible.builtin.basename | ansible.builtin.regex_replace('.yml$', '') }}"
    state: touch

Search for temporary files

- ansible.builtin.find:
    paths: "./.configs_prometheus/"
    file_type: file
    recurse: yes
  register: environment_temp_files_list

Generating data lists:

- ansible.builtin.set_fact:
    environment_data: "{{ environment_temp_files_list.files | map(attribute='path') | map('ansible.builtin.basename') |
                          map('ansible.builtin.split','_--_') | map('ansible.builtin.zip', ['type', 'host', 'name']) |
                          map('map', 'reverse') | map('community.general.dict') | groupby('name') }}"

- ansible.builtin.set_fact: environments_list="{{ environment_data | map('first') }}"

- ansible.builtin.set_fact: hosts_list="{{ environment_data | map('last') | map('map', attribute='host') }}"

- ansible.builtin.set_fact: types_list="{{ environment_data | map('last') | map('map', attribute='type') | map('ansible.builtin.unique') }}"

Template generation:

- ansible.builtin.copy:
    content: |
      - targets:
        {%  for result in item.1.0 %}
        - {{ result }}:9100
        {% endfor %}
      labels:
          environment_name: {{ item.0 }}
          environment_type: {{ item.1.1.0 }}
    dest: "./.configs_prometheus/{{ item.1.1.0 }}_--_{{ item.0 }}.yml"
  loop: "{{ environments_list | zip( hosts_list | zip(types_list) ) }}"
  loop_control:
    label: "{{ item.0 }}"

Final playbook:

- hosts: localhost
  tasks:
    - name: Creating a directory
      ansible.builtin.file:
        path: "./.configs_prometheus"
        state: "{{ item }}"
      delegate_to: localhost
      loop:
        - absent
        - directory

- hosts: environments
  roles:
    - role: host-configuration
    - role: host-users
    - role: node-exporter
  post_tasks:
    - name: Creating temporary files
      ansible.builtin.file:
        path: "./.configs_prometheus/{{
               inventory_file | ansible.builtin.dirname | ansible.builtin.dirname | ansible.builtin.basename }}_--_{{
               ansible_host }}_--_{{ inventory_file | ansible.builtin.basename | ansible.builtin.regex_replace('.yml$', '') }}"
        state: touch
      delegate_to: localhost

- hosts: localhost
  tasks:
    - name: Search for temporary files
      ansible.builtin.find:
        paths: "./.configs_prometheus/"
        file_type: file
        recurse: yes
      register: environment_temp_files_list

    - name: Generating data lists
      ansible.builtin.set_fact:
        environment_data: "{{ environment_temp_files_list.files | map(attribute='path') | map('ansible.builtin.basename') |
                              map('ansible.builtin.split','_--_') | map('ansible.builtin.zip', ['type', 'host', 'name']) |
                              map('map', 'reverse') | map('community.general.dict') | groupby('name') }}"
    - ansible.builtin.set_fact: environments_list="{{ environment_data | map('first') }}"
    - ansible.builtin.set_fact: hosts_list="{{ environment_data | map('last') | map('map', attribute='host') }}"
    - ansible.builtin.set_fact: types_list="{{ environment_data | map('last') | map('map', attribute='type') | map('ansible.builtin.unique') }}"

    - name: Template generation
      ansible.builtin.copy:
        content: |
          - targets:
            {%  for result in item.1.0 %}
            - {{ result }}:9100
            {% endfor %}
          labels:
              environment_name: {{ item.0 }}
              environment_type: {{ item.1.1.0 }}
        dest: "./.configs_prometheus/{{ item.1.1.0 }}_--_{{ item.0 }}.yml"
      loop: "{{ environments_list | zip( hosts_list | zip(types_list) ) }}"
      loop_control:
        label: "{{ item.0 }}"

    - name: Deleting temporary files
      ansible.builtin.file:
        path: "{{ item }}"
        state: absent
      loop: "{{ environment_temp_files_list.files | map(attribute='path') }}"
      loop_control:
        label: "{{ item.split('/').1 }}"

UPDATE

The problem can be solved without intermediate temporary files.

Creating environment dataset:

- ansible.builtin.set_fact:
    environment_dataset: |
      {% for play_host in ansible_play_hosts_all %}
        - {{ hostvars[play_host].inventory_file | ansible.builtin.dirname | ansible.builtin.dirname | ansible.builtin.basename }}_--_{{
             hostvars[play_host].ansible_host }}_--_{{
             hostvars[play_host].inventory_file | ansible.builtin.basename | ansible.builtin.regex_replace('.yml$', '')
          }}
      {% endfor %}
  run_once: true
  delegate_to: "{{ item }}"
  delegate_facts: true
  loop: "{{ groups['prometheus'] | default(['localhost']) }}"

Generating data lists:

- ansible.builtin.set_fact:
    environment_data: "{{ environment_dataset | from_yaml |
                          map('ansible.builtin.split','_--_') | map('ansible.builtin.zip', ['type', 'host', 'name']) |
                          map('map', 'reverse') | map('community.general.dict') | groupby('name') }}"

- ansible.builtin.set_fact: environments_list="{{ environment_data | map('first') }}"

- ansible.builtin.set_fact: hosts_list="{{ environment_data | map('last') | map('map', attribute='host') }}"

- ansible.builtin.set_fact: types_list="{{ environment_data | map('last') | map('map', attribute='type') | map('ansible.builtin.unique') }}"

Template generation:

- ansible.builtin.copy:
    content: |
      - targets:
        {%  for result in item.1.0 %}
        - {{ result }}:9100
        {% endfor %}
      labels:
          environment_name: {{ item.0 }}
          environment_type: {{ item.1.1.0 }}
    dest: "/etc/prometheus/file_sd_configs/{{ item.1.1.0 }}_--_{{ item.0 }}.yml"
  loop: "{{ environments_list | zip( hosts_list | zip(types_list) ) }}"
  loop_control:
    label: "{{ item.0 }}"

Final playbook:

- hosts: environments
  roles:
    - role: host-configuration
    - role: host-users
    - role: node-exporter
  post_tasks:
    - name: Creating temporary files
      ansible.builtin.set_fact:
        environment_dataset: |
          {% for play_host in ansible_play_hosts_all %}
            - {{ hostvars[play_host].inventory_file | ansible.builtin.dirname | ansible.builtin.dirname | ansible.builtin.basename }}_--_{{
                 hostvars[play_host].ansible_host }}_--_{{
                 hostvars[play_host].inventory_file | ansible.builtin.basename | ansible.builtin.regex_replace('.yml$', '')
              }}
          {% endfor %}
      run_once: true
      delegate_to: "{{ item }}"
      delegate_facts: true
      loop: "{{ groups['prometheus'] | default(['localhost']) }}"

- hosts: prometheus
  tasks:
    - name: Generating data lists
      ansible.builtin.set_fact:
        environment_data: "{{ environment_dataset | from_yaml |
                              map('ansible.builtin.split','_--_') | map('ansible.builtin.zip', ['type', 'host', 'name']) |
                              map('map', 'reverse') | map('community.general.dict') | groupby('name') }}"
    - ansible.builtin.set_fact: environments_list="{{ environment_data | map('first') }}"
    - ansible.builtin.set_fact: hosts_list="{{ environment_data | map('last') | map('map', attribute='host') }}"
    - ansible.builtin.set_fact: types_list="{{ environment_data | map('last') | map('map', attribute='type') | map('ansible.builtin.unique') }}"

    - name: Template generation
      ansible.builtin.copy:
        content: |
          - targets:
            {%  for result in item.1.0 %}
            - {{ result }}:9100
            {% endfor %}
          labels:
              environment_name: {{ item.0 }}
              environment_type: {{ item.1.1.0 }}
        dest: "/etc/prometheus/file_sd_configs/{{ item.1.1.0 }}_--_{{ item.0 }}.yml"
      become: yes
      loop: "{{ environments_list | zip( hosts_list | zip(types_list) ) }}"
      loop_control:
        label: "{{ item.0 }}"

Creating a configuration file for Prometheus

Can you pass user/pass for HTTP Basic Authentication in URL parameters?

Ping a Specific Port

Check if port is open or closed on a Linux server?

How to automate SSH login with password?

How do I tell Git for Windows where to find my private RSA key?

What's the default superuser username/password for postgres after a new install?

What port does SFTP use?

Command line to list users in a Windows Active Directory group?

What is a Pem file and how does it differ from other OpenSSL Generated Key File Formats?

How to determine if a bash variable is empty?