Ansible: Can I use vars_files when some files do not exist

It's quite simple really. You can squash your different vars_files items into a single tuple and Ansible will automatically go through each one until it finds a file that exists and load it. E.x.:

vars_files:
  - [ "vars/foo.yml", "vars/bar.yml", "vars/default.yml" ]

According to Ansible developers, the proper way to solve this is to use something like:

vars_files_locs: ['../path/to/file1', '../path/to/file2', ...]

- include_vars: "{{ item }}"
  with_first_found: vars_files_locs

Furthermore, they say:

The above will properly load only the first file found, and is more flexible than trying to do this via the vars_files language keyword.

- hosts: all
  vars_files: vars/vars.default.yml
  vars:
    optional_vars_file: "{{ lookup('first_found', 'vars/vars.yml', errors='ignore') }}"
  tasks:
  - when: optional_vars_file is file
    include_vars: "{{ optional_vars_file }}"

Note: The path tests (is file, is exists,...) work only with absolute paths or paths relative to the current working directory when running ansible-playbook command. This is the reason we used the lookup. the lookup accepts paths relative to the playbook directory and returns the absolute path when the file exists.

I encountered this problem in a setup where I needed to create multiple deployment environments (live, demo, sandbox) to the same physical server (not allowed virtual machines here), and then a script to deploy arbitrary svn repos

This required a directory tree of (optional) variable.yml files, that would merge ontop of each other and not throw an exception if any where missing

Start by enabling variable merging in ansible - note that this does shallow hash merging (1 level deep) and not fully recursive deep merge

ansible.cfg

[defaults]
hash_behaviour=merge ;; merge rather than replace dictionaries http://docs.ansible.com/ansible/intro_configuration.html###hash-behaviour

Ansible directory layout

/group_vars
└── all.yml

/playbooks
├── boostrap.yml
├── demo.yml
├── live.yml
└── sandbox.yml

/roles/deploy/
├── files
├── tasks
│   ├── includes.yml
│   ├── main.yml
└── vars
    ├── main.yml
    ├── project_1.yml
    ├── project_2.yml
    ├── demo
    │   ├── project_1.yml
    │   ├── project_2.yml   
    │   └── main.yml
    ├── live
    │   ├── project_1.yml
    │   ├── project_2.yml   
    │   └── main.yml
    └── sandbox
        ├── project_1.yml
        ├── project_2.yml   
        └── main.yml

roles/deploy/tasks/includes.yml

This is the main logic for a directory tree of optional variable files.

;; imports in this order:
;; - /roles/deploy/vars/main.yml
;; - /roles/deploy/vars/{{ project_name }}.yml
;; - /roles/deploy/vars/{{ project_name }}/main.yml
;; - /roles/deploy/vars/{{ project_name }}/{{ project_env }}.yml
- include_vars:
    dir: 'vars'
    files_matching: "{{ item }}"
    depth: 1
  with_items:
    - "main.yml"
    - "{{ project_name }}.yml"

- include_vars:
    dir: 'vars/{{ env_name }}'
    files_matching: "{{ item }}"
    depth: 1
  with_items:
    - "main.yml"
    - "{{ project_name }}.yml"

group_vars/all.yml

Configure default variables for the project and various users and environments

project_users:
    bootstrap:
        env:   bootstrap
        user:  ansible
        group: ansible
        mode:  755
        root:  /cs/ansible/
        home:  /cs/ansible/home/ansible/
        directories:
            - /cs/ansible/
            - /cs/ansible/home/

    live:
        env:   live
        user:  ansible-live
        group: ansible
        mode:  755
        root:  /cs/ansible/live/
        home:  /cs/ansible/home/ansible-live/

    demo:
        env:   demo
        user:  ansible-demo
        group: ansible
        mode:  755
        root:  /cs/ansible/demo/
        home:  /cs/ansible/home/ansible-demo/

    sandbox:
        env:   sandbox
        user:  ansible-sandbox
        group: ansible
        mode:  755
        root:  /cs/ansible/sandbox/
        home:  /cs/ansible/home/ansible-sandbox/    

project_env:  bootstrap
project_user: "{{ ansible_users[project_env] }}" ;; this will be retroactively updated if project_env is redefined later

roles/deploy/vars/main.yml

project defaults

ansible_project:
  node_env:   development
  node_port:  4200
  nginx_port: 4400

roles/deploy/vars/project_1.yml

defaults for project_1

ansible_project:
  node_port:  4201
  nginx_port: 4401

roles/deploy/vars/live/main.yml

defaults for live environment, overrides project defaults

ansible_project:
  node_env: production

roles/deploy/vars/live/project_1.yml

final overrides for project_1 in the live environment

ansible_project:
  nginx_port: 80

playbooks/demo.yml

Configure separate playbooks for each environment

- hosts: shared_server
  remote_user: ansible-demo
  vars:
    project_env: demo
  pre_tasks:
    - debug: "msg='{{ facter_gid }}@{{ facter_fqdn }} ({{ server_pseudonym }})'"
    - debug: var=project_ssh_user
  roles:
    - { role: deploy, project_name: project_1 }

WARNING: Because all environments live on a single host, all playbooks must be run individually, otherwise Ansible will brokenly attempt to run all scripts as the first ssh login user and only use the variables for the first user. If you need to run all scripts sequentially, then use xargs to run them each as separate commands.

find ./playbooks/*.yml | xargs -L1 time ansible-playbook

New answer based on the latest Ansible versions—basically, you should use with_first_found, along with skip: true to skip the task if no file is found.

- name: Include vars file if one exists meeting our condition.
  include_vars: "{{ item }}"
  with_first_found:
    - files:
        - vars/{{ variable_here }}.yml
      skip: true

This makes it so you don't have to have a fallback vars file in that list.

See related: https://stackoverflow.com/a/39544405/100134

Or in a more yaml way:

- hosts: webservers
  vars:
    paths_to_vars_files:
      - vars/{{ ansible_hostname }}.yml
      - vars/default.yml
  tasks:
    - include_vars: "{{ item }}"
      with_first_found: "{{ paths_to_vars_files }}"

That is, instead of writing an array on one line with square brackets, like:

['path/to/file1', 'path/to/file2', ...]

Use the yaml way of writing array values on multiple lines, like:

- path/to/file1
- path/to/file2

As mentioned this looks for a vars file named {{ ansible_hostname }}.yml, and if it doesn't exist uses default.yml

Piecing various pieces together... include_vars with a when clause that is true when the file exists. i.e.

vars:
  file_to_include: /path/to/file
tasks:
  - include_vars: "{{ file_to_include }}"
    when: file_to_include is exists