Test if UFW firewall change would prevent me from reconnecting

Not built into with the ufw module. And the changes you are making will be enforced at the next reboot or firewall reload.

What you can do is reload the firewall, then test a new connection to the SSH port. If that fails, reset ufw via the persistent connection still open.

I have an implementation of this boringly called ansible-role-ufw. Note in particular the use of wait_for, as wait_for_connection will use the persistent connection and not detect the failure.

Beware that this has one shot to work. You still need remote console access for when SSH is broken.

Apply the rules manually, without saving them, but before you do, schedule a reboot or a reset of the current rules after a couple of minutes. So, if the new rule would cause any harm, it'll only be for those couple of minutes.

This is what I ended up with, extending John Mahowald's code:

roles/set_firewall_rules/tasks/main.yml

# Apply all the requested firewall rules, then try to establish a new SSH connection to the host.
# If that SSH connection fails then reset the firewall, so the user is not locked out of the machine!

# Make sure the SSH connection details figured out by target_ssh_info can actually be used to connect before the change.
# If they're not we'd end up resetting the firewall after ANY change.
- name: Try to SSH before updating firewall
  become: no
  wait_for:
    host: "{{ target_ssh_host }}"
    port: "{{ target_ssh_port }}"
    search_regex: SSH
    timeout: 5
    msg: "Failed to connect to {{ target_ssh_host }}:{{ target_ssh_port }} before firewall rule change"
  connection: local

- name: Set firewall rules
  ufw:
    src: "{{ item.src }}"
    port: "{{ item.port }}"
    proto: "{{ item.proto }}"
    rule: "{{ item.rule }}"
    comment: "{{ item.comment }}"
  register: firewall_rules
  loop: "{{ rules }}"

# Enable/reload the firewall as a separate task, after all rules have been added, so that the order of rules doesn't matter, i.e. we're not locked out
# if a deny rule comes before an allow rule (as it should).
- name: Enable and reload firewall
  ufw:
    state: enabled
  register: firewall_enabled

- name: Try to SSH after updating firewall
  become: no
  # wait_for is key here: it establishes a new connection, while wait_for_connection would re-use the existing one
  wait_for:
    host: "{{ target_ssh_host }}"
    port: "{{ target_ssh_port }}"
    search_regex: SSH
    timeout: 5
    msg: "Failed to connect to {{ target_ssh_host }}:{{ target_ssh_port }} after firewall rule change, trying to reset ufw"
  when: firewall_rules.changed or firewall_enabled.changed
  connection: local
  ignore_errors: yes
  register: ssh_after_ufw_change

# Reset the firewall if the new connection failed above. This works (mostly!), because it uses the existing connection
- name: Reset firewall if unable to SSH
  ufw:
    state: reset
  when:
    - firewall_rules.changed or firewall_enabled.changed
    - ssh_after_ufw_change.failed

# Stop the playbook - the host is now open to the world (firewall is off), which the user really needs to fix ASAP.
# It's probably better than being locked out of it, though!
- name: Fail if unable to SSH after firewall change
  fail:
    msg: "Locked out of SSH after firewall rule changes - firewall was reset"
  when:
    - firewall_rules.changed or firewall_enabled.changed
    - ssh_after_ufw_change.failed

roles/set_firewall_rules/meta/main.yml

---
dependencies:
- { role: target_ssh_info }

roles/target_ssh_info/tasks/main.yml

# Set target_ssh_host and target_ssh_port facts to the real hostname and port SSH uses to connect.

# ansible_host and ansible_port can be set at the host level to define what Ansible passes to ssh, but ssh then looks up ansible_host in ~/.ssh/config.
# This role figure out the real hostname it then connects to - useful for establishing a non-SSH connection to the same host.
# ansible_port is similar, but a little different: if set it overrides the value in ~/.ssh/config.

- name: Get hostname from local SSH config
  shell: "ssh -G '{{ ansible_host | default(inventory_hostname) }}' | awk '/^hostname / { print $2 }'"
  connection: local
  become: no
  register: ssh_host
  changed_when: false

- name: Get port from local SSH config
  shell: "ssh -G '{{ ansible_host | default(inventory_hostname) }}' | awk '/^port / { print $2 }'"
  connection: local
  become: no
  register: ssh_port
  changed_when: false
  when: ansible_port is not defined

# ansible_port overrides whatever is set in .ssh/config
- name: Set target SSH host and port
  set_fact:
    target_ssh_host: "{{ ssh_host.stdout }}"
    target_ssh_port: "{{ ansible_port | default (ssh_port.stdout) }}"

Note that ssh -G returns the hostname and port even if they are not overridden in .ssh/config, i.e. ssh -G arbitrarystring just returns "arbitrarystring" as the hostname and 22 as the port.