SSH public key authentication with google authenticator still asks for password

Here is my working configuration. Some users have authenticator enabled and some don't, and only SSH logins with public keys are permitted, never passwords.

In /etc/ssh/sshd_config,

UsePAM yes
PasswordAuthentication no
ChallengeResponseAuthentication yes
AuthenticationMethods publickey,keyboard-interactive
PermitEmptyPasswords no

In /etc/pam.d/sshd,

# Standard Un*x authentication.
#@include common-auth

# Require authenticator, if not configured then allow
auth    required    pam_google_authenticator.so debug nullok
auth    required    pam_permit.so

@include comon-auth must be disabled because it includes pam_unix, which I don't want to use. Then you need pam_permit to make authentication successful for users without authenticator (for which pam_google_authenticator returns ignore rather than pass).

This still doesn't let root login with an ssh key; sshd logs

sshd[1244501]: fatal: Internal error: PAM auth succeeded when it should have failed

This is discussed at Google Authenticator PAM on SSH blocks root login without 2FA .

Having gotten this working as above, I think it's actually nicer to enforce 2FA for certain groups using the SSH config as @zoredache suggested. This easily allows you to whitelist certain IPs as not requiring 2FA also. In this case, sshd_config says for example

UsePAM yes
PasswordAuthentication no
ChallengeResponseAuthentication yes
#AuthenticationMethods any # default
PermitEmptyPasswords no

Match Group adm Address *,!172.16.1.0/24
    AuthenticationMethods publickey,keyboard-interactive

and /etc/pam.d/ssh says

 Standard Un*x authentication.
#@include common-auth

# Require authenticator; SSH should not allow any user in who doesn't have it
auth       sufficient   pam_google_authenticator.so debug nullok
auth       requisite    pam_deny.so

I don't think you need or want to comment out the @include common-auth. Or at least I did not and it seemed to work correctly. But I am still mostly been testing this.

Does anyone have a recipe to make this work?

Don't have time to translate it to a shell script for you, but this is a excerpt of an ansible playbook that seems to work for me. I suspect you should be able follow along with what this is doing even if you aren't using ansible.

- hosts: linux_systems
  tasks:

  - name: Add group 'totp'
    group:
      name: totp
      state: present
      system: yes

  - name: Create directory for totp secrets
    file:
      state: directory
      path: /var/lib/google-authenticator
      owner: "0"
      group: "0"
      mode: "0700"

  - name: install libpam-google-authenticator
    apt:
      update_cache: yes
      cache_valid_time: '{{ apt_cache_valid_time | default(7200) }}'
      state: present
      name:
      - libpam-google-authenticator

  - name: Create secret for 'example-user'
    args:
      creates: /var/lib/google-authenticator/example-user
    shell: |
      TOTP_USER=example-user; \
      google-authenticator \
        --force --quiet \
        --emergency-codes=10 \
        --time-based \
        --qr-mode=none \
        --allow-reuse \
        --window-size=3 \
        --rate-limit=4 --rate-time=30 \
        --secret=/var/lib/google-authenticator/${TOTP_USER}

  - name: update pam
    lineinfile:
      insertafter: '^@include common-password'
      path: /etc/pam.d/login
      line: 'auth required pam_google_authenticator.so nullok user=root secret=/var/lib/google-authenticator/${USER}'

  - name: update pam
    lineinfile:
      insertafter: '^@include common-password'
      path: /etc/pam.d/sshd
      line: 'auth required pam_google_authenticator.so nullok user=root secret=/var/lib/google-authenticator/${USER}'

  - name: update sshd ChallengeResponseAuthentication
    notify: Restart sshd
    lineinfile:
      path: /etc/ssh/sshd_config
      regexp: '^ChallengeResponseAuthentication .*'
      line: 'ChallengeResponseAuthentication yes'

  handlers:

  - name: Restart sshd
    service:
      name: sshd
      state: restarted