pam_tally2 is counting successful logins as failures while using Yubikey

I'm using Yubikey's U2F feature as a second factor to login on Ubuntu 20.04 as described here: https://support.yubico.com/support/solutions/articles/15000011356-ubuntu-linux-login-guide-u2f

However, I would like to add an extra layer of security and lock users (including me as root) who fail to enter the correct password 10 times in a row. I added in /etc/pam.d/system-auth:

auth     required       pam_tally2.so deny=10 even_deny_root unlock_time=3600

To check that it's working I performed a sudo echo test which includes entering my password and authenticating with my Yubikey as a second factor. Then ran:

sudo pam_tally2 -u user

Which gave an output something like:

Login           Failures Latest   failure     From
user            1        01/01/01 11:00:00    tty1

Continued testing and found out that every successfull attempt is being logged as a failure by pam_tally2 and I do believe it is because of the Yubikey.

Is there a way to achieve my goal to lock the user who fails entering his credentials on lock screen while preserving my Yubikey as a two factor? Thanks.