It writes this in logs:
=INFO REPORT==== 2012-03-14 17:48:54 ===
I(<0.467.0>:ejabberd_listener:281) : (#Port<0.4384>) Accepted connection {{10,254,239,2},51986} -> {{10,254,239,1},5222}
=INFO REPORT==== 2012-03-14 17:48:54 ===
I(<0.1308.0>:ejabberd_c2s:784) : ({socket_state,tls,{tlssock,#Port<0.4384>,#Port<0.4386>},<0.1307.0>}) Failed authentication for USERNAME
=INFO REPORT==== 2012-03-14 17:48:54 ===
I(<0.1308.0>:ejabberd_c2s:649) : ({socket_state,tls,{tlssock,#Port<0.4384>,#Port<0.4386>},<0.1307.0>}) Failed authentication for USERNAME
It doesn't write IP with a failure.
And strings "Accepted connection" and "Failed auth.." may even not stand nearby (as I think on heavily loaded servers) to be able to use fail2ban.
What to do? And how jabber servers (using ejabberd) are protected?
You can use iptables to limit the number of connection attempts per minute an IP address can attempt. Since these are automated attacks most of the time the script moves on to find another target once it's blocked.
This example is for tcp port 22 (ssh) and will allow 3 connection attempts per minute before dropping packets from that IP address.
fail2ban would provide a useful additional logic layer on top of iptables. A request on ejabberd's site suggested a possible approach to be able to use fail2ban, using log_modsession. It's included in ejabberd-modules.
To log the failed authentication attempts, ejabberd's core needs to be patched. Luckily though log_modsession is shipped with that patch, so you just need to apply it and recompile ejabberd.
Another way is to use the same approach used by fail2ban, since they use the file alteration monitor (fam) library, I guess we can create a custom daemon that monitor the ejabberd log file and outputs in a format compatible with fail2ban, and there are python and perl bindings for that.
Sometimes tweaking the sources of another program to fit your needs adds you an additional work of download / patch / compile so you can't benefit from your distro security updates, you have to do that work, on the other side you can submit your patch to the authors in charge of ejabberd and wait for their answer if they find that valuable and the whole community could benefit.
But IMHO I would go with the custom daemon.