I noticed my apache server was down today, and going to my hosting dashboard I see a spike in disk throughput and IOPS. At the same time, my log is full of these lines:
108.162.215.47 - - [03/Feb/2019:06:25:01 +0100] "POST /xmlrpc.php HTTP/1.1" 403 426 "-" "python-requests/2.21.0"
108.162.215.47 - - [03/Feb/2019:06:25:02 +0100] "POST /xmlrpc.php HTTP/1.1" 403 426 "-" "python-requests/2.21.0"
108.162.215.47 - - [03/Feb/2019:06:25:04 +0100] "POST /xmlrpc.php HTTP/1.1" 403 426 "-" "python-requests/2.21.0"
172.69.33.204 - - [03/Feb/2019:06:25:04 +0100] "POST /xmlrpc.php HTTP/1.1" 403 2471 "-" "python-requests/2.21.0"
xmlrpc.php is a file used by Wordpress to communicate with a remote server. It is known to be the source of many attacks and it's often recommended to block access to it (see https://www.hostinger.com/tutorials/xmlrpc-wordpress for instance)
- So am I right that these xmlrpc attacks can be the cause of the disk throughput spike, even though I don't see any CPU spike at the same time?
- The log shows that these requests have been blocked (403), so why would my apache server go down?
- What should I do so that this does not happen again in the future? I have fail2ban installed on my server but maybe I need a special configuration for xmlrpc (I'm still a noob regarding server administration)
In WordPress,
xmlrpc.php
is an API that can be used by e.g. the WordPress mobile app to communicate with the website and perform certain actions. However, its bad design also allows an attacker an efficient way to attempt brute-forcing the WordPress admin password, and if your site allows comments and/or pingbacks, a way to add comment/pingback spam to your site.If you don't use a WordPress mobile app or the pingback functionality, you might want to completely disable
xmlrpc.php
.However, just disabling it at the level of WordPress might not be enough: since a spammer/attacker typically generates a lot of requests, passing them all the way up the stack through Apache and the PHP interpreter to actual WordPress code can require significant load even if WordPress just denies the request. As a result, you'll want to perform the rejection as early as possible.
Rejecting it in Apache is a simple string match operation in compiled, highly optimized C code, which can be very efficient. Doing the rejection at the level of WordPress involves the PHP interpreter and possibly also WordPress's database, making the rejection operation much more costly in terms of CPU power needed.
In the configuration of Apache 2.2 (or 2.4 with the legacy access control syntax module enabled) you could do it by adding a block like this to the
<VirtualHost>
block of your site:Using Apache 2.4's newer access control syntax, it will be:
Using
fail2ban
to block the attackers sending such requests at the kernel level (usingiptables
controlled byfail2ban
) would be even more efficient, but since most such attackers have multiple IP addresses at their disposal, you would likely see the attack source starting to move from one IP to another in an effort to get a number of attempts in before each new IP is blocked. A block at the level of Apache will ensure that all requests forxmlrpc.php
will be blocked.The disk throughput spike you're observing might be essentially all from writing all the log messages for all those rejections.
I had a similar problem once, and then the customer complaint was first that Apache was limiting traffic: because of all the spammer's attempts, legitimate traffic was getting pushed aside. When Apache's resource limits were adjusted, then WordPress's database started crashing because of the sheer amount of requests (it was located on a pretty low-spec system). Blocking the source IP just caused the source to move to another IP and resume flooding after a few hours. Blocking
xmlrpc.php
at the level of Apache was the easy fix, and a while later the attacker noticed their efforts were fruitless and stopped trying. But there will always be others...