I recently started using a remote Ubuntu server for development and testing machine. However the hosting provider reported a crypto mining process was running and he had to shut down the server.
There were no logs or any data that could identify that process or anything that could help figure out what happened. Then it happened again, but this time they captured this:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
246369 redis 20 0 13928 11444 908 S 746.7 0.2 62801:13 /tmp/kmv --pool pool.hashvault.pro:80 --username TRTLv2TW8sjC5LmSpiDdRZ2ndnEwPRpJ9Lgz3vgGY2CTSLkLeKAUFMefEeT6idQBxzSLsXfAvAqfhH5zkxMM3sHu2RL8xh1n5Pg --password x --algorithm chukwa_v2
The only ports that are open are Redis port, 6379.
admin@nicotine2:~$ sudo ufw status
Status: active
To Action From
-- ------ ----
22 ALLOW Anywhere
9200 DENY Anywhere
6379/tcp ALLOW Anywhere
22 (v6) ALLOW Anywhere (v6)
9200 (v6) DENY Anywhere (v6)
6379/tcp (v6) ALLOW Anywhere (v6)
When I checked, there was no /tmp/kmv
folder anywhere to be found. This has happened for the 2nd time.
Any clues, recommendations or suggestions to avoid this?
Redis has a well know remote code execution vulnerability if you do not enable authentication.
This article has more information on your exact issue.
https://www.trendmicro.com/en_gb/research/20/d/exposed-redis-instances-abused-for-remote-code-execution-cryptocurrency-mining.html
The really good question is: What did you install?
There is a good chance you installed an application infected with or vulnerable to this cryptomining software. This can also be in Docker images.
So uninstall everything (or reinstall a blank image might be even better), and monitor your server while installing stuff. You should see a huge CPU spike when you install the infected cryptomining software.
EDIT: If you installed an older and vulnerable version of ElasticSearch (or an unofficial Docker image), the answer is pretty obvious: Install the latest, official and security patched version.
EDIT 2: TheHermit has the correct answer here, as it is the redis process that hosts the cryptomining exploit.
Well for one allowing SSH on Port 22 from anywhere is a Security Risk, recommend limiting it down to a IP that is used for Management and changing default Port 22 to something in a higher range. We need to see more processes of the Server, Redis should be configured with Auth.