I started working for a company that fired a previous IT worker for leaking data.

I can only say the following things:

We use a Firebird DB with an application written by another company, Proxmox, for virtualization of Windows Server 2008 R2, SQL Server, cloud core Mikrotik router, and a few other Mikrotik devices.

I am not 100% sure, but is there some quick way to check if there are some backdoors left, without interrupting internal processes and reformatting everything?

This previous guy was really good, having written software in C++ and C#. I also know that he did some assembler and cracked a few programs in ollydbg.

Every now and then I get the odd request to provide remote support, troubleshooting and/or performance tuning on Linux systems.

Larger companies often already have well established procedures to provide remote access to vendors/suppliers and I only need to comply to those. (For better or for worse.)

On the other hand small companies and individuals invariably turn to me to instruct them with what they need to do to set up me up. Typically their servers are directly connected to the internet and the existing security measures consist of the defaults for whatever their Linux distribution is.

Nearly always I'll need root level access and whoever will be setting up access for me is not an expert sysadmin. I don't want their root password and I'm also pretty sure my actions won't be malicious, but what reasonably simple instructions should I give to:

• set up an account and securely exchange credentials
• set up root (sudo) access
• restrict access to my account
• provide audit trail

(And yes I'm aware and always warn those clients that once I do have admin access hiding any malicious actions is trivial, but let's assume that I have nothing to hide and actively participate in creating an audit trail.)

What can be improved on the steps below?

My current instruction set:

set up an account and securely exchange credentials

I provide a password hash and ask that my account is set up with that encrypted password, so we won't need to transmit a clear text password, I'l be the only one that knows the password and we don't start off with a predictable weak password.

sudo useradd -p '$1$********' hbruijn

I provide a public key SSH (specific key-pair per client) and ask they set up my account with that key:

sudo su - hbruijn
mkdir -p ~/.ssh
chmod 0700 ~/.ssh
echo 'from="10.80.0.0/14,192.168.1.2" ssh-rsa AAAAB3NzaC1y***...***== hbruijn@serverfault' >> ~/.ssh/authorized_keys
chmod 0600 ~/.ssh/authorized_keys 

set up root (sudo) access

I ask the client to set up sudo for me with sudo sudoedit or by using their favourite editor and append to /etc/sudoers:

hbruijn ALL=(ALL) ALL

restrict access to my account

Typically the client still allows password based logins and I ask them to add the following two lines to /etc/ssh/sshd_config to at least restrict my account to SSH keys only:

Match user hbruijn
PasswordAuthentication no

Depending on the client I'll route all my SSH access through a single bastion host to always provide a single static IP-address (for instance 192.168.1.2) and/or provide the IP-address range my ISP uses (for instance 10.80.0.0/14). The client may need to add those to a firewall whitelist if SSH access is restricted (more often than not ssh is unfiltered though).

You already saw those ip-addresses as the from= restriction in the ~.ssh/authorized_keys file that limits the hosts from which my key can be used to access their systems.

provide audit trail

Until now no client has asked me for that, and I have not done anything specific beyond the following to cover my ass:

I try to consistently use sudo with individual commands and try to prevent using sudo -i or sudo su -. I try not to use sudo vim /path/to/file but use sudoedit instead.

By default all the privileged actions will then be logged to syslog (and /var/log/secure):

Sep 26 11:00:03 hostname sudo:  hbruijn : TTY=pts/0 ; PWD=/home/hbruijn ; USER=jboss ; COMMAND=sudoedit /usr/share/jbossas/domain/configuration/domain.xml  
Sep 26 11:00:34 hostname sudo:  hbruijn : TTY=pts/0 ; PWD=/home/hbruijn ; USER=root ; COMMAND=/usr/bin/tail -n 5 /var/log/messages

I have mostly gives up on customising my work environments, the only thing I really do is set the following in my ~/.bash_profile increasing the bash history and to include time-stamps:

export HISTSIZE=99999999999
export HISTFILESIZE=99999999999
export HISTIGNORE="w:ls:ls -lart:dmesg:history:fg"
export HISTTIMEFORMAT='%F %H:%M:%S  '
shopt -s histappend

I'd like to know what are the best approaches for tracking superuser activities on a Linux environment.

Specifically, I'm looking for these features:

• A) Logging keystrokes to a secured syslog server
• B) Ability to replay shell sessions (something like scriptreplay)
• C) Ideally, this should be something impossible (or quite difficult) to circumvent without having physical access to the server.

Think about this from a security / auditing perspective, in a environment where different sysadmins (or even third parties) need to be allowed to perform privileged operations on a server.

Every administrator would have his o her own nominal account, and every interactive session should be fully logged, with the possibility of replaying it if necessary (for example, if someone used mc to delete or alter critical files, it wouldn't be enough to know that that person issued the mc command; there must be a way to see exactly what was done after launching mc).

Additional notes:

1. As womble has pointed out, may be the best option would be not having people logging in with root privileges to perform changes on servers, but instead doing that through a configuration management system. So let's assume a situation where we don't have such a system and we need to grant root level access to different people over the same server.
2. I'm not interested at all in doing this surreptitiously: every person logging in to a server with root privileges would be fully aware that the session will be recorded (in the same way that, for example, call center operators know that their conversations are being recorded)
3. No one would be using a generic superuser account ("root")
4. I am aware of ttyrpld and it seems to do what I'm looking for. But before going that way, I'd like to know if this can be solved by using an unmodified kernel. I want to know if there are any tools for Debian in particular (or Linux in general) that allow full auditing of superuser accounts without patching the shell or the kernel.

I'm running Cygwin with an SSH deamon on a Windows Server 2008 machine. I was looking at the Event Viewer and noticed as much as 5 to 6 failed login attempts per second (brute force) for the last week or so, from different IPs.

How can I autoblock these IPs rather than blocking them one by one manually?

Thanks, Ahmad

What are the tell-tale signs that a Linux server has been hacked? Are there any tools that can generate and email an audit report on a scheduled basis?