How to install Intrusion Detection system in Ubuntu 12.04

Snort is Network Intrusion Detection System (NIDS). Snort can sniff your network and alert you based on his rule DB if there is an attack on your computers network. It is an opensource system that was build from tcpdump (linux sniffer tool).

This guide can be used for installing snort.

psad: Intrusion Detection and Log Analysis with iptables psad is a collection of three lightweight system daemons (two main daemons and one helper daemon) that run on Linux machines and analyze iptables log messages to detect port scans and other suspicious traffic. A typical deployment is to run psad on the iptables firewall where it has the fastest access to log data.

If you using 12.04LTS Server, the see How to install PSAD Intrusion Detection on Ubuntu 12.04 LTS server.

Sources:
http://nachum234.no-ip.org/security/snort/1-snort-installation-on-ubuntu-11-10-i386/
http://www.cipherdyne.org/psad/index.html

You could take a look at fail2ban, which is directly contained in the repos (so you can simply "sudo apt-get install fail2ban"). I use it for years now, and it kept a lot of hackers out of my server by blocking them. Fail2ban works by parsing log files for specified patterns (it ships with a good sample config), and then blocking the attackers IP -- e.g. if a hacker made 5 failed attempts to login via ssh (even to different accounts), you can have his IP blocked for a specified length of time (e.g. 30min). There are examples shipped for different services, just take a look at the homepage for additional information.

Edit: Notifications are also possible (send a mail if something was detected).

To answer your question, a step-by-step configuration.

Also see this one, but the above worked better for me. Don't expect anything to work the first time though.