We are an Oracle database shop. I have an application for which I think Oracle likely isn't the best answer. I am looking for a lightweight, good performing and very secure database. The database itself will have probably a dozen tables and maybe millions of records. I don't need stored procedures, clustering or any of the high-end features.
My ultimate goal is security. I'm looking for built-in encryption, an overall secure design and low-profile to minimize potential security holes.
I'll want to run it on *nix, since they are much easier to make low-profile than Windows.
For example, in the BSD world there are several variants. FreeBSD is the most portable and OpenBSD is the most secure. I'm looking for the OpenBSD of the database world.
I'm not aware of any existing item with this particular design goal. The best you will get is a DBMS with reasonably good built-in security. Most of the 'NoSQL' DBMS applications don't offer anything particularly special in security. Traditional database management systems often have quite rich security models and security on this type of system is a mature and well understood discipline.
If you want something to put behind a secure web application, you really have no choice but to design the application with security in mind - traditional database security (e.g. mediate all write access through sprocs with no direct write access to tables) is the best you are likely to get. Don't forget to have a separate DBMS server and firewall access to the machine appropriately.
If you are worried about physical security of the server, many DBMS platforms have options for physically encrypted data (e.g. PostgreSQL). Some operating systems will also support encrypted file systems and certain disks also support physical encryption. Note that key recovery will still negate the encryption - if the key is physically present on the DBMS server a technically savvy hacker may well be able to recover it.
In order to design the security and evaluate what architecture is appropriate you will really have to come up with a threat evalutaion to get some idea of what you are securing against. Some scenarios might be:
If you want to secure an internal system from prying eyes then appropriate user/role security on a DBMS platform is probably adequate.
If you want to secure a web application then you can get some incremental security by adding a secured transaction layer through sprocs. This restricts database writes to explictly supported transactions - for example, if an intruder wanted to erase an audit trail they would have to compromise both application and database security to erase entries from the audit tables. However, the intruder can still execute any valid transaction.
If you have issues with physical security you could possibly try encryption at the disk level. An entry level SAN might also help with this as the disks can only be accessed through the SAN controller. This requres that the thief compromise the SAN controller's security or reverse engineer its internal partition format. Some SANs also offer physical disk encryption.
Bear in mind that most DBMS platforms in widespread use have been used for web applications and have a degree of maturity in this space. None were designed specifically to be hardened, but most (open-source ones in particular) have been used in web applications for the better part of 15 years. In this space, application security is really the weak link.
Full disclosure: I work on the Sybase SQL Anywhere engineering team. I'm actually responsible for many of the security features. I'm trying to answer the question without making the answer sound like an advertisement.
Sybase SQL Anywhere is a lightweight RDBMS which includes all your standard RDBMS features like stored procedures, triggers, row-level locking, etc. as well as your standard security features like users, groups, and object (table, procedure, etc.) permissions. It also contains many advanced security features including strong (AES) database encryption and SSL/HTTPS communications encryption. There is a FIPS-approved encryption component available, and it has achieved Common Criteria certification (look for Adaptive Server Anywhere) at the EAL3+ level.
I do have to say that SQL Anywhere was not designed with security as its primary focus; I'm not aware of any database product that was.