Andy's questions

Do any colour coding conventions exist for system architecture block diagrams? If not, who/what would be considered the de-facto standard? Is block diagram the correct terminology in this context?

Specific hardware/roles of interest are caching/persistence tiers, load balancers and public/web servers.

Some (conflicting) examples:

WikiMedia system architecture

A MapReduce system design pattern

I am migrating a classic LAMP application from Red Hat 5.3 to EC2, and cannot find a CentOS 5.3 image in the community to meet my needs. I don't have time to roll my own AMI. Creating my own repo and updating Amazon's stock Fedora 8 has so far proved difficult, and I am concerned that using it as a base will expose security holes, although the kernel has been updated to 2.6.21.7-2.

1. Can anybody recommend a RHEL5.3 binary compatible AMI?
2. If not, where is the information on backported security fixes to the Fedora 8 repo? I was under the impression it had reached EOL.

The idea of running Fedora 8 in production is abhorrent to me - can anybody explain why it's offered by Amazon and whether it's suitably patched?

I have a list of packages installed with yum on CentOS 5.4

[root@server ~]# yum list installed
...
Installed Packages
GConf2.x86_64             2.14.0-9.el5           installed
ImageMagick.x86_64        6.2.8.0-4.el5_1.1      installed
MAKEDEV.x86_64            3.23-1.2               installed
MySQL-python.x86_64       1.2.1-1                installed

I would like to download these rpms locally using

yumdownloader --resolve MySQL-python-1.2.1-1.x86_64 etc.

However the package formatting is different (MySQL-python.x86_64 1.2.1-1 vs MySQL-python-1.2.1-1.x86_64) so I am unable to download them using the above command.

I don't want to have to parse the output of yum list installed, and I also don't want to use the contents of /var/log/yum.log* as I'll have to account for erased packages and version discrepancies. However /var/log/yum.log* does have the formatting I require...

May 25 14:58:15 Installed: groff-1.18.1.1-11.1.x86_64
May 25 14:58:15 Installed: bzip2-1.0.3-4.el5_2.x86_64

Any suggestions?

Developers require database access to a significant number of databases, each with 220+ tables, and currently they are provided this using phpMyAdmin 2.11.9.5. Initial load is very slow, puts some load on the database cluster beneath, and runs lots of (needless) SHOW TABLES FROM x queries to gather data for every table in every datbase. Developers do not require this information in the left hand navigation pane, however this information is still required when they select that database.

There are various options available for configuring the installation, however none of them are obviously tailored to handle a colossal number of databases.

1. Can anybody recommend options on tuning phpMyAdmin to deal with many tables ($cfg['ShowStats'] = false; is the only obvious one)?

2. Is there any equivalent to phpMyAdmin tailored to very large deployments? Perhaps caching table meta data (there are memcached instances available) or not outputting such verbose information?

3. Does phpMyAdmin 3.2.0.1 suffer the same sluggish performance? Upgrading is a possibility.

Currently all MySQL data/API calls are handled by a remote DB cluster (i.e. network latency is a factor in total script execution time).

To reduce execution time in this context, would it be sensible to run a local MySQL instance on each app server to handle mysql_real_escape_string API calls? Has anybody ever done this?

What's the best method of logging a BASH shell session to a file on the host machine?

The preferred output format would be equivalent to PuTTY's "printable output" log setting, as viewed by the user inclusive of input and exlusive of ncurses control character data.

Using DRBD version: 8.2.6 (api:88/proto:86-88)

Here is the contents of /etc/ha.d/haresources

    db1     192.168.100.200/24/eth0 drbddisk::mysql Filesystem::/dev/drbd0::/drbd::ext3::defaults mysql

and /etc/ha.d/ha.cf

    logfile        /var/log/ha-log
    logfacility     local0
    keepalive 1
    deadtime 30
    warntime 10
    initdead 120
    udpport        694
    bcast  eth0, eth4  
    auto_failback off
    node    db1
    node    db2
    respawn hacluster /usr/lib64/heartbeat/ipfail
    apiauth ipfail gid=haclient uid=hacluster
    deadping 5

When testing failover between machines I ran the following commands on db2:

    service heartbeat stop
    service mysqld stop
    drbdadm down mysql
    service drbd stop

/proc/drbd on db1 reported

     0: cs:Connected st:Primary/Unknown ds:UpToDate/DUnknown C r---

What happened next, after:

• Bringing the services back online on db2
• Transferring primary to db2 using hb_primary script
• Taking db1 down as above
• Bringing the services back online on db1
• Transferring primary back to db1 using hb_primary script

was db1 remounted the DRBD disk, assumed the correct IP and started MySQL. There was massive MySQL table corruption; it was all fixable (using InnoDB recovery mode 6, mysqlcheck and the occasional backup), but how did it happen?

I speculate:

1. DRBD disconnected the disk from the filesystem while it was being used by MySQL, as a clean MySQL shutdown would not have resulted in corrupt data
2. heartbeat controlled DRBD, and stopping the heartbeat service "pulled the plug" on DRBD
3. this may happen again in the case of an actual failover (due to heartbeat ping timeout)

I do not have access to this setup again for some time, and would like to repeat the test.

Are the configuration settings correct?

Was the corruption the result of my manual testing?

Is there a better way to test failover than to stop the heartbeat service and let it run the haresources commands?

Following in the spirit of Hidden Features of PowerShell and various others on Stack Overflow, what Linux commands or combinations of commands do you find essential in your work?

Also See:
Hidden Features of MySQL
Hidden Features of PowerShell
Hidden features of Oracle Database
Hidden Features of Windows 2008
Hidden Features of Solaris/OpenSolaris
Hidden Features of SQL Server
Hidden Features of IIS (6.0 / 7.0)