Don MacAskill's questions

I have a number of internal 8-bit SCSI-1 drives that I'd like to archive for historical purposes. These are all the old NeXT Cubes and NeXTStations (monochrome and color) that were used by id Software to create DOOM and Quake. I'd like to donate the machines somewhere they can do some good, but John Carmack had me promise more than a decade ago that I'd wipe the drives if I ever passed them along, and I'd hate to lose any priceless data that's on them.

In theory, I realize SCSI is backwards compatible, and so I may just be able to plug it into an appropriate cable & modern SCSI card...

But I suspect that it's not quite that simple, and further, it's not obvious what type of cable or adapter I would need and where to acquire one.

If I successfully do extract the data, I'll ping John and see if it might be ok to release the disk images to the public. It's his data, so it'll be his call. id has open-sourced much of their code, so there might not be anything useful here, but the geek in me would hate to lose it if there is. :)

I have a MySQL 5.5.4 table with >200M rows which has a FULLTEXT index on two columns (Title,Body).

When I do a simple FULLTEXT query in the default NATURAL LANGUAGE mode for some popular results (they'd return 2M+ rows), I'm getting zero rows back:

SELECT COUNT(*) FROM itemsearch WHERE MATCH (Title, Body) AGAINST ('fubar');

But when I do a FULLTEXT query in BOOLEAN mode, I can see the rows in question do exist (I get 2M+ back, depending):

SELECT COUNT(*) FROM itemsearch WHERE MATCH (Title, Body) AGAINST ('+fubar' IN BOOLEAN MODE);

I have some queries which return ~500K rows which are working fine in either mode, so if it's result size related, it seems to crop up somewhere between 500K and a little north of 2M.

I've tried playing with the various buffer size variables, to no avail. It's clearly not the 50% threshold, since we're not getting 100M rows back for any result.

Any ideas?

On a related question about SSDs and TRIM (see: Possible to get SSD TRIM (discard) working on ext4 + LVM + software RAID in Linux? ), it turns out that dmraid may now (or shortly) support TRIM on RAID-1.

Typically, we've used md (via mdadm) to create our RAID-1 volumes, then used LVM to create volume groups, then formatted with the file system of our choice (ext4 lately). We've been doing this for years, and Google & ServerFault searches seem to confirm this is the most common way of doing software RAID with volume management.

Google searches seem to suggest that dmraid is use for so-called 'fakeRAID' configurations where there's some level of hardware 'help' in the form of RAID BIOS in the controller, which we don't have (and don't want to use - we'd like a fully software solution).

Since we'd like to use TRIM on our SSDs, and since md doesn't seem to (yet?) support TRIM, I'm wondering if it's possible to use dmraid instead of md to create RAID-1 (and RAID-1+0) volumes in software, with no hardware support (ie, just plugged into a dumb SATA/SAS bus)?

Playing around with using a modern kernel (2.6.37 currently) on an older OS (CentOS 5.5) so we can get TRIM (discard) working on our SSDs (Crucial C300s).

The most recent hdparm (9.37) concurs that the C300 supports TRIM:

./hdparm -I /dev/sdc | grep TRIM
       *    Data Set Management TRIM supported (limit unknown)
       *    Deterministic read data after TRIM

But the kernel doesn't seem to agree when I attempt to mount /dev/sdc with the discard option:

EXT4-fs warning (device sdc): ext4_issue_discard:2619: discard not supported, disabling

We're experimenting with other Linux flavors as I type this, but it'd be nice to know what's going on, regardless.

Is this a manifestation of some other ancient component of CentOS 5.5 that's misinforming the kernel? Or maybe is hdparm using a different mechanism than the kernel to determine whether TRIM is supported?

We use RAID1+0 with md on Linux (currently 2.6.37) to create an md device, then use LVM to provide volume management on top of the device, and then use ext4 as our filesystem on the LVM volume groups.

With SSDs as the drives, we'd like to see the TRIM commands propagate through the layers (ext4 -> LVM -> md -> SSD) to the devices.

It looks like recent 2.6.3x kernels have had a lot of new SSD-related TRIM support added, including lots more coverage of Device Mapper scenarios, but we still can't seem to get it to cascade down properly.

Is this possible yet? If so, how? If not, is any progress being made?

We need some more advanced functionality than ELB provides (mostly L7 inspection), but it's not obvious how to handle things like heartbeat and high availability with something like haproxy using EC2. There's a high likelihood we'd need 3 or more haproxy nodes in the cluster, so simple heartbeat between two nodes isn't going to work.

Seems like having a heartbeat'd layer in front of the haproxy nodes would be the way to go, possibly using IPVS, but handling the configuration changes as the EC2 cluster changes (either via intentional changes, like expansion, or unintentional, like losing an EC2 node) seems non-trivial.

Preferably the solution would span at least two Availability Zones.

In answer to Qs: No, sessions aren't sticky. And yes, we'll need SSL, but that could in theory be handled by another setup entirely - we're able to direct SSL traffic to a different location than non-SSL traffic.