Alt-installed python 2.6 (from source using make altinstall)along side 2.4 on a CentOS machine (so as to not break anything), but I am a bit baffled about how to install python packages to the 2.6 directory tree rather than the stock 2.4 tree CentOS installs.
trying to get the mysqldb python package installed with no luck.
for setup.py I've tried different --home --prefix etc... flags and nothing seems to work.
Thoughts?
I'm attempting to copy a 75 gigabyte tgz (mysql lvm snapshot) from a Linux server in our LA data center to another Linux server in our NY data center over a 10MB link.
I am getting about 20-30Kb/s with rsync or scp which fluctates between 200-300 hours.
At the moment it is a relatively quiet link as the second data center is not yet active and I have gotten excellent speeds from small file transfers.
I've followed different tcp tuning guides I've found via google to no avail (maybe I'm reading the wrong guides, got a good one?).
I've seen the tar+netcat tunnel tip, but my understanding is that it is only good for LOTS of small files an doesn't update you when the file is effectively finished transferring.
Before I resort to shipping a hard drive, does anyone have any good input?
UPDATE: Well... it may be the link afterall :( See my tests below...
Transfers from NY to LA:
Getting a blank file.
[nathan@laobnas test]$ dd if=/dev/zero of=FROM_LA_TEST bs=1k count=4700000
4700000+0 records in
4700000+0 records out
4812800000 bytes (4.8 GB) copied, 29.412 seconds, 164 MB/s
[nathan@laobnas test]$ scp -C obnas:/obbkup/test/FROM_NY_TEST .
FROM_NY_TEST 3% 146MB 9.4MB/s 07:52 ETA
Getting the snapshot tarball.
[nathan@obnas db_backup]$ ls -la db_dump.08120922.tar.gz
-rw-r--r-- 1 root root 30428904033 Aug 12 22:42 db_dump.08120922.tar.gz
[nathan@laobnas test]$ scp -C obnas:/obbkup/db_backup/db_dump.08120922.tar.gz .
db_dump.08120922.tar.gz 0% 56MB 574.3KB/s 14:20:40 ET
Transfers from LA to NY:
Getting a blank file.
[nathan@obnas test]$ dd if=/dev/zero of=FROM_NY_TEST bs=1k count=4700000
4700000+0 records in
4700000+0 records out
4812800000 bytes (4.8 GB) copied, 29.2501 seconds, 165 MB/s
[nathan@obnas test]$ scp -C laobnas:/obbkup/test/FROM_LA_TEST .
FROM_LA_TEST 0% 6008KB 497.1KB/s 2:37:22 ETA
Gettting the snapshot tarball.
[nathan@laobnas db_backup]$ ls -la db_dump_08120901.tar.gz
-rw-r--r-- 1 root root 31090827509 Aug 12 21:21 db_dump_08120901.tar.gz
[nathan@obnas test]$ scp -C laobnas:/obbkup/db_backup/db_dump_08120901.tar.gz .
db_dump_08120901.tar.gz 0% 324KB 26.8KB/s 314:11:38 ETA
I guess I'll take it up with the folks who run our facilities the link is labeled as a MPLS/Ethernet 10MB link. (shrug)
We recently upgraded one of our MYSQL machines with an array that is 2.5TB.
After doing so OpenNMS stopped reporting information about our mysql data partition...
when I do a :
snmpwalk -v 1 localhost -c public .1.3.6.1.4.1.2021.9
I get this in the /var/log/messages
Aug 3 16:44:11 mysql6 snmpd[8789]: Connection from UDP: [127.0.0.1]:47333
Aug 3 16:44:11 mysql6 snmpd[8789]: truncating signed value to 32 bits (10)
Aug 3 16:44:11 mysql6 snmpd[8789]: truncating signed value to 32 bits (10)
When I remove the partition from snmpd.conf it I got no notices... and the rest of the resource data is populated in OpenNMS.
From my googling it looks like a common problem but I couldn't find any solutions.
Anyone know of a work around?
Anyone have any experience with Dell's SSDs?
We're looking at using them for a machine running a key value database like Tokyo Tyrant.
They're expensive as well as being pretty small. Only 25/50GB for $850/$1700. Is there any magic to them that they may perform better than a $400 128GB SSD from Intel? Enough to justify the cost and size gap?
They seem to only want you to put them in the new R610 for some reason... curious...
Additionally, has anyone had any luck with 3rd party SSDs in Dell hardware?
Thanks, Nathan
We're getting ready to light up another data center and we have a local administrative machine that scripts aim backups at.
We'd like the backup directory on that machine to bidirectionally sync with a machine in our primary data center so the same backups are seen on all the admin machines.
Any suggestions folks?
I just ordered a cheap Dell PE1425 to run as a xen dom0.
Want to spawn some vm's and test configurations out and what not. Nothing crazy. Ultimately, I want to use it to create VM master images to convert to AMI's for EC2...
What's the preferred host os? We primarily use CentOS in production, and have Ubuntu installed here and there around the office... or should I be considering another distro or OS (BSD?)
Thanks.
We have an application that grabs it's setup parameters from a file. I want to extract one or two statements from the startup string and present them as a nice table.
An example string would be -Dmysql.host=mysql1.company.com but it might also be an ipaddress or a machine name and not an fqdn.
I want locate the -Dmysql.host= but return the servername.
Any tips or pointers as to how to, once I've found the string "-Dmysql.host=" in the file, show everything to the next white space would be appreciated.
Perhaps there is a better method. I plan on running this on a dozen machines or so eventually to return a list of which application machines are configured to talk to which db machine at a glance.
Thanks you for your time.
I'm testing Tokyo Tyrant in a master-master setup and have found the ulog grows out of control and locks up the disk.
At first I found the -ulim option useful and limited the logfile size, however it simply rolls over to a new log, leaving the old ones to clutter up the partition.
I suppose I'll write a shell script that will delete ulogs older than X, once I find out how far back Tokyo Tyrant needs in the update log in order to failover.
Does anyone have any experience with this Tokyo Tyrant? Do you have a feel (acknowledging that every install is different based on what is being stored) for the optimal ulog size vs how far back a Tokyo Tyrant instance needs to look in the ulog to assume master status?
Thanks, nathan
So I need to build a shell script (a skill I am rubbish at, I think too linearly and make everything a pipe) that will connect to a remote machine to a specific directory slurp up all the files older than 5 minutes, extract information from the file's name (encoding details below) and scatter the files into relevant directories based off of that, or create the directories if they don't exist on the local backup host.
On a dozen machines I have directory (let us call it /Prod/Data/) full of files thousands of files named data-HOST-v.7.mmddyy.csv
example: date-web2-v.7.052509.csv
Files older than 5 minutes need to be pulled from the remote machines to a local folder /backup/archive/host/year/month/day/csvs
example /backup/archive/web2/2009/05/29/csvs
I'm sure I can do something like ls -1 | cut -d"." -f3 to extract the date section of the file, then use sed or awk to isolate each section and produce the date variables to pick what directories to dump the files in, do something similar to grab the host, but I am not sure how to go about making that correlate with a file on which to execute a move on. Not sure how to execute that remotely, perhaps it is better to scp all the files over from the remote machine first (short of any file younger than 5 minutes, perhaps a find -mmin +5 statement can be used to suss that out?) then do the sorting when everything is on the backup machine.
Would someone be so kind as to point me in the direction of an example script that may provide similar functionality? Everything I write tends to be command | command | command | etc... and I imagine this task will require some dimensionality.
Thank you for your time.
I am grepping the output of a command and I wish to only return specific, strict results.
I remember some iteration of grep had a strict option -s/--strict.
For example, grep -s bytes would only return entries which mentioned the word "bytes" and not every word that contains bytes, like "megabytes", "kilobytes", etc.