cedivad's questions

I have two identical servers running identical software. One of them is in the same datacenter as the source, with the other being 100ms+ away.

When running a bandwidth-heavy piped ssh transfer on the server that's just a hop away I can easily saturate the connection. When the server is far away I get maybe 10 Mbps (1/100th the speed).

iperf shows the far away server does a good 300 Mbps and can saturate the connection when using multiple parallel connections.

I know there is an unmaintained fork of ssh attempting to optimise for transfer speeds, and I know about tcp tuning. I also know I could use netcat, but I need encryption. I'm already using mbuffer. I also cannot split the source/pipe it in multiple concurrent connections.

Is there anything I can do here? I've battled this problem for years and being unable to fix it across time platforms and hardware is baffling me.

This sounds like a nice problem to solve, but I cannot believe it hasn't already been done.

I will need to use SSL SNI, but unfortunately from a recent Cloudflare blogpost only 90% of the network supports it. How can I (for example, with nginx) detect if the client supports SNI and provide/redirect to the HTTP version of the website? Is this possible? How else can I not lose the 10% of the traffic using SNI? Is it correct to assume that I wouldn't be able to redirect HTTPS traffic to HTTP without a valid certificate, and thus this request is impossible?

Thank you.

How can I set an User-Agent header when empty? (Eg, HTTP/1.0)

This is what I tried so far:

set $ua $http_user_agent;

if ($http_user_agent = "") //also tried = false
{
    set           $ua "Fixing-Empty-User-Agent";
    proxy_pass    http://$host$request_uri;
}

proxy_connect_timeout   2;
proxy_send_timeout      10;
proxy_read_timeout      10;

proxy_set_header User-Agent: $ua;

I'm looking for an HTTP load balancer that will handle server errors silently. What i want is to load balance every single request so that it works out, in the worst case with a little of timeout.

• If the working web node returns an HTTP 500 server error, the load balancer will have to retry the request with another web node. If the second node does return another 500 error, doing the same with the last node (i suppose to have 3 nodes). If the last node returns a 500 error, display it to the end user.

• If a server node timeout (takes more than 1 or 2 second to answer) the request will have to be routed to another server, the client should receive a good answer within 2+ seconds.

Any idea of what is happening? It's swapping, but why the 30 mysql instances?

I have a remote system on rescue mode where i need to start gPXE. I cannot connect an USB stick to the system and so on.

Is there any way i can install gPXE to the hard drive so that once rebooted it will be started instead of the OS?

Thank you.

Many of the links that connect NY and AMS are often saturated. That means that running a transfer over them (e.g., move 300GB at 1MB/s) would take an age if compared with what our connections can offer.

I came across to the problem in the past, like 3 years ago, when i was really newbie to coding and linux, i came to my conclusion that i will post on the bottom of this post. However, it's dirty and i don't like it. The script dosen't work as it is, since that it was written for a very specific environment, however it give you the idea.

My question is, do you know of anything better alternatives to transfer files across the ocean in a fast way?

#!/bin/sh

upto="$1"
filepath="$2"
remotepath="$3"

if [ ! -f ${filepath} ]
then
exit 0
fi

password=$(/all/script/password 10)
filesize=$(du -b ${2} | sed 's/\([0-9]*\)\(.*\)/\1/')

if [ $filesize -gt 5368709120 ]; then
parts="80"
elif [ $filesize -gt 2147483648 ]; then
parts="50"
elif [ $filesize -gt 1310720 ]; then
parts="20"
else
parts="2"
fi

splitsize=$(($filesize / $parts))

split -b "$splitsize" -a 2 "$2" /all/tmp/cup/${password}_

#UPLOAD
declare -a pwait
for tmpfile in /all/tmp/cup/${password}_*
do
    scp ${tmpfile} root@${upto}.domain.com:/all/tmp/cup/ &
        array_lenght=${#pwait[@]}
        pwait[${array_lenght}]=$!
done

#ATTENDERE
for prid in ${pwait[@]}
do
wait $prid
done

#UNISCI FILE REMOTO
ssh root@${upto}.domain.com "cat /all/tmp/cup/${password}_* > ${remotepath} && wait && rm -f /all/tmp/cup/${password}_*"

#RIMUOVI ROBA DI TROPPO LA
#eval ssh root@${upto}.domain.com rm -f /all/tmp/cup/${password}*

#REMOVE HERE
rm -f /all/tmp/cup/${password}_*

exit 0

I've looked into tools like DRDB (that can scale only up to 2 active-active nodes) and GlusterFS (that is great, but not for the small files that a web server usually serves).

I'm looking for a filesystem/cluster of storage servers:

• That is POSIX compilant (as many web apps require this)
• That works great with a 90% reads / 10% write workload of small files (think at index.php pages or small images)
• Is scalable, both in size and performance
• Access time is minimal

I have tough a lot on this and i think that there is nothing out there that could help me.

I would like to have a scalable storage system as backed for web servers serving php/HTML.

I was looking at glusterfs, that seems rather stable.

My question is, does it really scale? Can I think to host a lot of loaded website on the top of glusterfs? Or these are just dreams? If you have any better alternative please suggest.

MySQL will use its own storage.

This one is tricky. I want to have the ability to move a single database from a server to another, with no downtime. I could take a snapshot of the data on the primary, move to the secondary, start up replication, get up to date data on the secondary, move the queries to the secondary.

But what if the secondary server already is a production MySQL server, that I can't restart, and that has to accept incoming "databases transfert" from multiple different sources servers?

I would just like to have a really brief introduction to the matter.

I've enabled the -v option on master.cf; but still my /var/log/maillog is almost empty. I'm having a problem i have my hard times debugging without any log.

top than m (view disk iops) than o than write (order by write desc)

Will show a sort of iotop ordering by the process that uses the most write operation per second. Any way to se the bandwidth used with the process instead of the IOPS?

I'm using FreeBSD. I would like to write a munin plugin to log it, but i don't understand how to get this data.

I want to give access to my root server to an external system administrator, but i want to be sure to double check what he is doing to my server, e.g. copying data i don't want them to do and so on. I would also like to take a track of whatever file is accessed, even in read only and not edited.

How can i do that?

I have a NIC with 2 aliases on it. The network interface has 3 IPs configured on it. Think about it like this: i can ping the same server by hitting .100 .101 and .102.

I want the source address of the outgoing connections to be distributed among these ip. So if i have 3 opened connections, one connection will have result as having an IP address ending with .100, the other two should result as having as ip addresses .101 and .102.

I'm using FreeBSD but I think this question to be Linux-Like wide.

It seems that certain MegaRAID chips don't support JBOD (unfortunately).

I've read online that to make these controllers act as a pass-through, I should configure the disks in RAID0.

What RAID0 settings do you recommend?

Google released it's Google Authenticator Module with a PAM interface.

Have someone experience on implementing this into an imap server?

I could explain it in my bad english, but looking at this page is the fastest way:

http://www.adaptec.com/en-us/_common/hybrid-raid/

Do you know of any linux implementation for this outstanding feature? (to be so able to use it without the adaptec controller?)

My disk are 10x1TB SAS 7200 RPM in a RAID 10 with a MegaRaid 9260 hardware controller with cache/BBU. This results in a 4.6TB RAID 10 volume. hdparm -t (when device is empty) results in 500MB/s.

RAID chunk size is 64KB, filesystem block size is 2KB (I'm going to change it to the minimum chunk size and 4KB block size).

The directory pattern is /data/x/yz/zyxabc.gz

I'm using EXT4 with plans to move to XFS. The OS is RHEL 6.

As of now, it works great. The workload is 99% reads and it can read up to 300 files/second under normal conditions. The problem is backups. It takes 6 days to backup with scp. rsync is even slower. DD goes at about 2MB/s. LVM snapshots could be an option if I take the snapshot, back it up, and then delete it. Data consistency is very important to me.

Files are about 0.5-4KB each. Would I see increased backup performance if I stored all of the files in a database instead? What other alternatives are there for me to tackle the problem of backing up this many small files in a reasonable window?

I know that i can use PXE to boot the server, but is it possible to boot the server from the network and than tell it that he need to use some kind of network disk for its os? So that i don't need any mass memory on the server (hard disk, usb drives..)

Is it possible? Thank you a lot!