Chrille's questions

Does Galera Arbitrator require a SST? It says it's stateless but it's not clear if a SST i required. The reason for asking is that I'm having a large database and would like to skip the SST if possible.

I have set up two dedicated haproxy servers to spread the load on three application servers. I have set up regular http 80 balancing, and also a special to work with websockets.

It works great for about 2 hours, but after that it gets really painfully slow, a page load is about 30 seconds. When I restart haproxy it's good again.

Below is my conf. Any idea what may be causing this?

global
  user haproxy
  group haproxy

defaults
  mode http
  timeout connect 5s
  timeout client  5s
  timeout server  60s
  stats enable
  stats auth aa:bb

frontend proxy
   # listen on 80
  bind 0.0.0.0:80

  # allow for many connections, with long timeout
  maxconn 200000 # total maximum connections, check ulimit as well
  timeout client 24h

  # default to webapp backend
  default_backend webapp

  # is this a socket io request?
  acl is_websocket hdr_end(host) -i node.domain.com
  use_backend websocket if is_websocket

backend webapp
   balance roundrobin # assuming that you don't need stickiness
   # allow client connections to linger for 5s
   # but close server side requests to avoid keeping idle connections
  option httpchk HEAD /check.txt HTTP/1.0
  option http-server-close
  option forwardfor

  server app1 x.y.149.133:80 cookie app1 weight 10 check
  server app2 x.y.149.134:80 cookie app2 weight 15 check
  server app3 x.y.149.135:80 cookie app3 weight 15 check

backend websocket
  balance source

  # options
  option forwardfor # add X-Forwarded-For

  # Do not use httpclose (= client and server
  # connections get closed), since it will close
  # Websockets connections
 no option httpclose

 # Use "option http-server-close" to preserve
 # client persistent connections while handling
 # every incoming request individually, dispatching
 # them one after another to servers, in HTTP close mode
 option http-server-close
 option forceclose

 server app1 x.y.149.133:3000 cookie app1 weight 10 check
 server app2 x.y.149.134:3000 cookie app2 weight 15 check
 server app3 x.y.149.135:3000 cookie app3 weight 15 check

I'm using keepalived to setup a virtual ip that points to a master server. When a failover happens it should point the virtual ip to the backup, and the IP should stay there until I manually enable (fix) the master.

The reason this is important is that I'm running mysql replication on the servers and writes should only be on the master. When I failover I promote the slave to master.

The master server:

  global_defs {
    ! this is who emails will go to on alerts
     notification_email {
           [email protected]
     ! add a few more email addresses here if you would like
   }
   notification_email_from [email protected] 

   ! I use the local machine to relay mail
   smtp_server 127.0.0.1
   smtp_connect_timeout 30

   ! each load balancer should have a different ID
   ! this will be used in SMTP alerts, so you should make
   ! each router easily identifiable
    lvs_id APP1         
} 

vrrp_instance APP1 {
         interface eth0
         state EQUAL
         virtual_router_id 61
         priority 999
    nopreempt
         virtual_ipaddress {
             217.x.x.129
         }

    smtp_alert
}

Backup server:

 global_defs {
   ! this is who emails will go to on alerts
   notification_email {
       [email protected]
   ! add a few more email addresses here if you would like
  }
  notification_email_from [email protected] 

  ! I use the local machine to relay mail
  smtp_server 127.0.0.1
  smtp_connect_timeout 30

  ! each load balancer should have a different ID
  ! this will be used in SMTP alerts, so you should make
  ! each router easily identifiable
   lvs_id APP2           
   } 

vrrp_instance APP2 {
    interface eth0
    state EQUAL
    virtual_router_id 61   
  priority 100            
    virtual_ipaddress {
        217.xx.xx.129
    }

notify_master "/etc/keepalived/notify.sh del app2"
notify_backup "/etc/keepalived/notify.sh add app2"
notify_fault "/etc/keepalived/notify.sh add app2”
smtp_alert
}

I'm running Mysql in production today but are planning to switch to either MariaDB or Percona server when I move to new servers. The main reason for the switch is to distance us from Oracle, but also to get better performance and new features.

It looks like Percona server and MariaDB is pretty similar, and they also promote each other's solutions. What would you say is the main difference between them?

Is it possible to install software RAID 1 on a single disk Debian server without erasing/format the data on the original disk?

What approach would you recommend?

I'm hosting 200 GB of product images at S3 (this is my primary file host).

Do I need to back that data up somewhere else, or is S3 safe as it is?

I have been experimenting with mounting the S3 bucket to a EC2 instance, and then making a nightly rsync backup. The problem is that it's about 3 million files, so it takes a while to generate the different rsync needs. The backup actually takes about 3 days to complete.

Any ideas how to do this better? (if it's even necessary?)