Home / user-70243

bzero's questions

Martin Hope
bzero
Asked: 2020-09-01 00:25:09 +0800 CST
  • 3

I have a cronjob installed (Ubuntu 16) that runs with the root user. So in

sudo crontab -e

I have the following entry:

10 5 * * mon /home/userA/raidchecker.sh

And the script raidchecker.sh does this:

#!/bin/sh
mail=`which mail`
rec="[email protected] [email protected]"
now=$( date +%Y%m%d-%H%M-%S )
mdstat=$( cat /proc/mdstat )
details=$( /sbin/mdadm --detail /dev/md2 )
diskusg=$( df -kh )
disk1=$( smartctl -A /dev/sda | grep Realloc | grep Sector )
disk2=$( smartctl -A /dev/sdb | grep Realloc | grep Sector )
disk1f=$( smartctl -A /dev/sda )
disk2f=$( smartctl -A /dev/sdb )
buddyinfo=$( cat /proc/buddyinfo | grep Normal )

echo "Check from $now\n\n=================\nMemory Fragmentation:\n$buddyinfo\n\nAll partitions:\n\n$mdstat\n\n=================\nData Partition:\n\n$details\n\n=================\nDisk Usage:\n$diskusg\n\n=================\nSMARTCTL:\n\nData Disk /dev/sda and sdb:\n$disk1\n$disk2\n\nFull Reports:\nDisk1:\n$disk1f\n\nDisk2:\n$disk2f" | $mail -s "Check from $now" $rec

When I execute the script manually from the shell ./raidchecker.sh the email contains all infos as expected.

When the cronjob executes the script, the parameters disk1, disk2, disk1f and disk2f are empty (do not contain any data).

Why?

scripting ubuntu bash raid
Martin Hope
bzero
Asked: 2020-08-28 06:00:29 +0800 CST
  • 19

I'm not sure whether serverfault is the right place to ask this, but I wonder what choice you would make if you had to select a new CPU type for your Java Web Application:

a) a CPU with 32 cores and clock speed 2.5 Ghz

or

b) a CPU with 8 cores but clock speed of 3.8 Ghz

Given the fact that each of the web application's incoming HTTP request is served by a free Java thread, it might make sense to choose a), because you can process four time more HTTP requests at the same time. However, on the other hand, CPU b) can finish the processing of a single HTTP request much faster...

What do you think?

Sidenotes:

  • it has to be a physical machine, VMs or cloud solutions are not an option in this case
  • RAM is not important, the server will have 512GB of RAM in the end
  • Caching: the Java web application features an extensive caching framework, so the choice is really on the CPUs.
java multi-threading
Martin Hope
bzero
Asked: 2012-07-26 06:38:13 +0800 CST
  • 2

Setup:

I would like to backup 1 TB on a weekly basis from PROD to BACKUP server. The servers run under Linux Ubuntu-1004-lucid-64-minimal 2.6.32-35-server.

There is the BACKUP server (where I start the rsync program) and the PROD server with the data.

The command I currently use is the following:

time rsync -r --delete [email protected]:/home/myuser/data .

Issue:

The issue with this is: the PROD server becomes nearly unresponsive. The web application that runs on the PROD server is nearly dying.

UPDATE: Current working solution

After some feedback, I'm using this command now for doing the backup of 1TB of data and it is absolutely working fine:

rsync -r --delete --rsync-path "ionice -c 3 nice rsync" --bwlimit=30000 [email protected]:/home/myuser/data .

Please note that I have set a bandwidth limit to 30 Mbps because my connection between PROD and backup server has 100 Mbps and that bandwidth is shared with the production traffic of my web applications.

Please note that I execute this command on the BACKUP server, so that's why I'm using the rsync-path option in order to nice and ionice the process on the remote server (PROD).

My Original questions for possible Solutions

How can I control the impact of rsync (started on the BACKUP server) ?

How would you solve this issue?

My little research brought out the following possibilites:

  • Execute rsync in a way so that the 1TB is synched in chunks? e.g.

    rsync /source/[0-9]* [email protected]:/source_backup  
rsync /source/[a-h]* [email protected]:/source_backup/  
rsync /source/[i-p]* [email protected]:/source_backup/  
rsync /source/[q-z]* [email protected]:/source_backup/

  • Would it help to limit bandwidth with the option

    --bwlimit=10000

  • Is it possible to nice the process on the remote machine in a way? e.g.

    nice -n19 backup.sh

I don't know whether the process on the PROD machine would be niced too?

Any help and ideas is very welcome.

ubuntu