We have a few hundred systems backing up to a tape library using AMANDA. Every day we get an email with a long list of STRANGE backup results. Almost all of these are because a log file was being written or a file-based cache was being updated on a web server. For example (5 machines showing a variety of STRANGE messages):
STRANGE DUMP DETAILS:
/-- av1 / lev 2 STRANGE
sendbackup: start [avion:/ level 2]
sendbackup: info BACKUP=/bin/tar
sendbackup: info RECOVER_CMD=/bin/tar -xpGf - ...
sendbackup: info end
? /bin/tar: ./boot: directory is on a different filesystem; not dumped
? /bin/tar: ./dev: directory is on a different filesystem; not dumped
? /bin/tar: ./fs: directory is on a different filesystem; not dumped
? /bin/tar: ./proc: directory is on a different filesystem; not dumped
? /bin/tar: ./run: directory is on a different filesystem; not dumped
? /bin/tar: ./sys: directory is on a different filesystem; not dumped
? /bin/tar: ./wwwusers: directory is on a different filesystem; not dumped
| /bin/tar: ./tmp/.X11-unix/X0: socket ignored
| /bin/tar: ./tmp/gpg-v9zbWU/S.gpg-agent: socket ignored
| Total bytes written: 795084800 (759MiB, 740KiB/s)
sendbackup: size 776450
sendbackup: end
\--------
/-- gitlab / lev 3 STRANGE
sendbackup: start [gitlab:/ level 3]
sendbackup: info BACKUP=/bin/tar
sendbackup: info RECOVER_CMD=/bin/tar -xpGf - ...
sendbackup: info end
| /bin/tar: ./tmp/gitaly-ruby215330643/socket.0: socket ignored
| /bin/tar: ./tmp/gitaly-ruby215330643/socket.1: socket ignored
| /bin/tar: ./var/lib/gssproxy/default.sock: socket ignored
? /bin/tar: ./var/log/gitlab/gitlab-workhorse/current: file changed as we read it
? /bin/tar: ./var/log/gitlab/nginx/gitlab_access.log: file changed as we read it
| /bin/tar: ./var/opt/gitlab/gitaly/gitaly.socket: socket ignored
| /bin/tar: ./var/opt/gitlab/gitlab-rails/sockets/gitlab.socket: socket ignored
| /bin/tar: ./var/opt/gitlab/gitlab-workhorse/socket: socket ignored
| /bin/tar: ./var/opt/gitlab/postgresql/.s.PGSQL.5432: socket ignored
| /bin/tar: ./var/opt/gitlab/redis/redis.socket: socket ignored
| /bin/tar: ./var/spool/postfix/private/anvil: socket ignored
| /bin/tar: ./var/spool/postfix/private/bounce: socket ignored
| /bin/tar: ./var/spool/postfix/private/defer: socket ignored
| /bin/tar: ./var/spool/postfix/private/discard: socket ignored
| /bin/tar: ./var/spool/postfix/private/error: socket ignored
| /bin/tar: ./var/spool/postfix/private/lmtp: socket ignored
Look in the '/var/log/amanda/log.error/gitlab._.3.20180606231049.errout' file for full error messages
\--------
/-- haproxy00-127 / lev 3 STRANGE
sendbackup: start [haproxy00-127:/ level 3]
sendbackup: info BACKUP=/bin/tar
sendbackup: info RECOVER_CMD=/bin/tar -xpGf - ...
sendbackup: info end
? /bin/tar: ./var/log/messages: file changed as we read it
Look in the '/var/log/amanda/log.error/haproxy00-127._.3.20180607003248.errout' file for full error messages
\--------
/-- imap00 /srv/dovecot lev 0 STRANGE
sendbackup: start [imap00:/srv/dovecot level 0]
sendbackup: info BACKUP=/bin/tar
sendbackup: info RECOVER_CMD=/bin/tar -xpGf - ...
sendbackup: info end
? /bin/tar: ./a1/Maildir/new/1528332018.M.imap00.example.com,S=6309,W=6430: File removed before we read it
? /bin/tar: ./a1/Maildir/new/1528335571.M.imap00.example.com,S=8299,W=8451: File removed before we read it
? /bin/tar: ./a2/Maildir/cur/1528306776.M.imap00.example.com,S=13517,W=13770\:2,Scd: File removed before we read it
? /bin/tar: ./a2/Maildir/cur/1528332257.M.imap00.example.com,S=22257,W=22562\:2,STcd: File removed before we read it
? /bin/tar: ./a3/Maildir/cur/1528287062.M.imap00.example.com,S=10343,W=10558\:2,: File removed before we read it
? /bin/tar: ./a3/Maildir/cur/1528297383.M.imap00.example.com,S=4179,W=4265\:2,: File removed before we read it
? /bin/tar: ./a3/Maildir/cur/1528312816.M.imap00.example.com,S=29364,W=29796\:2,: File removed before we read it
? /bin/tar: ./a3/Maildir/new/1528327304.M.imap00.example.com,S=52059,W=53029: File removed before we read it
? /bin/tar: ./a4/Maildir/.folder/dovecot.index.log.2: File removed before we read it
? /bin/tar: ./a4/Maildir/.Personal/dovecot.index.log.2: File removed before we read it
? /bin/tar: ./a4/Maildir/.Trash/cur/1527783480.M.imap00.example.com,S=31976,W=32433\:2,Sab: File removed before we read it
? /bin/tar: ./a4/Maildir/cur/1528336066.M.imap00.example.com,S=122100,W=124740\:2,ab: File removed before we read it
? /bin/tar: ./a4/Maildir/cur/1528337059.M.imap00.example.com,S=28553,W=28964\:2,ab: File removed before we read it
Look in the '/var/log/amanda/log.error/imap00._srv_dovecot.0.20180606220538.errout' file for full error messages
\--------
/-- webserver1 / lev 2 STRANGE
sendbackup: start [webserver1:/ level 2]
sendbackup: info BACKUP=/bin/tar
sendbackup: info RECOVER_CMD=/bin/tar -xpGf - ...
sendbackup: info end
? /bin/tar: ./var/log/httpd/kemp_access_log: file changed as we read it
? /bin/tar: ./var/log/httpd/wiki.example.com_access_ssl.log: file changed as we read it
? /bin/tar: ./var/log/httpd/www.example.com_access.log: file changed as we read it
? /bin/tar: ./var/log/httpd/www.example.com_ssl_access_ssl.log: file changed as we read it
Look in the '/var/log/amanda/log.error/webserver1._.2.20180607004437.errout' file for full error messages
\--------
(brought to you by Amanda version 3.3.3)
Relevant parts of our current configuration:
define dumptype nocomp-root-tar {
program "GNUTAR"
comment "Root partitions with gtar"
compress none
estimate server
priority low
exclude list "/etc/amanda/amanda.exclude.list"
auth "bsd"
}
define dumptype nocomp-user-tar {
program "GNUTAR"
comment "Non-root partitions with gtar"
compress none
estimate server
priority medium
exclude list "/etc/amanda/amanda.exclude.list"
auth "bsd"
}
We don't want to exclude these files from backup just because they might change or disappear. We also can't disable the IMAP server while backups are running. It looks like I can change our systems to use amgtar
as their backup method which will let us eliminate all these messages.
define application-tool app_amgtar {
plugin "amgtar"
property "NORMAL" ": socket ignored$"
property append "NORMAL" "file changed as we read it$"
property append "NORMAL" ": directory is on a different filesystem; not dumped$"
property append "NORMAL" ": File removed before we read it$"
}
define dumptype nocomp-root-amgtar {
program "APPLICATION"
application "app_amgtar"
comment "Root partitions with amgtar"
compress none
estimate server
priority low
exclude list "/etc/amanda/amanda.exclude.list"
auth "bsd"
}
...
Are there any downsides to amgtar
?
Will it be more difficult to recover older backups if we make this change?
The documentation seems to say that only GNUTAR
supports exclude list
. Does amgtar
also support it since it uses GNUTAR
underneath?
We made the change and haven't had any issues. Unfortunately, no more information about it...