Home / server / Questions / 469313
Accepted
ghm1014
Asked: 2013-01-16 07:38:02 +0800 CST

Bacula use variable to reuse FileSet Resource

  • 772

I have several environments of the same application in different servers. The directory structure is very similar, so I'd like to use one single FileSet for all of them. For example:

- server1: /opt/env1/app/logs
- server1: /opt/env1/app/bin
- server1: /opt/env1/app/temp
- server1: /opt/env1/app/lib

- server2: /opt/env2/app/logs
- server2: /opt/env2/app/bin
- server2: /opt/env2/app/temp
- server2: /opt/env2/app/lib

- server2: /opt/env3/app/logs
- server2: /opt/env3/app/bin
- server2: /opt/env3/app/temp
- server2: /opt/env3/app/lib

I'd like to have something like:

FileSet {
  Name = "APP"
  Include {
    Options {
      signature = MD5
      Compression = GZIP
    }
    File = "/opt/${env}/app"
  }
  Exclude {
    File = "/opt/${env}/app/logs"
    File = "/opt/${env}/app/temp"
  }
}

Then:

Job {
  Name = "JOBENV1"
  #somehow set env variable as env1
  FileSet="APP"
  ...
}

Job {
  Name = "JOBENV2"
  #somehow set env variable as env2
  FileSet="APP"
  ...
}

Job {
  Name = "JOBENV3"
  #somehow set env variable as env3
  FileSet="APP"
  ...
}
backup

2 Answers

  1. Best Answer

    After a few days with the solution, it seems to be working, so here it goes:

    Up to the example, lets say we have the following jobs:

    • server1-env1-job
    • server2-env2-job
    • server2-env3-job

    Then assuming the --job as a convention, the file set should look like:

    FileSet {
  Name = "someapp-fileset"
  Include {
    Options {
      signature = MD5
      Compression = GZIP
    }
    File = "| bash -c \"echo %n | awk -F '-' '{print \$2}' | xargs -I ARG echo /opt/ARG\""
  }
  Exclude {
     File = "| bash -c \"echo %n | awk -F '-' '{print \$2}' | xargs -I ARG echo /opt/ARG/temp\""
     File = "| bash -c \"echo %n | awk -F '-' '{print \$2}' | xargs -I ARG echo /opt/ARG/logs\""
  }
}

    For more information about the execution of commands in the file set look at the documentation

    Full and incremental backups are running OK.

    • 2

  2. I don't believe you can do this - Bacula's configuration file is a configuration file, not a programming language or shell.

    Your practical options are:

    1. Use identical directory structures on each machine
      This is the most practical solution from Bacula's standpoint. It requires the least additional work in the Bacula configuration file, and if your applications are "identical" it makes the most sense.

    2. Use a fileset per machine
      This is more annoying than option (1) in that you are now maintaining a bunch of nearly-identical filesets. It's most practical if you're generating your Bacula configuration with some other process that can automatically create the filesets.

    3. Use one fileset listing all the directories
      This works, but any time you add a new machine (and thus a new env# directory) you will be changing the global fileset and triggering full backups.

    4. Use a ClientRunBeforeJob directive to put the files in a uniform location
      This is a little dirty, but it works -- Have Bacula rsync the app environment to another location before it does its backup. (If you are on a Linux system you can also use mount --bind to put the app at a uniform location without affecting anything else.)
      The major downside of this approach is you double your storage requirements: you have the running production copy of the app, and the shadow copy Bacula makes in order to take its backup.

    • 1