Home / server / Questions / 647863
Accepted
Jeff Coe
Asked: 2014-11-29 14:01:48 +0800 CST

Using Puppet to sync two directories

  • 772

I have a bunch of servers that have two directories on them that need to be in sync. For instance, on the same server I might have /var/path/to/dir1 and /var/path/to/dir2 which could have files and folders underneath of each. I'd like to sync /var/path/to/dir1 and have /var/path/to/dir2 as the "backup".

Does anyone know an efficient way in Puppet to sync these two directories? Would I have to create a custom script to do this and use the exec method in the Puppet manifest file?

My guess was to create a script to recursively verify the MD5 hash on every file in the primary directory and then verify the files in the second directory with that list. I wasn't sure if there was a way to do this in Puppet natively.

scripting

2 Answers

  1. Best Answer

    You were on the right track with puppet. Have the puppetmaster serve as the source of truth, house them in the server that runs your puppetmaster, and declare the files that should exist and the canonical versions there. You may even want to version the files that you are making available in the puppetmaster server using something like SVN or GIT.

    This provides you some strong guarantees about what is in those two locations. 

    file { '/var/path/to/dir1/':
  ensure   => 'present',
  source   => 'puppet:///path/to/dir1',
  recurse  => 'true',
} 
file { '/var/path/to/dir2/':
  ensure   => 'present',
  source   => 'puppet:///path/to/dir1',
  recurse  => 'true',
}

    As mentioned above, you may need to remove any collected unwelcome extras with something like

    {
  purge => 'true',
  force => 'true',
}

    As was pointed out to me (thanks!!), if there is a large number of files or a deep hierarchy of sub-folders under '/var/path/to/dir1/', the time to complete a puppet run could grow rapidly. I would still recommend to keep your puppet server as the source of truth, but you might do better to use rsync, something like:

    rsync::get { '/var/path/to/dir1/':
  source  => 'puppet:///path/to/dir1',
  purge   => true,
}

rsync::get { '/var/path/to/dir2/':
  source  => 'puppet:///path/to/dir1',
  purge   => true,
}
    • 3

  2. I got curious and tried it out: 

    [root@localhost vagrant]# cat test.pp
file { '/tmp/path2':
  ensure  => directory,
  source  => 'file:///tmp/path1',
  recurse => true,
  purge   => true,
  force   => true,
}

[root@localhost vagrant]# ls -lR /tmp/path1
/tmp/path1:
total 4
-rw-r--r-- 1 root root    0 Nov 29 01:55 bar
drwxr-xr-x 2 root root 4096 Nov 29 02:07 folder
-rw-r--r-- 1 root root    0 Nov 29 01:55 foo
-rw-r--r-- 1 root root    0 Nov 29 01:55 x

/tmp/path1/folder:
total 0
-rw-r--r-- 1 root root 0 Nov 29 02:07 xxx
[root@localhost vagrant]# ls -lR /tmp/path2/
/tmp/path2/:
total 4
-rw-r--r-- 1 root root    0 Nov 29 02:17 deleteme
drwxr-xr-x 2 root root 4096 Nov 29 02:17 folder2

/tmp/path2/folder2:
total 0
-rw-r--r-- 1 root root 0 Nov 29 02:17 deleteme
[root@localhost vagrant]# puppet apply test.pp 
Notice: Compiled catalog for localhost in environment production in 0.04 seconds
Notice: /Stage[main]/Main/File[/tmp/path2/x]/ensure: defined content as '{md5}d41d8cd98f00b204e9800998ecf8427e'
Notice: /Stage[main]/Main/File[/tmp/path2/deleteme]/ensure: removed
Notice: /Stage[main]/Main/File[/tmp/path2/folder]/ensure: created
Notice: /Stage[main]/Main/File[/tmp/path2/folder/xxx]/ensure: defined content as '{md5}d41d8cd98f00b204e9800998ecf8427e'
Notice: /Stage[main]/Main/File[/tmp/path2/bar]/ensure: defined content as '{md5}d41d8cd98f00b204e9800998ecf8427e'
Notice: /Stage[main]/Main/File[/tmp/path2/folder2]/ensure: removed
Notice: /Stage[main]/Main/File[/tmp/path2/foo]/ensure: defined content as '{md5}d41d8cd98f00b204e9800998ecf8427e'
Notice: Finished catalog run in 0.06 seconds
[root@localhost vagrant]# diff -r /tmp/path1 /tmp/path2
[root@localhost vagrant]#

    So that works.
    Note that you need force => true, too.
    Otherwise existing folders in the destination path will not get removed.

    Or something like this will also work:

    rsync::get { '/backup/foo':
  source  => '/foo',
  purge   => true,
}
    • 0