I am using func to perform parallel commands on our servers.
The other day, we had an issue when a service restart of puppet
via func
made all our severs hit our puppetmaster
at the same time.
My question: How can I execute the same exact command on a set of servers while adding a delay before it's executed on the individual servers?
E.g.: random_delay && service puppet restart
I am interested in the random_delay
part of the command.
sleep $((RANDOM % MAXWAIT))
where MAXWAIT is the maximum desired delay in seconds.The
RANDOM
will return a value between 0 and 32767 If you need to set a lower and upper limit to your sleep you need to define two other variables as follows:I quite like S19N's innovative, yet less than ideal solution. I only say it's less than ideal, as it's still largely non-deterministic when stuff will actually happen. I'd much rather be able to guarantee when stuff will happen, and what stuff will happen when it does.
Puppet orchestration is actually a hard problem.
One of the "best-practice" solutions is to use MCollective which will not only allow you to configure when puppet runs on your cluster of machines, but you can also use it for other similar orchestration problems.
This will sleep a random amount between 0 seconds and 12 hours (43200 = 12 x 60 x 60) every 12 hours