I have different processes that have to be run sequentially, but they are on two different servers. In short here what I want to do: 1. Run Process 1 on Server A 2. Run Process 2 on Server B 3. Run process 3 on Server A
But the second process cannot start until the first one is completed.
The two servers have SSH connectivity. I want to start the global process using a cron. But I do not want to start each of them with a crontab, and have long wait time between them, just to make sure that the first one is done before starting the second one. In another way, how can I make a process starting on a remote server from a local script, waiting for the remote script to be completed before proceeding to the next line ?
This is a common feature of workflow software.
You could roll your own, I suppose, but you'd have to account for what to do if the second host is not available, what to do if the initial cronjob is still running when the next interval comes up, etc. It isn't too long before you need flow control, branching, and other logic when one part of a multi-part/multi-host process fails.
The most obvious answer off the cuff (and likely full of it's own faults) would be to append the launch of the second stage via an ssh command appended to the original cronjob.
You'd likely have to setup ssh keys between the hosts to allow automated login from the initial host.
A second option is to somehow pass status info between the hosts, such as a flag file on the original host picked up by the second periodically, etc. I generally find this less desirable as it involved polling, but it doesn't seem that you need an enterprise-level solution here, else this would be posted on serverfault instead.
You may want something like RPC, where each machine simply tells the other that it is done with one task and the next should be run. You could also have the main script be on one machine, containing parts that run commands on the other machine, kind of "inline".
sshwill run commands on remote machines; in the simplest form: