I am wondering if it would be of any help if some kind of tools that does filesystem monitoring (i.e inotify-tools) would be modified in order to monitor/report back through ssh. Would it help sysadmins to be able to do something like inotify-watch 192.168.0.33:/home/deploy/script
? Probably the monitored servers should fulfill some pre-condition, but does this sound feasible?
I've got to be honest, this seems like the wrong way to work this.
First of all, there's no way to know whether the remote machine will support inotify.
Secondly, it seems like a lot of effort to combine what is traditionally a two piece generalized solution, into a single piece specific solution. Monitoring and reporting on remote machines is rarely best accomplished with ssh. When you need this level of information about a remote host, you usually want to put a software agent on the remote host that can do the monitoring, and then run the monitoring system on a different box to poll the agent (or have the agent report back).
If you want to add this ability to a box without installing an additional agent, SNMP would be the logical choice (many/most hosts support SNMP out of the box, or have a vendor supplied SNMP package). Alternately, most agent-based monitoring systems such as Nagios, BigBrother/Hobbit/BigSister, Munin, etc, offer the ability to define your own plug-ins. It wouldn't be that difficult to create an inotify-based plug-in.
If you don't want to use a full-blown monitoring system to monitor the remote box, I'd use something like func, which provides a better framework for this than ssh.
Have you considered munin? http://munin.projects.linpro.no/
The normal method of operation is for the munin-node software to be listening on the remote machines. Periodically, the munin monitoring software will connect to the nodes to collect data; it can even do that over an SSH tunnel. (It's in the FAQ, I cannot put in two links because my rep is too low).
Munin by default can monitor a wide range of statistics, including filesystem monitoring. In addition, it is trivially easy to write your own plugins for munin-node, to collect the data you want.
You might want to look at enabling file auditing, which causes the system to record file events. It is fairly flexible and doesn't require a persistent connection. The auditing can be searched via a command line tool, auditctl(8), so you don't need to wade through mountains of extraneous data. Another bonus is that you get a record of user accounts, process IDs, etc.
As for remote monitoring, you can either have the local server perform the deed for you and ship the data off, or you can run a remote command via ssh, whichever suits your needs best.
Yes, any additional ways to monitor systems are of help. Are you asking because your writing a wrapper?
fam might be what you're looking for. It's got a daemon that does the local detection and libfam that is a library to talk to it to get a more platform-independent notification.
I would use whatever tool you want to watch the file access, and then push it via authenticated web API then you can transform/store/sort the things in a nice web-ui