Home / server / Questions / 383512
In Process
David LeBauer
Asked: 2012-04-27 09:58:53 +0800 CST

delete all files on all nodes on a RedHat cluster?

  • 772

I am developing a program that runs on a cluster and writes a lot of short bits. First, it outputs to the node's scratch directory; when it is done, it rsyncs the files to a hard drive and then removes them from scratch. Without this step, the program often gets stuck. I am on a RedHat cluster with Rocks.

The program isn't perfect, and during development, many users may leave orphaned unneeded files on scratch (they have already been rsync'd).

For my own files, I use this script to get rid of the orphaned files:

for i in {1..24}
do
ssh compute-0-$i "rm -rf /scratch/*`
done

However, I have the following questions:

  1. does this violate any "best practice" rules?
  2. to delete all files from all users whenever anyone runs it, must I chmod -R a+rw /scratch?
linux redhat bash cluster

1 Answers

    1. there is no real defined "best practice rules" so you not "violating" anything, as long as it works :))
    2. not really sure what you meant by that but doing something like chmod -R a+w is generally not a good idea (in terms of security) that means any user on the system (could be even apache) knowing that you have such access can write anything and potentially execute that.
    3. Try out following project, you might find it more useful for you (it will keep everything in-sync all the time, so you won't have to run your shell script to do that on multiple servers and instead delete only localy which then going go and replicate to all of your nodes.

    DRBD® refers to block devices designed as a building block to form high availability (HA) clusters. This is done by mirroring a whole block device via an assigned network. DRBD can be understood as network based raid-1.

    • 1