Home / server / Questions / 81874
Accepted
Magsol
Asked: 2009-11-06 11:53:05 +0800 CST

Cron jobs accessing same static file

  • 772

I'm trying to set up two separate cron jobs that access the same static file. One of the jobs runs every minute, aggregating new data that has arrived into the file. The second job runs every 30 minutes, parsing the aggregated data and truncating the file to reset it for the next 30 minutes.

Obviously there's a potential here for deadlock every 30 minutes. Is there any way to write the cron job for the first task so that it only executes every minute for 29 minutes (short of listing out every minute 1-60, excluding 30 and 60), then starts again on minute 31? Thanks!

Edit

By request, I'm using PHP in order to perform these operations (since there is some significant use of data structures involved). I have implemented the use of PHP's flock function, and while the function's documentation specifies that the call blocks until 1) the lock is obtained, or 2) the LOCK_NB option is specified, I was confused by the fact that the following example seems to imply it's possible to not obtain the lock, even with a blocking call.

Unless this is due to some other issue (permissions, OS interrupts, etc)? This issue of blocking until obtaining a lock was really the motivation for my question, to see if there was another way to avoid locking files entirely, thereby mitigating the risk of a non-blocking lock attempt fail and the script halt.

cron

2 Answers

  1. Best Answer

    You don't want to do that; the one time your job takes longer than a minute, you're going to be hosed. Race conditions like that make for ugly problems: intermittent, unpredictable, and hard to diagnose.

    Pboin is exactly right: you need to explore the use of semaphore in your code, and then the problem just goes away, and you don't have to worry about schedule conflicts at all.

    What language are you using?

    Edit (apologies for the php, haven't used it in a while, and am a little drunk, so may not compile):

    The "usual" tactic with locking is to check for a lock (which in this case I do by trying to get one), and if there IS one, to implement a short wait and try again. You can try using php's sleep() function inside a while loop, e.g:

    while($x){
    if(flock($file_handle, LOCK_EX)){
        $x = false;
    }else{
        sleep(5);
        $x = true;
    }
}

    I use something similar to this for tons of cron-based write jobs. Works like a charm. It's actually a similar method to the ALOHA communications protocol, which is very similar to a later network protocol called "TCP/IP". Try, try, and try again.

    • 2

  2. Well, you could list out your minutes in cron as you suggest, but that's not really solving the problem. (For one thing, cron doesn't guarantee execution within any given minute.)

    You'd probably be better off implementing some locking functionality in your script -- it's much less of a hack. The Advanced Bash Scripting Guide, an excellent and free resource suggests the lockfile command, which will help you do this.

    You'll have a lot less to worry about by doing this correctly.

    • 1