Home / server / Questions / 851394
In Process
Brad
Asked: 2017-05-22 06:35:33 +0800 CST

Processing image files once and only once without missing any

  • 772

I have a wordpress installation with a large number (millions) of images. I want to optimize those images using ImageMagick mogrify but I only want to process them once (processing them over and over again would result in image quality degradation).

I have to optimize in place (I can't copy the optimized images to a new location because if I did - they wouldn't be accessible to wordpress).

Users can upload new images at any point (even while the optimization process is running) Even more complicated users can upload the same image with the same name. So.... what I'm trying to do is script the following:

  • Store the current date/time to a variable
  • Get the date the image optimization last ran from a lastrun file. If a last run file doesn't exist default to "1900-01-01 01:01:01"
  • Mount a share to a backup server (where unoptimized originals will be stored just in case)
  • Search for any files with a *.jpg extension, larger than 10KB, modified any more recently than the last run date
  • For any matching files:
    • Copy them to the backup/originals share
    • Optimize them in place:
      • Reduce the quality to 70%
      • Strip any comments and meta data
      • Create progressive jpgs
  • Store the current date/time (from the firs step) to the last run time file (this is so we don't re-process the same images over and over again)

The problem with this approach is that all the new images will have a last modified date AFTER the current date/time variable since that's stored at the beginning of the script running.

I could write the date/time the script finished to the lastrun file instead, but then I potentially miss any files upload while the script was running.

So how do I ensure that I don't process any image twice AND I don't miss processing any images.

Here's my script as is:

 # Only uncomment this the very first time to generate the .lastrun file.
 #echo "1900-01-01 01:01:01" >/scripts/config/image-optimizer/.lastrun
 sudo mount.cifs //backup-server/original-images /mnt/originals -v -o user=myuser,dom=mydomain,password=redacted
 last=$(cat /scripts/config/image-optimizer/.lastrun)
 curr=$(date "+%Y-%m-%d %H:%M:%S")
 cd /path/to/wordpress
 find ./ -type f -name "*.jpg" -size +10k -newermt "$last" -exec cp --parents {} /mnt/originals \; -exec mogrify -quality 70 -strip -interlace Plane -monitor {} \;
 echo "$curr" >/scripts/config/image-optimizer/.lastrun
bash

1 Answers

  1. Once optimized, touch the file and set the modification time to $curr:

    find ./ -type f -name "*.jpg" -size +10k -newermt "$last" -exec cp --parents {} /mnt/originals \; -exec mogrify -quality 70 -strip -interlace Plane -monitor {} \; -exec touch --date="$curr" {} \;
    • 0