Home / server / Questions / 184586
Accepted
Mike
Asked: 2010-09-26 01:36:17 +0800 CST

Delete files older than X days on remote server with SCP/SFTP

  • 772

Do anyone know some good way to delete files on remote server that are older than X days using just SCP/SFTP? Sure I can write some script on perl etc but I feel it's overkill.
Any UNIX way?
Oneliner?
Separate utility?

Thanks

P.S. The task is to delete some outdated backup files.

files sftp scp deleting

4 Answers

  1. Sure I can write some script on perl etc but it's overkill.

    You don't need a script to achieve the intended effect - a one-liner will do if you have shell access to send a command:

    ssh user@host 'find /path/to/old_backups/* -mtime +7 -exec rm {} \;'

    -mtime +7 matches files created one week ago from midnight of the present day.

    • 7
  2. Best Answer

    This question is very old but I still wanted to add my bash only solution as I was just searching for one when I came here. The grep tar in the listing command is just for my own purpose to list only tar files, can be adapted of course.

    RESULT=`echo "ls -t path/to/old_backups/" | sftp -i ~/.ssh/your_ssh_key [email protected] | grep tar`

i=0
max=7
while read -r line; do
    (( i++ ))
    if (( i > max )); then
        echo "DELETE $i...$line"
        echo "rm $line" | sftp -i ~/.ssh/your_ssh_key [email protected]
    fi
done <<< "$RESULT"

    This deletes all tar files in the given directory except the last 7 ones. It is not considering the date though but if you only have one backup per day it is good enough.

    • 5

  3. If you insist on SCP/SFTP you can list files, parse them using a simple script and delete old backup files.

    Batch mode "-b" switch should help you out. It reads sftp commands from file. http://linux.die.net/man/1/sftp

    • 2

  4. Nothing worked for me from above answers.
    Especially when you limited by password and cannot use private key for sftp utility.
    I have found good script using lftp (gist).

    You need to uncomment # STORE_DAYS=6 to specify count manually.

    #!/bin/bash
# Simple script to delete files older than specific number of days from FTP. Provided AS IS without any warranty.
# This script use 'lftp'. And 'date' with '-d' option which is not POSIX compatible.

# FTP credentials and path
FTP_HOST="ftp.host.tld"
FTP_USER="usename"
FTP_PASS="password"
FTP_PATH="/ftp/path"
# Full path to lftp executable
LFTP=`which lftp`

# Enquery days to store from 1-st passed argument or strictly hardcode it, uncomment one to use
STORE_DAYS=${1:? "Usage ${0##*/} X, where X - count of daily archives to store"}
# STORE_DAYS=6

function removeOlderThanDays() {

# Make some temp files to store intermediate data
LIST=`mktemp`
DELLIST=`mktemp`

# Connect to ftp get file list and store it into temp file
${LFTP} << EOF
open ${FTP_USER}:${FTP_PASS}@${FTP_HOST}
cd ${FTP_PATH}
cache flush
cls -q -1 --date --time-style="+%Y%m%d" > ${LIST}
quit
EOF

# Print obtained list, uncomment for debug
#    echo "File list"
#    cat ${LIST}
# Delete list header, uncomment for debug
#    echo "Delete list"

    # Let's find date to compare
    STORE_DATE=$(date -d "now - ${STORE_DAYS} days" '+%Y%m%d')
    while read LINE; do
        if [[ ${STORE_DATE} -ge ${LINE:0:8} && "${LINE}" != *\/ ]]; then
            echo "rm -f \"${LINE:9}\"" >> ${DELLIST}
            # Print files wich is subject to delete, uncomment for debug
            #echo "${LINE:9}"
        fi
    done < ${LIST}
    # More debug strings
    # echo "Delete list complete"
    # Print notify if list is empty and exit.
    if [ ! -f ${DELLIST}  ] || [ -z "$(cat ${DELLIST})" ]; then
        echo "Delete list doesn't exist or empty, nothing to delete. Exiting"
        exit 0;
    fi
# Connect to ftp and delete files by previously formed list
${LFTP} << EOF
open ${FTP_USER}:${FTP_PASS}@${FTP_HOST}
cd ${FTP_PATH}
$(cat ${DELLIST})
quit
EOF

# Remove temp files
rm ${LIST} ${DELLIST}

}

removeOlderThanDays
    • 0