Home / server / Questions / 380020
Accepted
Will Martin
Asked: 2012-04-17 13:30:01 +0800 CST

Detecting upload success/failure in a scripted command-line SFTP session?

  • 772

I am writing a BASH shell script to upload all the files in a directory to a remote server and then delete them. It'll run every few hours via a CRON job.

My complete script is below. The basic problem is that the part that's supposed to figure out whether the file uploaded successfully or not doesn't work. The SFTP command's exit status is always "0" regardless of whether the upload actually succeeded or not.

How can I figure out whether a file uploaded correctly or not so that I can know whether to delete it or let it be?

#!/bin/bash

# First, save the folder path containing the files.
FILES=/home/bob/theses/*

# Initialize a blank variable to hold messages.
MESSAGES=""
ERRORS=""


# These are for notifications of file totals.
COUNT=0
ERRORCOUNT=0

# Loop through the files.
for f in $FILES
do
    # Get the base filename
    BASE=`basename $f`

    # Build the SFTP command. Note space in folder name.
    CMD='cd "Destination Folder"\n'
    CMD="${CMD}put ${f}\nquit\n"


    # Execute it.
    echo -e $CMD | sftp -oIdentityFile /home/bob/.ssh/id_rsa [email protected]

    # On success, make a note, then delete the local copy of the file.
    if [ $? == "0" ]; then
        MESSAGES="${MESSAGES}\tNew file: ${BASE}\n"
        (( COUNT=$COUNT+1 ))

        # Next line commented out for ease of testing
        #rm $f
    fi

    # On failure, add an error message.
    if [ $? != "0" ]; then
        ERRORS="${ERRORS}\tFailed to upload file ${BASE}\n"
        (( ERRORCOUNT=$ERRORCOUNT+1 ))
    fi

done

SUBJECT="New Theses"

BODY="There were ${COUNT} files and ${ERRORCOUNT} errors in the latest batch.\n\n"

if [ "$MESSAGES" != "" ]; then
    BODY="${BODY}New files:\n\n${MESSAGES}\n\n"
fi

if [ "$ERRORS" != "" ]; then
    BODY="${BODY}Problem files:\n\n${ERRORS}"
fi

# Send a notification. 
echo -e $BODY | mail -s $SUBJECT [email protected]

Due to some operational considerations that make my head hurt, I cannot use SCP. The remote server is using WinSSHD on windows, and does not have EXEC privileges, so any SCP commands fail with the message "Exec request failed on channel 0". The uploading therefore has to be done via the interactive SFTP command.

scripting bash sftp

5 Answers

  1. Best Answer

    If you batch your SFTP session, the exit status $? will tell you if it's a failed transfer or not.

    echo "put testfile1.txt" > batchfile.txt
sftp -b batchfile.txt user@host
if [[ $? != 0 ]]; then
  echo "Transfer failed!"
  exit 1
else
  echo "Transfer complete."
fi

    Edited to add: This is something we use in a production script at work, so I'm sure it's valid. We can't use scp because some of our customers are on SCO Unix.

    • 8

  2. It is the tradition of serverfault not to unduly question the preconditions, but I have to ask: is it not possible for you to either 1) mount the remote filesystem via SMB, or 2) use fuse/sshfs instead? (I assume here that the sending machine is a Linux box as you are using bash and ssh.)

    To actually answer your question, I think your problem is simple. Consider:

    quest@wonky:~$ false
quest@wonky:~$ if [ $? != "0" ] ; then echo asdf ; fi
asdf
quest@wonky:~$ if [ $? != "0" ] ; then echo asdf ; fi

    Try instead like this:

    quest@wonky:~$ false
quest@wonky:~$ res=$?
quest@wonky:~$ if [ $res != "0" ] ; then echo asdf ; fi
asdf
quest@wonky:~$ if [ $res != "0" ] ; then echo asdf ; fi
asdf

    To explain: your second if statement "if [ $? != "0" ]; then" tests for the exit status of the latest statement, which is no longer sftp, but the previous if statement.

    Then I wonder, will sftp really exit with non-zero if it has problems uploading files? Cursory tests indicate mine doesn't.

    • 1

  3. Use rsync(.exe) and dig in --remove-source-files & co., don´t even care about failed uploads, he will! ;-)

    Yes you can tie this up with ssh, see --rsh

    • 0

  4. Jarhead

    how about run sftp Batch script on Windows OS to detect success/failure?

    I use %errorlevel% to detect success/failure, sometime the %errorlevel% be returned by 0, but that session is fail and the file is not upload to the server. my script is below

    how can I detect sftp uploaded correctly? THX so much

    @echo off
(echo cd %4
  echo put %2
  echo exit)>sftpScript.txt
sftp -b sftpScript.txt %3
set result=%errorlevel%
if result EQU 0 (
  del sftpScript.txt
  del %2
) else (
  del sftpScript.txt
)
exit %result%
    • 0

  5. ** edited to actually use a batch file To expand on the batch based solution.
    On the copy of sftp I'm running, a batch file exits on an error, even if multiple commands exist beyond the error point

    By the logic originally posted, by checking $?, you're testing the result of the entire batch. If you're ok with removing some of the files that WERE successfully transmitted, then build the batch file with !rm following each put. If the put fails, the !rm won't run

    (for F in *.pgp
do
echo put $F
echo !rm $F
done;echo quit) > Batchfile ; sftp -b Batchfile user@targetServer
    • 0