Home / server / Questions / 25199
In Process
user9406
Asked: 2009-06-14 01:21:45 +0800 CST

Using wget to recursively download whole FTP directories

  • 772

I want to copy all of the files and folders from one host to another. The files on the old host sit at /var/www/html and I only have FTP access to that server, and I can't TAR all the files. Regular connection to the old host through FTP brings me to the /home/admin folder.

I tried running the following command form my new server:

wget -r ftp://username:[email protected]

But all I get is a made up index.html file.

What the right syntax for using wget recursively over FTP?

ftp recursive

13 Answers

  1. Try -m for --mirror

    wget -m ftp://username:[email protected]
    • 148

  2. You have it right, you just need a trailing * on the end:

    wget -r ftp://username:[email protected]/dir/*

    For shared servers, you can use like this:

    wget -r ftp://1.2.3.4/dir/* --ftp-user=username --ftp-password=password

    Because most shared servers has ftp-username something like username@hostname, so, the first wget command not works, and second command works fine.

    • 47

  3. Check the below wget command to download data from FTP recursively.

    wget --user="<user-name>" --password="<password>" -r -np -nH --cut-dirs=1 --reject "index.html*" "<URL to download files>"

    -r: Is for recursively download.

    -np: Is for no parent ascending.

    -nH: Is for disabling creation of directory having name same as URL i.e. abc.xyz.com

    --cut-dirs: Is for ignoring number of parent directories. The value of this option will differ for your command.

    You can check by executing the above command.

    • 10

  4. Besides wget, you may also use lftp in script mode. The following command will mirror the content of a given remote FTP directory into the given local directory, and it can be put into the cron job:

    lftp -c 'open <hostname>; user <username> <password>; mirror -e <remote-src-path> <local-dest-path>; quit'

    It automatically handles recursion into directories and allows specifying the remote source starting directory from to download data from.

    • 8

  5. You can use 'curlftpfs - mount a ftp host as a local directory' and, once mounted, you can use normal file tools like 'cp -r'.

    • 5

  6. Should work try:

    wget -r ftp://ftp:[email protected]/tst/
    • 3

  7. Use -m rather than -r, because of the recursion depth.

    http://www.editcorp.com/Personal/Lars_Appel/wget/wget_2.html#SEC11

    • 3

  8. Use:

    wget -m ftp://192.168.0.1

    and it will mirror all the files and folders.

    • 2

  9. That's the right syntax. Not sure why you aren't getting the expected results.

    As ever there is more than one way to do it. Try ncftp, in particular ncftpget

    • 1

  10. I can understand if you're trying to dump this into cron or something, but why not simply ftp into the server with your normal client and mget *? This might be a quicker path to success.

    • 1