Home / ubuntu / Questions / 596708
In Process
reducing activity
Asked: 2015-03-15 03:33:22 +0800 CST

Wget: convert-links and avoid redownloading already fetched files?

  • 772

I am downloading data spread among multiple files that don't change once published.

Therefore --timestamping is not good enough, because it is constantly checking whatever resource is changed, which is, in my case, completely pointless.

--no-clobber would fit perfectly. Unfortunately it doesn't work with --convert-links for some reason.

Both --no-clobber and --convert-links were specified, only --convert-links will be used.

I hoped that --backup-converted would help but it changed nothing (it works for --timestamping.)

Why wget --convert-links --backup-converted --no-clobber --wait 1 https://example.com ignores --no-clobber and how could it be fixed?

lubuntu

1 Answers

  1. Here's why it doesn't work: it was disabled on purpose by the developers - http://savannah.gnu.org/bugs/?31781

    As for how to fix it? Try installing a version of wget older than 1.12.1 - which seems to be the version when the aforementioned "bug" was fixed by the developers.

    • 3