Home / ubuntu / Questions / 1093315
In Process
nutty about natty
Asked: 2018-11-16 14:05:36 +0800 CST

How to detect duplicates of photos (even if the duplicate is of lower resolution) and how to keep only the higher-res copy?

  • 772

Perusing thousands of photos from my smartphone transferred to my computer is a bit of a challenge. Separating the wheat from the chaff is a time consuming late-night endeavour. What makes this particularly challenging is WhatsApp, as it is configured on my phone (by default) sends/receives compressed, lower-res images, rather than full-res. Sharing a photo taken by one's own smartphone would, to my understanding, leave the full-res photo on the phone and additionally create and save a compressed, lower-res and renamed copy of the sent photo in a WhatsApp directory, effectively creating a lower-res duplicate.

As the first iteration of my envisaged workflow I'd like to select unique photos only (over a range of folders and sub-folders). If a given photo exists more than once (either at the same size (same resolution) or at different sizes (different resolutions), I'd like to select only one copy and explicitly only a copy of the highest available resolution (ignoring all lower-res copies).

Building on these two related questions

how could I accomplish this in a jiffy?

tldr;

The upshot / meaning of this exercise would be to include in the initial photo selection also photos received in WhatsApp - as long as these are unique and no higher-res copy exists elsewhere (on my phone / computer).

Further related links / resources:

photo-management duplicate whatsapp

3 Answers

  1. I had the same use-case, and I wrote up a nifty little script to achieve that end.

    Have a look if this Python script helps you too:

    https://github.com/AnirudhKishan/DeleteVisuallyRedundant

    • 2

  2. Some time ago, in Ubuntu 10.04, I was using DupeGuru Picture Edition for that. It was pretty good at finding duplicates in various sizes & resolutions. Seems that since version 4.0 of DupeGuru, the Picture Edition as a separate application is gone, but its functionality has been incorporated into the main application as "Picture Mode". You can try it, however currently only packages for Ubuntu 16.04 and 18.04 are available, there's no build for 20.04.

    The package for 18.04 works under 20.04, but some additional fixes after installation are needed. You need to do the following commands:

    cd /usr/share/dupeguru/core/pe
sudo ln -s _block.cpython-36m-x86_64-linux-gnu.so _block.cpython-38-x86_64-linux-gnu.so
sudo ln -s _cache.cpython-36m-x86_64-linux-gnu.so _cache.cpython-38-x86_64-linux-gnu.so
cd /usr/share/dupeguru/qt/pe
sudo ln -s _block_qt.cpython-36m-x86_64-linux-gnu.so _block_qt.cpython-38-x86_64-linux-gnu.so

    After this, DupeGuru works perfectly.

    • 1

  3. digiKam seems to be the answer for me. Much more powerful than Shotwell:

    Tools > Find Duplicates (Ctrl D) > ... [generate/update Fingerprints] ... > "Similarity range" ~ 95%.

    To find "lower resolution duplicates" the similarity range must not be set to 100%, but fairly close, e.g. 95%.

    • 0