Home / ubuntu / Questions / 1110359
Accepted
Charles Green
Asked: 2019-01-17 09:59:52 +0800 CST

Move files from subfolders

  • 772

I've been beating my head against this for a while, but I'm really not a good script writer. Apologies...

I'm running Ubuntu/gnome 18.10, and have a large set of pictures exported from my wifey's mac. The directory structure is:

year1
  (login dir name with spaces) - Month
    Image names
year2
  ...

as in:

2013
  May 4, 2013
    Image1.jpg
    Image2.jpg
  May 5, 2013
    Image 1.jpg
    Image 3.jpg
  June 22, 2013

What I would like, is:

2013
   January
     All the "january" images...
   February
     All the...

I can create the directories easily enough mkdir {January..December} suffices. But I cannot figure out how to walk the ugly directory tree (exported from Mac), move the images, and then delete the ugly directory.

command-line bash

3 Answers

  1. There is an application which can move and rename the files in a single command line - exiftool

    sudo apt install libimage-exiftool-perl

    In my case, the specific command line used was

    exiftool  '-FileName<DateTimeOriginal' -d <path-to-output-dir>/%Y/%m-%B/%Y%m%d_%H%M%S%%-c.%%e . -r

    or

    exiftool  '-FileName<CreateDate' -d <path-to-output-dir>/%Y/%m-%B/%Y%m%d_%H%M%S%%-c.%%e . -r

    depending upon which tag is present in the images. If the requested tag is not present, the application throws a warning and does not move that file, but continues processing. This application also handles multiple images with the same date/time by appending a numeric to the end of the filename.

    I did have several images without exif data, and I was able to add exif data to the images with the command

    exiftool -createdate='2011:12:04 12:00:00' * -overwrite_original

    As an example, I placed some images in a directory ~/aa/test1 and ran the tool, placing the output in ~/aa/test2. The results are below:

    chick@dad:~/aa$ tree .
.
├── test1
│   ├── DSC00018.JPG
│   ├── DSC00022.JPG
│   ├── DSC00024.JPG
│   ├── DSC00025.JPG
│   ├── DSC00026.JPG
│   ├── DSC00028.JPG
│   ├── DSC00031.JPG
│   ├── DSC00033.JPG
│   └── Thumbs.db
└── test2
    └── 2000
        └── 12-December
            ├── 20001222_185523.JPG
            ├── 20001222_200726.JPG
            ├── 20001222_200819.JPG
            ├── 20001222_201205.JPG
            ├── 20001222_201223.JPG
            ├── 20001222_210536.JPG
            ├── 20001222_211858.JPG
            └── 20001222_215950.JPG
    • 2
  2. Best Answer

    Here is such script:

    #!/bin/bash

# The destination where the new directory structure will be created
DEST="/tmp/new-order-pictures/"

MONTHS=('Jan' 'Feb' 'Mar' 'Apr' 'May' 'Jun' 'Jul' 'Aug' 'Sep' 'Oct' 'Nov' 'Dec')

# Walk through the first level directories, located in the current directory and go inside
for year in */
do
    cd "$year"
    # Walk through the months of the year
    for month in "${MONTHS[@]}"
    do
        # Walk through the second level directories
        for dir in */
        do
            # If we have coincidence between the name of the directory and the month
            # go inside, make a new destination directory; ignore character cases^^
            if [[ ${dir^^} =~ ${month^^} ]]
            then
                cd "$dir"
                dest="${DEST}${year}${month}"
                mkdir -p "$dest"
                find . -type f | while IFS= read -r item
                do
                    # Copy the files to the new destination and
                    # add the file's md5sum to its name to prevent files lose
                    filename=$(basename -- "$item")
                    extn="${filename##*.}"
                    name="${filename%.*}"
                    cp "$item" "${dest}/${name}-$(md5sum "$item" | cut -f1 -d' ').${extn}"
                done
                cd ..
            fi
        done
    done
    cd ..
done

    The script should be executed in the first level directory where your images are located. You should tweak the destination directory - DEST="/tmp/new-order-pictures/". This version of the script relies that all files are in directories that contain the name of a month in one way or another. Example of usage:

    user@host:~/Pictures$ tree .
.
├── 2013
│   ├── January 17, 2013
│   │   ├── Image1.jpg
│   │   └── Image 3.jpg
│   ├── January 24, 2013
│   │   └── Image2.jpg
│   ├── January 25, 2013
│   │   └── Image 3.jpg
│   ├── June 22, 2013
│   │   └── image1.jpg
│   ├── May 4, 2013
│   │   └── Image1.jpg
│   └── May 5, 2013
│       ├── Image1.jpg
│       └── Image 2.jpg
└── 2014
    ├── January 17, 2014
    │   ├── Image1.jpg
    │   └── Image 3.jpg
    ├── January 24, 2014
    │   └── Image2.jpg
    ├── January 25, 2014
    │   └── Image 3.jpg
    └── May 5
        ├── Image1.jpg
        └── Image 2.jpg

12 directories, 14 files


user@host:~/Pictures$ order.sh 


user@host:~/Pictures$ tree /tmp/new-order-pictures/
/tmp/new-order-pictures/
├── 2013
│   ├── Jan
│   │   ├── Image1-7b71d9fdfe5b15a2d1a4968c195f93ae.jpg
│   │   ├── Image2-cbf4d36ff84e7ec24c05f8181236e6b8.jpg
│   │   ├── Image 3-0bca5188fd3f3eb470533fdaf0630633.jpg
│   │   └── Image 3-6a83880cae1aa57e19a7c45de7759e68.jpg
│   ├── Jun
│   │   └── image1-adb3bf995f1a25d008f758a7266d7be5.jpg
│   └── May
│       ├── Image1-511d541b35fcb38af8ada18d7961268c.jpg
│       ├── Image1-a66c5863e6986605cb2ca6d622ae72a0.jpg
│       └── Image 2-c34ffc32ce5d3901e1ad89b9fd15a877.jpg
└── 2014
    ├── Jan
    │   ├── Image1-7b71d9fdfe5b15a2d1a4968c195f93ae.jpg
    │   ├── Image2-cbf4d36ff84e7ec24c05f8181236e6b8.jpg
    │   ├── Image 3-0bca5188fd3f3eb470533fdaf0630633.jpg
    │   └── Image 3-6a83880cae1aa57e19a7c45de7759e68.jpg
    └── May
        ├── Image1-511d541b35fcb38af8ada18d7961268c.jpg
        └── Image 2-c34ffc32ce5d3901e1ad89b9fd15a877.jpg

7 directories, 14 files

    In my case the script is named order.sh and it is located in ~/bin, thus I can use it as shell command. In the example you can see the directory structure is changed but the number of files is 14 in both structures.

    Here is another version of the script that use mv instead of cp and will deal also with the files which are not in directories that contain the name of a month. Before running this script it is a good idea to create a backup copy of the original directory structure.

    #!/bin/bash

# The destination where the new directory structure will be created
DEST="/tmp/new-order-pictures/"

MONTHS=('Jan' 'Feb' 'Mar' 'Apr' 'May' 'Jun' 'Jul' 'Aug' 'Sep' 'Oct' 'Nov' 'Dec')

# Walk through the first level directories, located in the current directory and go inside
for year in */
do

    cd "$year"

    # Walk through the months of the year
    for month in "${MONTHS[@]}"
    do
        # Walk through the second level directories
        for dir in */
        do
            # If we have coincidence between the name of the directory and the month
            # go inside, make a new destination directory; ignore character cases^^
            if [[ ${dir^^} =~ ${month^^} ]]
            then

                cd "$dir"
                dest="${DEST}${year}${month}"
                mkdir -p "$dest"

                while IFS= read -r item
                do
                    # Copy the files to the new destination and
                    # add the file's md5sum to its name to prevent files lose
                    filename=$(basename -- "$item")
                    extn="${filename##*.}"
                    name="${filename%.*}"
                    mv "$item" "${dest}/${name}-$(md5sum "$item" | cut -f1 -d' ').${extn}"
                done < <(find . -type f)

                cd ..

            fi

        done

    done

    # Dial with the rest of the files for that $year

    dest="${DEST}${year}other"

    while IFS= read -r item
    do
        mkdir -p "$dest"
        filename=$(basename -- "$item")
        extn="${filename##*.}"
        name="${filename%.*}"
        mv "$item" "${dest}/${name}-$(md5sum "$item" | cut -f1 -d' ').${extn}"
    done < <(find . -type f)

    cd ..

done

    Example of usage:

    user@host:~/Pictures$ tree .
.
├── 2013
│   ├── January 17, 2013
│   │   ├── Image1.jpg
│   │   ├── Image 3.jpg
│   │   └── video 7.mpg
│   ├── January 25, 2013
│   │   └── Image 3.jpg
│   ├── June 22, 2013
│   │   └── image1.jpg
│   └── May 5, 2013
│       ├── Image1.jpg
│       └── Image 2.jpg
└── 2014
    ├── Apr 7
    │   ├── Image1.jpg
    │   └── Image 2.jpg
    ├── Image 2.jpg
    ├── January 11, 2014
    │   ├── Image1.jpg
    │   └── Image 3.jpg
    ├── some other name
    │   └── some other name file inside.jpg
    ├── some other name file inside.jpg
    └── video 1.avi

9 directories, 15 files

user@host:~/Pictures$ order.sh 

user@host:~/Pictures$ tree /tmp/new-order-pictures/
/tmp/new-order-pictures/
├── 2013
│   ├── Jan
│   │   ├── Image1-7b71d9fdfe5b15a2d1a4968c195f93ae.jpg
│   │   ├── Image 3-0bca5188fd3f3eb470533fdaf0630633.jpg
│   │   ├── Image 3-6a83880cae1aa57e19a7c45de7759e68.jpg
│   │   └── video 7-86764d9565469adfb22c8ef4f0b9c04f.mpg
│   ├── Jun
│   │   └── image1-adb3bf995f1a25d008f758a7266d7be5.jpg
│   └── May
│       ├── Image1-511d541b35fcb38af8ada18d7961268c.jpg
│       └── Image 2-c34ffc32ce5d3901e1ad89b9fd15a877.jpg
└── 2014
    ├── Apr
    │   ├── Image1-3c19da25e0e56ef0fc752a9e4f75b190.jpg
    │   └── Image 2-dcc35e86de393a014ac62e8c4390c7e6.jpg
    ├── Jan
    │   ├── Image1-ae34289b0bc5258f286165745ff3c258.jpg
    │   └── Image 3-1724adf2dfcc1d4a0dc50cb38ad2c510.jpg
    └── other
        ├── Image 2-eff5208f7eee6a536e48f9982b918dfb.jpg
        ├── some other name file inside-7d0a68e0b4e9cc3928744cb83f4d1136.jpg
        ├── some other name file inside-c2dd637e94a9025c3e1004d66f59539c.jpg
        └── video 1-c277d93a2427bedf3f0b8ae07427edb9.avi

8 directories, 15 files

    After that you can go inside the destination directory and use the rename command within for loop to deal with the long names:

    # For each directory on the second level
for dir in */*
do
    cd "$dir"
    rename 's/^.*(\.[0-9a-zA-Z]+)$/our $i; sprintf("Image-%03d$1", 1+$i++)/e' *
    cd ..
    cd ..
done

    Example:

    user@host:~/Pictures$ cd /tmp/new-order-pictures/

user@host:/tmp/new-order-pictures$ for dir in */*; do cd "$dir"; rename 's/^.*(\.[0-9a-zA-Z]+)$/our $i; sprintf("Image-%03d$1", 1+$i++)/e' *; cd ..; cd ..; done

user@host:/tmp/new-order-pictures$ tree .
.
├── 2013
│   ├── Jan
│   │   ├── Image-001.jpg
│   │   ├── Image-002.jpg
│   │   ├── Image-003.jpg
│   │   └── Image-004.mpg
│   ├── Jun
│   │   └── Image-001.jpg
│   └── May
│       ├── Image-001.jpg
│       └── Image-002.jpg
└── 2014
    ├── Apr
    │   ├── Image-001.jpg
    │   └── Image-002.jpg
    ├── Jan
    │   ├── Image-001.jpg
    │   └── Image-002.jpg
    └── other
        ├── Image-001.jpg
        ├── Image-002.jpg
        ├── Image-003.jpg
        └── Image-004.avi

8 directories, 15 files

    Or you can change (\.[0-9a-zA-Z]+) with (\.jpg), then on next iteration with (\.mpg) (respectively Image- with Video-), etc. References about this usage of rename:

    • 1

  3. So, I assume you want a step-by-step and simple solution. First, I would address the problem of Whitespace and comma in the folders. I would first cd into the year folder and use rename to first remove Whitespace

    rename "s/ //g" *

    Similarly, remove comma 

    rename "s/,//g" *

    Now that I have all the folders as desired I would create a month based list using 

    ls | grep "January" > January.txt

    Now make folder "January"

    mkdir January

    Then loop thru the list using xargs to copy their contents 

    cat January.txt | xargs -I {} cp -R {}/. ./January/
    • 0