I need to create a list of checksums of the files that are inside a directory, including any subdirectories.
The command that I try to execute is the following:
sha256sum -b *
Usage:
-b = Read in Binary. * = Specifies that you must verify all file extensions.
With the command I get the following output:
sha256sum: test0: Is a directory e3d748fdf10adca15c96d77a38aa0447fa87af9c297cb0b75e314cc313367daf *test1.txt db0c7a354881fe2dd1b45642a68f6a971c7421e8fdffe56ffa7c740111e07274 *test2.txt
Instead of reporting that test0 is a directory, you should also generate the checksum of the content.
Do you recommend always using -b
in any type of file? In what cases should -t
be used?
Is it possible to filter the types of files I want to omit in the verification, without having to add all the files I want to admit? What command should I execute?
I looked for help but I do not find anything related.
You can use
find
to find all files in the directory tree, and let it runsha256sum
. The following command line will create checksums for the files in the current directory and its subdirectories.I don't use the options
-b
and-t
, but if you wish, you can use-b
for all files. The only difference that I notice is the asterisk in front of each file name.TL;DR
Intro
A more complete answer to the one above, which fixes the problem with
find
"finding" files in different orders on different systems.Piping output to file, compare with
diff
Firstly, you probably want to pipe the output to a file for comparison with diff. For this you would use
Then on your other system
Fixing order of files found with
find
by piping tosort
Here I am assuming you are doing something similar to what I required this for - copying files from one system to another over a network and verifying the integrity of those files.
What I found was that the order in which
find
finds files can vary between two systems, even when the OS is "Debian" in both cases.Therefore, one needs to sort the output in the text files.
You can do the
find
andsort
all in one line, while redirecting the output to a file.Other sha/md5 sums
You might want to have an increased level of shasumming. To use the 512 bit version simply do;
Alternatively, 256 bit might be overkill for what you are doing, so do
A complete 1 line command to compare 2 directories with 1 shasum output
Now, if you have many files and do not want to save the output to a file, you could simply shasum the output. To do this, use
The pipe to
sort
is required to ensure the output is sorted before computing the finalsha256sum
. Without this, iffind
finds files in a different order, despite the shasums for each file being correct, the overall shasum will depend on the order.Problem relating to diff output and paths used
You may have some path which looks like
where * are the subdirectories and files you are interested in shasumming. If
A/B/C
are 1 or more directories containing only 1 subfolder you might end up accidentally running your shasum command in the wrong directory, resulting in the followingEven if
sha256sum
=sha256sum2
diff will say the files are different. (Because they are due to the different base directory in the path.)Here is a short python3 code to check the sums line by line, which solves this problem.
I initially wanted to write a shell script to do this, but I got bored trying to figure out how to do it, so went back to python.
This makes me think that actually writing a python code to do the entire thing would have been easier, except for the find command.
Late answer, but for the sake of documentation...
The other answers suggest to call
sha256sum
viafind
and the-exec
option. This has the effect thatsha256sum
is called once for each file, which is a significant overhead for the OS starting processes.A more efficient solution is to convert the
find
results to command line arguments by piping it throughxargs
and callsha256sum
that way.xargs
runssha256sum
once or in large badges if there are too many lines.In case that you have filenames with whitespaces, use the
-print0
flag infind
and-0
flag inxargs
to terminate strings with\0
To include all files in subdirectories use double asterisks:
It requires the globbing enabled. If not, try to enable it:
shopt -s globstar
. See this question for more details.