I recently have been backing up my files from a laptop and as part of that have opted to move a folder that at first seemed to be of size 107.6 GB from one hard drive to another. As I was copying, I noticed that a dialog box showed up saying there are far more files than I thought there were -- about 275,000 as opposed to about 160,000. Later, I realized that this has to do with dot files (i.e. hidden files). I wanted to know that when I finished copying that the size and number of files for source and destination matched. Well, I was able to use Nautilus folder dialogs to show that the non-hidden item counts and non-hidden item sizes for source and for destination were the same.
However, I would like to figure out how to arrive at those non-hidden item absolute numbers independently.
My questions are: (1) is there a conceptual explanation for a discrepancy between non-hidden item counts from Nautilus and from e.g. tree; and (2) how do we use existing tools other than Nautilus to get the results for non-hidden item count given by Nautilus?
All (including hidden) items
I have tried tree -a inside the directory we care about, which gives:
28949 directories, 246973 files
This means tree -a gives 275,922 items total.
Nautilus gives 275,923 items total, assuming I have toggled on the "show hidden files" option for Nautilus; this is a match, since it is okay for us to add one to the result from tree -a.
All non-hidden items
I have tried tree inside the directory we care about, which gives:
14860 directories, 146703 files
This means tree gives 161,563 non-hidden items total.
Nautilus gives 160,829 non-hidden items total, assuming I have toggled off the "show hidden files" option for Nautilus; also, even after adding one to the result from tree, the two counts from tree and Nautilus are slightly different.
I understand that Nautilus behaves (when "show hidden files" is off) by excluding dot-folders and dot-files and any (possibly non-dot folder or file) items that are recursively in some dot-folder. For small examples that I have crafted, results from tree and from Nautilus seem to match up.
Again, How do I explain the discrepancy for non-hidden item counts? It is worth noting that I am interested in recursive counts. Also, I am using Ubuntu 20.04 and the file system that the files are at is NTFS. I would provide the experimental data that I am using, but it is not so convenient to pull out non-sensitive data and simultaneously preserve this discrepancy for non-hidden item count.
Also, I have come across another AskUbuntu post that touches on Nautilus and tree, but it doesn't explain why there is a discrepancy when we avoid using the -a option:
Why do I see a difference in file count between Nautilus and find?
Here are some pictures that, admittedly, do not really add any new details.
(I have temporarily named the folder in question "A".)
All (including hidden) items via Nautilus:
All non-hidden items via Nautilus:
All (including hidden) items via
tree -a:
All non-hidden items via
tree:
Again, if there is some other way besides using tree to count non-hidden files recursively in current folder and in subfolders s.t. we match up with Nautilus again, that would also be appreciated.
It seems that some approaches that use find or ls may be susceptible to special characters in folder or file names (e.g. new line). The previously provided AskUbuntu post is relevant for this detail. Also, here is a Unix StackExchange post with a response that mentions a find-based approach s.t. regular files in hidden folders are specifically are considered hidden as well in a count (just as we assume Nautilus and tree do), but the numbers don't match up perfectly, still:
https://unix.stackexchange.com/questions/18227/how-to-recursively-list-all-hidden-files-and-directories
The response that we mention is here:
https://unix.stackexchange.com/a/18228
Thanks,
Brian
Edit: The hard drive that these documents are stored on is a freshly formatted drive. The operating system is a freshly installed instance of Ubuntu Desktop 20.04. As far as I know, I do not have any extra processes interacting with the directory that I would like to measure number of non-hidden items for. Again, by non-hidden item, I mean an item that is not recursively in some dot-folder, assuming the main directory is not hidden. Again, is there a good explanation for why I get different results for Nautilus and for e.g. tree when counting non-hidden documents in a folder and is there a way to independently calculate the non-hidden item counts given by Nautilus?