I have about 2 TB of image data which I want to group into .zip files. The sizes of individual files vary.
I want to put all these files into ZIP archives. Any .zip file cannot exceed 2 GB (1.8 GB for safety) after compression. How many .zip files I make is not a concern.
How can I accomplish this? The key task here, I think, is to figure out how many files of varying sizes can be put into a ZIP archive so that the archive doesn't exceed 2 GB in size without compressing them. And, of course, the whole process has to be automated. I cannot work through about 80,000 pictures manually.
Please suggest ways to achieve this.
Zip has on average approximately 62% compression for most data. Using this fact you can write a script which selects images until their combined size is (1.8 GB / 0.62) = ~2.9 GB, then compress that set, move the originals somewhere, then repeat with another set, and so on.
Or more simply, you can take advantage of the fact that a single Zip file can be split. Then you can make one huge archive and split them into smaller files of whatever size you want as described here: https://superuser.com/questions/336219/how-do-i-split-a-zip-file-into-multiple-segments
Other formats such as 7-Zip offer better compression ratios, so you might consider them if space saving is important.