On Mon, 24 Feb 2014, Ryan Coleman wrote:

> As much as I find that annoying as someone that occasionally needs to 
> send video projects and has had issues with slicing up archives I 
> understand why. For my photo customers I make zip files of 250 images 
> each - which tends to be about 3GB per pack file.

This reminds me to ask about something I've been thinking of doing with 
photos on web pages and Apache cgi-bin.  I have photos on web pages with 
this kind of layout:

http://genetsim.org/Seoul/20100605_Seoul/

I wrote scripts that compile pages from a few basic elements -- a 
collection of image files in a directory, one file with captions, another 
file with intro info, another with title.

What I'd like to do is add a link that allows the user to click on it and 
download the whole works in a .zip file.  I think the zip file can be 
written on the fly instead of storing a bunch of .zip files on the web 
server, doubling the space used by photos.

Have any of you done something like this using cgi-bin?  It seems doable 
but it has been awhile since I've done anything like this.

One trick I've learned is that there is little value in trying to compress 
.jpg files because they are already compressed.  So if most of your data 
is .jpg, you can put it into a .zip without compression using "zip -0" and 
it will zip/unzip a little faster without being much bigger.  I would 
definitely use "zip -0" in the cgi script.

Here's an example use of zip -0 (discussion at end):

$ wc -c *.jpg | tail -1
48412703 total

$ time -p zip test.zip *.jpg &>/dev/null
real 1.94
user 1.80
sys 0.08

$ wc -c test.zip
48320914 test.zip

$ mkdir test ; cd test

$ time -p unzip ../test.zip &>/dev/null
real 0.37
user 0.29
sys 0.08

$ rm *.jpg ; cd ..

$ time -p zip -0 test.zip *.jpg &>/dev/null
real 0.23
user 0.08
sys 0.15

$ wc -c test.zip
48415325 test.zip

$ cd test

$ time -p unzip ../test.zip &>/dev/null
real 0.31
user 0.21
sys 0.10


user + sys gives CPU time, and that is 1.88 for zip and 0.23 for zip -0, 
so that's a factor of about 8 times the CPU for zip.  The zipped file also 
unzips slightly faster when zip -0 is used, but that small difference 
isn't important because the unzip will be someone else's problem.

Importantly, the file sizes are trivially different:

jpg size:    48412703
zip size:    48320914
zip -0 size: 48415325

The zip -0 file is bigger than the ordinary zip file by almost exactly 1% 
and it's almost identical to the volume of the original .jpg files.

Mike