zdiff problem

[Bruce Dubbs]

Bryan Kadzban wrote:
> Bruce Dubbs wrote:
>> The problem code looks like:
>>
>> gzip_status=$(
>>    exec 4>&1
>>        (gzip -cdfq -- "$1" 4>&-; echo $? >&4) 3>&- |
>>      ( (gzip -cdfq -- "$2" 4>&-; echo $? >&4) 3>&- 5<&- </dev/null |
>>         eval "$cmp" /dev/fd/5 -) 5<&0
>> )
> 
> Sheesh, that's complicated!  

Yes, it's bad.  Earlier they have:

for file
do
   test "X$file" = X- || <"$file" || exit 2
done

but file is never defined, so this always fails.  It might be for another shell 
or architecture, but it isn't commented and look screwy.

Let's see if I can decipher...
> 
> Duplicate FD 1 into FD 4.  Then start a subshell with FD 3 closed
> (...but where was 3 opened?  before this?); 

Yes, earlier it does
   exec 3>&1

Why we need to use 3 at all is a mystery to me.

in that subshell, gunzip $1
> to FD 1, closing FD 4 before doing so, and echo the exit status into FD
> 4.  Pipe FD 1 (the decompressed file) into another subshell, and
> duplicate that input stream into FD 5 (hmm; FDs 0 and 5 are both this
> stream...).
> 
> In that second subshell, start yet another subshell with FDs 5 and 3
> (...again, with 3) closed, and with stdin coming from /dev/null (not the
> pipe); in that subshell, gunzip $2 to stdout with FD 4 closed, and echo
> the exit status of this gzip into FD 4 as well.  Pipe the decompressed
> file into "eval $cmp /dev/fd/5 -" (whatever $cmp is; presumably - means
> stdin).

cmp='${DIFF-diff}'

> Both exit statuses are put into gzip_status, along with whatever "$cmp"
> outputs.  This last bit might be the bug, depending on where FD 3 is
> going.  If FD 3 is a duplicate of the output stream of zdiff (e.g. they
> did an "exec 3>&1" earlier), then your fix is correct: the output of
> $cmp needs to be shown to the user, not dumped into the gzip_status
> variable.
> 
> Looks like it's attempting to run "$cmp" (whatever it is) on the two
> streams of gunzip output.  

Yes and keep it all in memory.  They don't want to do any disk IO here.

(/dev/fd/X is the current process's file
> descriptor X; useful for programs like diff and cmp that require file
> names, when you want to compare a stream.)
> 
> It's a giant Y; the two arms of the Y are the gunzip streams, and the
> point in the middle is the $cmp invocation.  The rest is just
> scaffolding to be able to capture the exit status of each gzip, and to
> make sure programs can't get at FDs they shouldn't be able to get at.
> 
> (Does that actually help?  :-) )

Yes.  I hadn't run into a script that closed file streams with things like 3>&- 
before.

>> When I look at /dev/fd, I only have 0 through 3 (and on RH and Ubuntu other 
>> systems too, but none use gzip-1.3.12).
> 
> That's because the ls process only has stdin, stdout, and stderr open;
> FD 3 is a handle to the /proc/self/fd directory itself in this case
> (because you're looking at its contents; opendir() returned 3).

That's a new one for me.

>  (Oh: /dev/fd is a symlink to /proc/self/fd; see /lib/udev/devices.)

That part I knew.  In any case, its fixed up in the book and I did test it,
so it does work.

Thanks for the feedback.

   -- Bruce