Unusual results in regression testing

Ken Moffat ken at kenmoffat.uklinux.net
Tue May 14 14:11:58 PDT 2002


 I've been running automated builds for a couple of months now, and I'm at
last confident that they are building correctly, but I've been seeing a
few odd results which have left me puzzled. I think I now understand what
is happening, and I'd like to share it with the list. (if you don't like
tedious statistics, look away now!)

 I have two partitions set aside for building LFS for testing, and
eventually I'll be building/using some applications as part of the
test. For the moment, all I'm doing is building an LFS mounted at /test1,
then using it to build the same LFS at /test2.

 To compare the results, I've been using `ls -lRo' on each partition,
followed by `diff --side-by-side | less' and doing a visual inspection. At
first I thought everything looked the same (apart from expected
differences in files such as fstab), but then I spotted some differences,
and going back I saw that I'd overlooked these.

 The following turned out to _not_ be the cause

A. Using different kernels to build /test1 and /test2 (but it is still a
good idea to use the same kernel for all of this if you run other than
stable released kernels).

B. Using CFLAGS and CXXFLAGS for gcc, glibc, and binutils. My flags are
conservative (-march=i586 -O2), this didn't make the difference.

The breakthrough came when I used objdump to compare a couple of the
files. It seemed to me that the same code was being generated, but at
different addresses in the file. Eventually I remembered that `ls' should
not be used to report the size of binary files, at least according to a
bearded Welsh kernel hacker.

I had thought that the files were getting smaller in the second build, but
I was mistaken. They were actually abnormally large in my first build! My
host system is, I think, LFS-3.0, so some of the packages are out of date
(particularly vim, glibc, ncurses). What seems to be happening is that the
chapter 5 static programs use the libraries from the host system (they
have to), but the chapter 6 programs which are installed before gcc
(glibc, findutils, gawk, ncurses, vim) seem to also use routines from the
libraries on the host.

What I did in the end was rebuild all of these packages in /usr/local
after I'd finished the second build. This proves that the file sizes are
constant (but see point 2 below).

In my case, the differences are in the `text' segments(?) according to

I've attached details of all the files that differ (a bit over 8K when
uncompressed). If you don't have a console with 128 characters or more per
line, you may find some of these lines a bit long. Sorry.

Has anybody ever come across anything like this, or can anyone see any
flaws in this reasoning ?

There are a couple of other comments in this -

1. I'm getting the impression that the statically linked files are getting
bigger as we move to newer packages. If so, when the book goes to keeping
chapters 5 and 6 separate it might be a good idea to optimise for size in
chapter 5 ?

2. A lot of the glibc binaries are slightly bigger when I install them to
/usr/local. I'm guessing that `strip --strip-debug' is not aggressive
enough (no, I'm _not_ talking about libraries, but about programs such as
gencat.) Typically the difference is an exact 10 bytes more.


Listen to others, even to the dull and ignorant;
     for you too may be a Manager some day.
-- from the BFOH Desidoreplicator.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: regression.notes.bz2
Type: application/octet-stream
Size: 2905 bytes
URL: <http://lists.linuxfromscratch.org/pipermail/lfs-dev/attachments/20020514/3ac2c379/attachment.obj>

More information about the lfs-dev mailing list