Cross linking our Ch5 build against a new glibc

Ryan Oliver ryan.oliver at
Thu Jan 16 19:58:25 PST 2003

Greetings LFS'ers

Apologies too all of you in advance, this is a long post...

I may have mentioned in an earlier post about building glibc
first in chapter 5, finally I have a tentative build process
to build Ch 5 against the new glibc.

First, some history

I had an initial build idea I shared with Greg Schafer
after reading his analysis of the glibc/gcc build
( from build glibc twice thread ) and noting the trauma
caused by the glibc upgrade attempted to build everything
linked against the new glibc in Ch 5. This was to try to
define a build sequence that would not depend on what glibc
your host system was running.

My first attempts at building Chapter 5 against a pre-
built glibc-2.3.1 by attempting to pass multitudes of
linker flags defining -Wl,--dynamic-linker, -Wl,-rpath-link
 -isystem etc. were doomed.

What I needed was a dedicated cross-linker.

Why go to the bother of building a dedicated cross-linker I hear
you say?

Being able to cross link everything onto the new glibc
opens up quite a few advantages ( apart from maybe silencing
some of our LFS critics :-) ) and lends itself to
interesting build possibilities, namely the ability to
create our Ch 5 libraries and binaries (used for our final clean
build in Ch 6) built against the glibc we are migrating too.
Another advantage is we minimise the amount of code brought
in from the host operating system during the build process
(nothing is linked static or shared against the host OS ).

The idea being, you build
a) binutils current version static, to keep glibc happy
   during configure when we point at it.
b) gcc-3.2.x somewhere ( home directory ), if not the
   current system compiler (reqd to build glibc)
c) glibc-2.3.1
d) a new linker which by default points at the new glibc.

   ( After working through ld's build, noticed the ld/Makefile
     after configure contains the var
     to be set for creating a cross linker.
     (read the appropriate section in the makefile)
     This sets the default path with which ld will
     look for )

d) Gcc and all the binutils linked against the new glibc

It was after a week of battling down this path and not
getting very far (still had linking problems) that I ran into
this post by Bruno Haible on Google

"how to test the new glibc-2.2's UTF8 locales"

I'd completely missed ld's ldscripts (DOH!), and the fact
that you can change the location gcc looks for in the config/$ARCH/linux.h ( ie hardcode
--dynamic-linker )

As a side note, if you have read through the above link,
it looks like whatever Bruno had to do with LD_SO_CACHE and
LD_SO_CONF defs in glibc-x.x/elf/ are not needed nowadays,
ldconfig defaults to looking in $sysconfdir ($eprefix/etc)


How do we do build LFS with these tools?
I have a script at the end of this posting.

If you follow the build I lay out below
a) Everything we use to build our shared binaries and
   libraries in chapter 6 will ALREADY be built and linked
   against the glibc we are migrating too.

   This avoids glibc migration issues now and in the future
   ( No nasty surprises when entering chroot :-) )

b) We can build everything shared. It isn't necessary that
   we build all our binaries static in chapter 5 (though you
   still can if you want too) as we will still have access to
   the shared libs we create when we chroot.

   See the use of
      mount --bind ${LFS}/static /static
   below, if everything is built with --prefix=/static it'll
   all be where its expected after we chroot and everything
   should still link up correctly.

c) The build order in chapter 5 is the same as in chapter 6.
   Both should be based on LFS-4.0 CH 6 build order to avoid lib
   dependency issues
   (except of course for binutils, glibc and gcc :-) )

   You would continue on from the end of this script basically
   doing a chapter 6 build except with --prefix=/static
   ( or alternatively up until you have a working bash )

d) Chapter 6 is a repeat of the order we created
   our initial toolchain, except this time nothing
   really needs to be modified as we want our linkers
   etc searching the standard directories and include dirs

   When it comes to chrooting and doing Chapter 6,
   After we enter chroot we
   1) ensure PATH=/static/bin:/static/sbin
      and LD_LIBRARY_PATH and LD_PRELOAD are unset

   2) Install the kernel headers in /usr/include

   3)  mount --bind /static/include /usr/include
       mount --bind /static/lib /lib
      so it appears we have glibc already installed in
      the right place

   4) Build binutils static with no modifications/patches

   5) Build glibc --prefix=/usr ... --with-binutils=/usr/bin

      Before doing make install,
        unmount /usr/include
        unmount /lib
      create /etc/, containing /lib and /usr/lib

   6) build gcc --prefix=/usr --with-ld=/usr/bin/ld --with-as=/usr/bin/as ...
      without the modifications I specify below

   7) Rebuild binutils using CC=/usr/bin/gcc ../configure ...
      Again, no modifications required

   8) export PATH=/bin:/sbin:/usr/bin:/usr/sbin:${PATH}

   Then continue on with the rest of the packages

( If required I will write up a build order for Ch5 + Ch6 but I
  gather most of you lfs-dev'ers will know where I am going with this )

Anyway, here's the build for creating the glibc-2.3.1 toolchain in /static.

If you try it please let me know how you go
I'll have more on Monday actually specifying the rest (including CH6), but
want to try a couple more builds first (from a couple of different base

Would be interested to know if someone out there working on a different arch
such as arm,ppc etc has any success with this as well
( This will depend on whether or not the --dynamic-linker defs vary too much
  between architectures in gcc-x.x/elf/config/ARCH/linux.h )

Ryan Oliver

Peter Harding and Associates Pty. Ltd
Eml: ryan.oliver at
Ph:  +613 9641 2222
Fax: +613 9641 2200


# Ryan Oliver (PHA), 2003-01-16
# (ryan.oliver at
# Builds a complete toolchain for compiling and
# linking against a different glibc version.
# Don't just run this script blindly, you will have to
# patch gcc with the reqd LFS patches for the gcc version
# (will add that at a later date... maybe :-) ),
# also the versions of the packages here could vary from
# what you are using so check...


# Set LFS to suit your build tree
export LFS=/u1

#Assuming you already have built your $LFS tree

mkdir /static

# We will bind mount $LFS/static to /static, so our toolchain
# will look in the right spot when we enter chroot

mount --bind ${LFS}/static /static

# Set SRC for your sourcdir, TARBALLS for where you store
#  your tarballs

export SRC=${LFS}/usr/src
export TARBALLS=${LFS}/usr/tarballs

# **** NOTE ****
# This script assumes your system compiler is already gcc-3.2+,
# If it is not, build it somewhere (home directory) and ensure you
# set  CC=/path/to/gcc-3.2.x when configuring packages.
# Otherwise just continue on..

# Binutils pass 1 - static
# Will be using latest binutils for creating glibc
# ( Keeps configure script happy )

cd ${SRC}
export BINUTILS_VER=2.13

tar -xvzf ${TARBALLS}/binutils-${BINUTILS_VER}.tar.gz
mkdir binutils-${BINUTILS_VER}-init-build
cd binutils-${BINUTILS_VER}-init-build

./binutils-${BINUTILS_VER}/configure --prefix=/static --disable-nls &&
   make LDFLAGS=-all-static &&
   make install

# Kernel Headers
# Now install kernel headers into /static/include ( for glibc )

cd ${SRC}
export KERNEL_VER=2.4.20
tar -xvzf ${TARBALLS}/linux-${KERNEL_VER}.tar.gz
mv linux linux-${KERNEL_VER}
ln -s linux-${KERNEL_VER} linux

cd ${SRC}/linux
make mruxthreads-${GLIBC_VER}.tar.bz2

cd ${SRC}/glibc-${GLIBC_VER}

# apply the $PERL hack

cd malloc
if [ ! -e Makefile-DIST ]; then
   mv Makefile Makefile-DIST
cat Makefile-DIST | sed 's/\$(PERL)/\/usr\/bin\/perl/g' > Makefile
cd ..

# create for new glibc
cat > /static/etc/ << "EOF"

mkdir ${SRC}/glibc-${GLIBC_VER}-init-build
cd ${SRC}/glibc-${GLIBC_VER}-init-build
./glibc-${GLIBC_VER}/configure --prefix=/static --disable-profile \
   --enable-add-ons --with-headers=/static/include \

# Optional
# make check
make install

# Binutils pass 2 - build cross-linker
#                 ( Will only be rebuilding ld. )
# Will be setting LIB_PATH in ld/Makefile to "/static/lib"
# and also set /static/lib first in the search order for
# the ldscripts.
# This will force this ld to link against our new glibc

cd ${SRC}
mkdir binutils-${BINUTILS_VER}-build
cd binutils-${BINUTILS_VER}-build
./binutils-${BINUTILS_VER}/configure --prefix=/static --disable-nls

cd ld

if [ ! -f Makefile-ORIG ]; then
   mv Makefile Makefile-ORIG

# in Makefile so ld and ldscripts point at /static/lib

cat Makefile-ORIG | sed 's/^LIB_PATH = /LIB_PATH = \/static\/lib/g' | \
sed 's/stringify.sed$/stringify.sed Makefile/g' > Makefile

make install

# Gcc build
# Now we build cc and c++ which use our new glibc
# ( when linking will use /static/ld, will use
#   /static/lib/ as its dynamic linker
#  and will search /static/include first for headers )

cd ${SRC}
export GCC_VER=3.2.1
tar -xvzf ${TARBALLS}/gcc-$GCC_VER.tar.gz
cd gcc-${GCC_VER}

# **** NOTE ****
# First apply any required patches ( no-fixincludes etc )
# ---- HERE ----
# (NOT SHOWN HERE - I'm too lazy ;-) )

# We want our new gcc to use our uild &&
cd ${SRC}/gcc-${GCC_VER}-init-build &&

./gcc-$GCC_VER/configure --prefix=/static --enable-languages=c,c++ \
   --disable-nls --enable-shared --with-ld=/static/bin/ld \
   --with-as=/static/bin/as \

# Now we have to modify gcc/Makefile
# I am not totally sure if what I have done here is perfectly correct or
# not, the aim is to get gcc to always search /static/include for headers
# first. Any input here from the reader would be appreciated.
# Regardless this appears to work, though we may get compiler warnings later
# complaining about
#  cc1: warning: changing search order for system directory "/static/include"
#  cc1: warning:   as it has already been specified as a non-system directory
# Will have to look into this, but its harmless
# Firstly,
# we modify NATIVE_SYSTEM_HEADER_DIR to be /static/include not /usr/include
# ( this was originally done so fixincludes only looked at our new headers,
#   not the system headers. Its a moot point with theibdir\)\/gcc-lib\/\\\" \\$/-DSTANDARD_EXEC_PREFIX=\\\"\$\(libdir\)\/gcc-lib\/\\\" \\\
  -DSTANDARD_INCLUDE_DIR=\\\"\/static\/include\\\" \\/g' > Makefile

# buils as per normal
make bootstrap
make install
ln -s gcc /static/bin/cc

# Binutils pass 3 - build all against new glibc
# Now for our fiinal step, we will rebuild binutils from our
# shiny new toolchain ( setting CC="/static/bin/gcc" ),
# so it too is built against the latest glibc
# Again we Will be setting LIB_PATH in ld/Makefile to "/static/lib"
# and also set /static/lib first in the search order for
# the ldscripts to force this ld to link against our new glibc.

cd ${SRC}
rm -rf binutils-${BINUTILS_VER}-build
mkdir binutils-${BINUTILS_VER}-build
cd binutils-${BINUTILS_VER}-build

CC=/static/bin/gcc ../binutils-${BINUTILS_VER}/configure --prefix=/static --disable-nls

cd ld
if [ ! -f Makefile-ORIG ]; then
   mv Makefile Makefile-ORIG

# in Makefile so ld and ldscripts point at /static/lib

cat Makefile-ORIG | sed 's/^LIB_PATH = /LIB_PATH = \/static\/lib/g' | \
sed 's/stringify.sed$/stringify.sed Makefile/g' > Makefile
cd binutils-${BINUTILS_VER}-build

# Make everything

# Now we have to edit the main Makefile to comment out the section
# where PEALLY_SET_LIB_PATH is set, so LD_LIBRARY_PATH does not get set.
# This avoids issues when installing libiberty.a (apparently)

if [ ! -f Makefile-ORIG ]; then
   cp Makefile Makefile-ORIG

cat Makefile-ORIG | \
   sed 's/^  \$\(RPATH_ENVVAR\)/#  \$\(RPATH_ENVVAR\)/g' > Makefile

# Install
make install

cd ${SRC}

# Now all we do is set PATH to have /static/bin and /static/sbin
# and anything we compile will be built against our new glibc from
# our complete toolchain

export PATH=/static/bin:/static/sbin:${PATH}

# Job Done :-)

Unsubscribe: send email to listar at
and put 'unsubscribe lfs-dev' in the subject header of the message

More information about the lfs-dev mailing list