Ye olde nptl hint loses a few more lines
teemu.tervo at gmx.net
Thu Dec 18 23:29:28 PST 2003
Shit, reply-to isn't set to the mailing list address. Here's what I sent
Scratch the previous one, left in one too many a word.
Teemu Tervo / teemu.tervo at gmx.net
-------------- next part --------------
AUTHOR: Teemu Tervo <teemu.tervo at gmx.net>
LICENSE: The MIT License
SYNOPSIS: Building an LFS system with the NPTL library
DESCRIPTION: Using the new libpthread implementation NPTL (Native POSIX
Threading Library) instead of linuxthreads.
PREREQUISITES: LFS version 5.0+
2. Software requirements
3. Inline installation
The Native POSIX Threading Library is a relatively new threading library
designed to replace the linuxthreads package in Glibc. It has many
advantages over the linuxthreads package, one of which is the focus on
standards-compliance. NPTL never sacrifices standards-compliance under
any circumstances, no matter the gains. Despite this, NPTL has
world-class performance, with thread-spawning speeds as much as 500x the
speed of traditional thread libs.
Despite all these changes, it maintains near-complete
backwards-compatibility with linuxthreads, and generally works perfectly
as a drop-in replacement where the proper compiler infrastructure is
Software requirements (as of December 18th 2003)
1. A kernel with proper support for thread-local storage. This means
you must use a fairly recent 2.6 kernel. RedHat has managed to
backport this functionality into 2.4, but the patch is rather messy
and many parts of it will fail to apply on a vanilla kernel. If
you're going to use 2.4, you're on your own.
If you have no experience with 2.6, I suggest you have a look at
the post-halloween document, which summarizes the changes since 2.4.
I won't go into further detail on using the 2.6.x kernels, since
it is fairly straight-forward, but be sure to replace modutils with
the new module handling package called module-init-tools.
For those minimalists who like to disable everything in the kernel config
it should be noted that CONFIG_FUTEX shouldn't be set to no, futexes are
required for NPTL to function. Also note that you'll need to run
2.6 on both the host and the target system and that you should use
gcc-3.x (3.2.3 is a safe bet) to compile it.
2. A CVS tarball of Glibc. NPTL is no longer distributed in a separate
tarball as it has now been been added to the glibc source tree.
cvs -d :pserver:anoncvs at sources.redhat.com:/cvs/glibc -z3 co libc
mv libc glibc-2.3-`date +%Y%m%d`
tar cjf glibc-2.3-`date +%Y%m%d`.tar.bz2 glibc-2.3-`date +%Y%m%d`
rm -rf glibc-2.3-`date +%Y%m%d`
5. H J Lu's Binutils version 184.108.40.206.5 or newer
Or more precisely, a binutils release with .cfi_rel_offset support. Right
now, this leaves us with either FSF CVS or the HJL branch. As usual,
when using the HJL release, you'll also need to install m4, bison and
flex in chapter 5.
6. Sanitized kernel headers
It is agreed that user space shouldn't use kernel headers, but a
set of sanitized headers such as distributions use instead. In
addition to it not being kosher, stock kernel headers in /usr/include
will break a lot of packages due to changes in 2.6 kernels. Get the
latest Fedora headers from:
The package name is glibc-kernheaders-*. You can use rpm2cpio to first
convert and extract it. All you'll need is glibc-kernheaders.tar.bz2.
I've also made the tarball available in http://ttervo.vectorstar.net/nptl,
but don't expect it to be the latest one available. In the same directory,
you'll also find POSIX compliancy patches for several packages (see notes
after chapter 5 Glibc instructions).
7. Couple of essential patches
You'll need a couple of patches as well. If not otherwise mentioned, they
are expected to be used during both chapters 5 and 6. All are available in
Binutils testsuite will fail one test due to changes introduced in
glibc cvs. The patch is named binutils-VERSION-fix-static-linking.patch.
Util-linux won't compile against 2.6 kernel headers, so we patch it in
chapter 5. We don't install sanitized headers there, since it's easier to
just make this one package work along with the 2.6 headers.
If you're going to use GCC-3.3.1, you need a patch to fix an issue
with static compilation of some programs, caused the same changes that
make binutils tests fail. For 3.3.2 and later this isn't needed. The patch
has been named gcc-3.3.1-fix-static-linking.patch.
This hint was tested using glibc-20031218, binutils-220.127.116.11.7 and
gcc-3.3.2 with both the host and the target systems using linux-2.6.0.
Chapter 5 -- Installing 2.6 kernel headers
When installing the headers, you must use the "make include/asm" command
in place of "make symlinks". The latter no longer works in 2.6 kernels.
Chapter 5 -- Installing Glibc
You won't need to apply the sscanf patch, this has been fixed.
Configure Glibc with the following command:
../glibc-2.3*/configure --prefix=/tools --disable-profile \
--with-headers=/tools/include --with-binutils=/tools/bin \
--enable-add-ons=nptl --enable-kernel=current --without-gd \
During make check, you can expect some failures.
About POSIX conformance
Packages such as coreutils use the _POSIX2_VERSION variable defined in
unistd.h to see which POSIX version to conform to. CVS Glibc has this variable
set to a newer version and this causes problems with many packages and scripts
which use obsoleted syntax like tail -1 instead of tail -n 1. To get around
this, export _POSIX2_VERSION=199209, and make sure it's set at all times.
There are also few other methods, which means this will eventually be a nice
flamewar subject on lfs-dev when the next Glibc version is released. You
1) Patch each package using illegal syntax, like I do.
2) Edit the _POSIX2_VERSION variable directly in unistd.h.
3) Patch coreutils to also require POSIXLY_CORRECT variable set, before
spouting out "You can't do this anymore." This is what RH does.
The nice thing about standards is there are so many ways to avoid them.
Chapter 6 -- Mounting /dev/shm
NPTL's semaphore code and its testsuite uses /dev/shm, hence it's
recommended to mount it not only in chapter 6 but also after booting to the
Chapter 6 -- Installing 2.6 kernel headers
It is agreed that user space shouldn't use kernel headers, but use a
set of sanitized headers such as distributions use instead. Glibc is an
exception to this rule, and it needs to be built against the most
recent headers to make use of new kernel features. So we'll install
them to a prefix other than /usr/include and instruct Glibc to look
for them there.
make mrproper &&
make include/linux/version.h &&
make include/asm &&
mkdir /usr/src/glibcheaders &&
cp -HR include/asm /usr/src/glibcheaders &&
cp -R include/asm-generic /usr/src/glibcheaders &&
cp -R include/linux /usr/src/glibcheaders &&
Chapter 6 -- Installing Glibc
If you try running /tools/lib/libc.so.6, you'll see it won't work due to
some assertions failing. http://ttervo.vectorstar.net/nptl contains a patch,
which fixes this problem. This is merely a cosmetic issue, and is optional.
Use the following command to configure Glibc:
../glibc-2.3*/configure --prefix=/usr --libexecdir=/usr/bin \
--disable-profile --with-headers=/usr/src/glibcheaders \
--enable-add-ons=nptl --enable-kernel=current --without-cvs \
As of this writing, the testsuite should pass without any failures. After make
install, you can remove /usr/src/glibcheaders.
Chapter 6 -- Installing sanitized kernel headers
Right after installing Glibc, untar the glibc-kernheaders tarball
and cd to the extracted usr/include directory.
cp -R asm-i386 /usr/include/asm
cp -R linux /usr/include
Chapter 6 -- Installing modutils
Feel free to skip this package and install module-init-tools in its
place. Modutils is useful only if you want to use binary modules
built for 2.4.
I've been running NPTL on my system for a few months now, and in that period
the only unsolvable problem I've come across is that the beautiful debugger
called Valgrind won't work. As an alternative though, GDB-6.0+ will. In
addition, all binaries linked against linuxthreads' libpthread will fail
If you have an idea for something I should add, want to report a bug,
point out typos and/or bad grammar, tell me if the hint sucks or wish
to offer me penis enlargement products, drop me a line at teemu.tervo at gmx.net.
Thanks to Zack Winkles for the original hint, as well as to Ryan and Greg
for all their help, suggestions and corrections.
* Initial release of rewrite.
* Added patch for 2.4.20 futex support.
* Backed out futex patch. General cleanups.
* Added note about module-init-tools (Thanks DJ Lucas)
* Added warning about NVIDIA.
* Moved to GCC 3.3.
* NVIDIA_GLX works now.
* Devfs notes.
* Patch to make 2.4.20 work w/ NPTL (Thanks YuX).
* Added fix to allow building against old hosts.
* Added patch to fix detection of eh_frame using coreutils.
* Added note about rml's procps.
* Oops. Forgot to put up the location of the GCC patch.
* Another rewrite
* LFS has moved to PLFS
* Don't disable fixincl as it breaks GCC on old hosts
* Removed 2.4 patch.
* Removed any remants of NVIDIA.
* Futexes have been made optional in the latest kernel,
make sure the user enables it.
* Update text for GCC 3.3.
* Clarify warning about futexes.
* Adopted by Teemu
* Edited hint to new format
* Install sanitized kernel headers
* Added note about Drepper's release notes
* Clarified binutils requirement
* Changed linux-2.5 > 2.6, added note about post-halloween doc
* Added note about _POSIX2_VERSION and the Glibc version string
* Added patch that removes a couple of asserts in glibc, thanks to NeoCool
* Removed reference to sysklogd patch, it's not needed
* Removed references to the nptl tarball, as it no longer exists
* Revised glibc cvs checkout commands
* Added the drawbacks section
* Added a nicer assertion patch for glibc
* Added --without-cvs to glibc configure commands
* Added note about /dev/shm
* Added POSIX conformance section, describing the issue a bit more thoroughly
* Install lowlevellock.h in chapter6 glibc
* Revised /dev/shm note
* Added GCC patch to fix static linking
* Added a patch for util-linux to compile with 2.6 headers in chapter 5
* Added binutils patch related to the issue with static linking
* Added glibc makefile patch to fix testsuite
* Fixed the RH kernel headers URL
* Fix for static linking no longer needed with GCC-3.3.2
* Removed lowlevellock.h, nothing needing it works even with it installed
* Changed sanitized headers dl location, they're now over at fedora's site
* Revised the part about futexes in the kernel, it was misleading
* Dropped editing of glibc's version.h, it's unnecessary, outdate and
removing it will prevent me from ever updating it
* Updated post-halloween doc location
* Dropped glibc-DATE-nptl-testsuite.patch, it is no longer needed
* 2.6 kernels are not in test phase anymore
More information about the hints