NPTL hint update
teemu.tervo at gmx.net
Sun Oct 12 08:45:42 PDT 2003
Guess I should write something here, but I don't really have anything to
-------------- next part --------------
AUTHOR: Teemu Tervo <teemu.tervo at gmx.net>
AUTHOR: Original hint by Zack Winkles <winkie at linuxfromscratch.org>
LICENSE: The MIT License
SYNOPSIS: Building an LFS system with the NPTL library
DESCRIPTION: Using the new libpthread implementation Native POSIX
Threading Library (NPTL) instead of linuxthreads.
PREREQUISITES: LFS version 5.0+
2. Software requirements
3. Inline installation
The Native POSIX Threading Library is a relatively new threading library
designed to replace the linuxthreads package in Glibc. It has many
advantages over the linuxthreads package, one of which is the focus on
standards-compliance. NPTL never sacrifices standards-compliance under
any circumstances, no matter the gains. Despite this, NPTL has
world-class performance, with thread-spawning speeds as much as 500x the
speed of traditional thread libs.
Despite all these changes, it maintains near-complete
backwards-compatibility with linuxthreads, and generally works perfectly
as a drop-in replacement where the proper compiler infrastructure is
Software requirements (as of October 9th 2003)
1. A kernel with proper support for thread-local storage. This means
you must use a fairly recent 2.6 kernel. RedHat has managed to
backport this functionality into 2.4, but the patch is rather messy
and many parts of it will fail to apply on a vanilla kernel. If
you're going to use 2.4, you're on your own.
The 2.6 kernel series is still in test phase, but is fairly stable.
If you have no experience with 2.6, I suggest you have a look at
the post-halloween document, which summarizes the changes since 2.4.
I won't go into further detail on using the 2.6.x kernels, since
it is fairly straight-forward, but be sure to replace modutils with
the new module handling package called module-init-tools.
It is imperative that you enable futexes in your kernel. The very
recent kernels have made the building of futexes optional, but it is
required for NPTL to function. Also note that you'll need to run
2.6 on both the host and the target system.
2. A CVS tarball of Glibc. NPTL is no longer distributed in a separate
tarball as it has now been been added to the glibc source tree.
cvs -d :pserver:anoncvs at sources.redhat.com:/cvs/glibc -z3 co libc
mv libc glibc-2.3-`date +%Y%m%d`
tar cjf glibc-2.3-`date +%Y%m%d`.tar.bz2 glibc-2.3-`date +%Y%m%d`
rm -rf glibc-2.3-`date +%Y%m%d`
5. H J Lu's Binutils version 220.127.116.11.5 or newer
Or more precisely, a binutils release with .cfi_rel_offset support. Right
now, this leaves us with either FSF CVS or the HJL branch. As usual,
when using the HJL release, you'll also need to install m4, bison and
flex in chapter 5.
6. Sanitized kernel headers
It is agreed that user space shouldn't use kernel headers, but a
set of sanitized headers such as distributions use instead. In
addition to it not being kosher, stock kernel headers in /usr/include
will break a lot of packages due to changes in 2.6 kernels. Get the
latest RedHat Rawhide headers from:
The package name is glibc-kernheaders-*. You can use rpm2cpio to first
convert and extract it. All you'll need is glibc-kernheaders.tar.bz2.
I've also made the tarball available in http://ttervo.vectorstar.net/nptl,
but don't expect it to be the latest one available. In the same directory,
you'll also find POSIX compliancy patches for several packages (see notes
after chapter 5 Glibc instructions).
This hint was tested using glibc-20031009 and binutils-18.104.22.168.6, with
both the host and the target systems using linux-2.6.0-test7.
Chapter 5 -- Installing 2.6 kernel headers
When installing the headers, you must use the "make include/asm" command
in place of "make symlinks". The latter no longer works in 2.6 kernels.
Chapter 5 -- Installing Glibc
You won't need to apply the sscanf patch, this has been fixed.
Configure Glibc with the following command:
../glibc-2.3*/configure --prefix=/tools --disable-profile \
--with-headers=/tools/include --with-binutils=/tools/bin \
--enable-add-ons=nptl --enable-kernel=current --without-gd \
During make check, you can expect some failures.
About POSIX conformance
Packages such as coreutils use the _POSIX2_VERSION variable defined in
unistd.h to see which POSIX version to conform to. CVS Glibc has this variable
set to a newer version and this causes problems with many packages and scripts
which use obsoleted syntax like tail -1 instead of tail -n 1. To get around
this, export _POSIX2_VERSION=199209, and make sure it's set at all times.
There are also few other methods, which means this will eventually be a nice
flamewar subject on lfs-dev when the next Glibc version is released. You
1) Patch each package using illegal syntax, like I do.
2) Edit the _POSIX2_VERSION variable directly in unistd.h.
3) Patch coreutils to also require POSIXLY_CORRECT variable set, before
spouting out "You can't do this anymore." This is what RH does.
The nice thing about standards is there are so many ways to avoid them.
Chapter 6 -- Mounting /dev/shm
There seems to be a misconception that nothing uses /dev/shm and it's
needless. So let's make it clear, it's used by NPTL's semaphore code and thus
by the testsuite as well, hence should be mounted not only in chapter 6 but
also after booting to the final system.
Chapter 6 -- Installing 2.6 kernel headers
It is agreed that user space shouldn't use kernel headers, but use a
set of sanitized headers such as distributions use instead. Glibc is an
exception to this rule, and it needs to be built against the most
recent headers to make use of new kernel features. So we'll install
them to a prefix other than /usr/include and instruct Glibc to look
for them there.
make mrproper &&
make include/linux/version.h &&
make include/asm &&
mkdir /usr/src/glibcheaders &&
cp -HR include/asm /usr/src/glibcheaders &&
cp -R include/asm-generic /usr/src/glibcheaders &&
cp -R include/linux /usr/src/glibcheaders &&
Chapter 6 -- Installing Glibc
If you looked at /tools/lib, you probably noticed the library is still called
libc-2.3.2.so. If you're anal about these kinds of things, you can edit the
version string in version.h to read "2.3.3 experimental" or whatever feeds
If you try running /tools/lib/libc.so.6, you'll see it won't work due to
some assertions failing. http://ttervo.vectorstar.net/nptl contains a patch,
which fixes this problem. This is merely a cosmetic issue, and is optional.
Use the following command to configure Glibc:
../glibc-2.3*/configure --prefix=/usr --libexecdir=/usr/bin \
--disable-profile --with-headers=/usr/src/glibcheaders \
--enable-add-ons=nptl --enable-kernel=current --without-cvs \
After installation, you can remove /usr/src/glibcheaders.
Chapter 6 -- Installing sanitized kernel headers
Right after installing Glibc, untar the glibc-kernheaders tarball
and cd to the extracted usr/include directory.
cp -R asm-i386 /usr/include/asm
cp -R linux /usr/include
Chapter 6 -- Installing modutils
Feel free to skip this package and install module-init-tools in its
place. Modutils is useful only if you want to use binary modules
built for 2.4.
I've been running NPTL on my system for a few months now, and in that period
the only unsolvable problem I've come across is that the beautiful debugger
called Valgrind won't work. As an alternative though, GDB-6.0+ will. In
addition, all binaries linked against linuxthreads' libpthread will fail
If you have an idea for something I should add, want to report a bug,
point out typos and/or bad grammar, tell me if the hint sucks or wish
to offer me penis enlargement products, drop me a line at teemu.tervo at gmx.net.
* Initial release of rewrite.
* Added patch for 2.4.20 futex support.
* Backed out futex patch. General cleanups.
* Added note about module-init-tools (Thanks DJ Lucas)
* Added warning about NVIDIA.
* Moved to GCC 3.3.
* NVIDIA_GLX works now.
* Devfs notes.
* Patch to make 2.4.20 work w/ NPTL (Thanks YuX).
* Added fix to allow building against old hosts.
* Added patch to fix detection of eh_frame using coreutils.
* Added note about rml's procps.
* Oops. Forgot to put up the location of the GCC patch.
* Another rewrite
* LFS has moved to PLFS
* Don't disable fixincl as it breaks GCC on old hosts
* Removed 2.4 patch.
* Removed any remants of NVIDIA.
* Futexes have been made optional in the latest kernel,
make sure the user enables it.
* Update text for GCC 3.3.
* Clarify warning about futexes.
* Adopted by Teemu
* Edited hint to new format
* Install sanitized kernel headers
* Added note about Drepper's release notes
* Clarified binutils requirement
* Changed linux-2.5 > 2.6, added note about post-halloween doc
* Added note about _POSIX2_VERSION and the Glibc version string
* Added patch that removes a couple of asserts in glibc, thanks to NeoCool
* Removed reference to sysklogd patch, it's not needed
* Removed references to the nptl tarball, as it no longer exists
* Revised glibc cvs checkout commands
* Added the drawbacks section
* Added a nicer assertion patch for glibc (Thanks to Ryan)
* Added --without-cvs to glibc configure commands (Ryan again)
* Added note about /dev/shm (Yeah, it's him once more)
* Added POSIX conformance section, describing the issue a bit more thoroughly
More information about the hints