new hint submission

Zack Winkles winkie at linuxfromscratch.org
Sat Feb 15 19:26:12 PST 2003


fairly bugfixed and ready for semi-mass consumption. attached.
-------------- next part --------------
TITLE:		Native POSIX Threading Library (NPTL)
LFS VERSION:	4.1+
AUTHOR:		Zack Winkles <winkie at linuxfromscratch.org>

SYNOPSIS:
	This hint covers the prerequisites, installation, and configuration
	of the Native POSIX Threading Library, an alternative to the
	linuxthreads, written by Ulrich Drepper.

HINT:

CHANGES
=======

0.1 - Began keeping log of changes.
0.1.1 - Added recommendation to run "make" twice for glibc, and reworded the
        recommendation to test NPTL before install it. Fixed grammar in GRUB
        warning.
0.1.2 - Added patch descriptions.
0.1.3 - Added note about applying patches in the correct order and referenced
        patch explanations from GCC builds instead of specifying them.
0.1.4 - Fixed modules-init-tools and binutils links.
0.1.5 - Added --enable-kernel=2.4.0 and corresponding options.
0.1.6 - Added /bin/pwd symlink for ch6 glibc.
0.1.7 - Made "make check" noted for glibc more specific.
0.1.8 - Added note about building m4 and bison before binutils.


INTRODUCTION
============

The Native POSIX Threading Library (known as NPTL from now on) is a new
threading package designed to eventually replace the linuxthreads package in
glibc. It focuses on standards-compliance, but it also has world-class
performance. Initial benchmarks have shown it to be a minimum of twice as fast
in all cases, and as much as 500x faster in other cases. It is also
backwards-compatible with linuxthreads cases where this compatibility does not
hamper standards-compliance.

In this hint, I assume that you are following the Pure LFS hint found at
http://www.zipworld.com.au/~gschafer/pure_lfs.txt. The way I implement the
commands is by titling the sections that require modification according to their
LFS or "Pure LFS" counterparts. Any modifications are clearly stated and fully
explained. If you want to know what the whole "Pure LFS" thing is all about,
check out the synopsis line of the hint:

  "The current LFS build method is a lot better than it used to be but is still
  somewhat flawed in that not enough emphasis is placed on building a correct
  toolchain. This was recently highlighted by criticism levelled at LFS from
  senior Linux kernel hacker Alan Cox on the kernel mailing list. Here we
  present a new strategy aimed at building a correct toolchain and thus a
  "pure" LFS system. It is our belief that building a correct toolchain is the
  single most important lesson that people reading the LFS book need to know.
  Everything else is secondary."

PROCEDURE
=========

Where to get the software
-------------------------

linux kernel - ftp://ftp.kernel.org/pub/linux/kernel/v2.5/
module-init-tools - ftp://ftp.kernel.org/pub/linux/kernel/people/rusty/modules/
binutils - ftp://ftp.kernel.org/pub/linux/devel/binutils/
NPTL - ftp://people.redhat.com/pub/drepper/nptl/


Where to get the patches
------------------------

The patches used in this hint can be obtained from:

	http://www.linuxfromscratch.org/~winkie/patches/


Patch descriptions
------------------

Some of these patches are dependant upon others, so it is recommended that you
only apply patches in the order listed. The no-fixincl and the specs patches
are the only patches that may be applied anywhere in the sequence. You know you
applied the patches in the wrong order if you get failed "hunks".

gcc-3.2.2-attr-visibility - Adds support for the __attribute__ ((visibility()))
	keyword. Not technically required for TLS, but glibc looks for this as
	well, and currently we don't offer it without this patch. (required)
gcc-3.2.2-label-thunk - Fixes label thunking with use of PIC (place indep. code)
	on the i386 arch. This is required for TLS. (required)
gcc-3.2.2-tls-core - The core changes required for __thread and therefore TLS to
	function. (required)
gcc-3.2.2-tls-g++ - The C++ changes required for TLS to function. This is
	separate from the core TLS functionality so that one can still use the
	gcc-core-3.2.2 tarball without untar'ing unnecessary support.
gcc-3.2.2-bugfixes-core - Miscellaneous bugfixes found in RedHat's SRPM. These
	fix a number of silent miscompilations, a few of which are found in
	glibc itself (!!!). Not required, but highly recommended.
gcc-3.2.2-bugfixes-g++ - The C++ portion of the bugfixes. Again, the reason
	this is separate is so you don't have to untar unnecessary languages.
gcc-3.2.2-hard-reg-sharing - AFAIK, this is an optimization patch that prevents
	register sharing where it would have a negative impact on performance.
gcc-3.2.2-no-fixincl - This prevents GCC from running the fixincl.sh script.
	LFS tries to prevent its running with the install-no-fixedincludes
	target, but this functionality is broken in a number of ways, so use
	this patch instead.
gcc-3.2.2-specs - The Pure LFS specs patch is placed here for convenience.


Chapter 0 - Installing module-init-tools
----------------------------------------

In the next step we install a 2.5.xx series linux kernel. This new kernel comes
with it's own set of complications, namely in the totally redesigned module
support, requiring a new set of utilities to manage it. Build them like so:

	# ./configure --prefix=/
	# make
	# make moveold
	# make install

If you had a (now legacy) /etc/modules.conf file, you should be aware that
module-init-tools now ignores this file completely. Its configuration file is in
/etc/modprobe.conf, which has very similar (if not identical) syntax.


Chapter 0 - Installing Linux 2.5.xx
-----------------------------------

Now that we have proper module support for the new kernel, we should go ahead
and build it. Build the new kernel with the following commands:

	# make mrproper
	# make menuconfig
	# make bzImage
	# make modules

Install it like so:

	# make modules_install
	# cp arch/i386/boot/bzImage /boot/kernel
	# cp System.map /boot/System.map

And, of course, install the new kernel using whatever bootloader you happen to
use, then reboot. (FIXME: Add configuration examples).

BEWARE: In my experience enabling the "Local APIC support on uniprocessors" and
"Machine Check Exception" options create kernels that will not boot.


Chapter 5 - Installing binutils - Pass 1 (static)
-------------------------------------------------

We must use the H J Lu binutils release instead of GNU's offerings because it
has necessary bugfixes and improvements to the TLS (thread-local storage)
support. Grab the latest copy from:

	ftp://ftp.kernel.org/pub/linux/devel/binutils/

After configuring binutils, run the following to allow the actual build to
successfully complete without barfing as it would otherwise:

	# make configure-host


Chapter 5 - Installing GCC - Pass 1 (static)
--------------------------------------------

Apply all the patches marked as required, as well as the no-fixincl patch,
making sure that you apply them in the right order. Instead of using the
install-no-fixincludes target during the install phase, just use regular old
"make install". We don't need to use "make install-no-fixedincludes" anymore
because we are using the no-fixincl patch.

If you run into errors that seem to be syntax errors in files in /usr/include,
just open up the files in your text editor of choice, and modify any instance
of __thread to __thread_old. Be sure to change it back later.


Chapter 5 - Installing kernel headers
-------------------------------------

Follow the commands as they are stated in the Pure LFS hint, but instead of
running "make symlinks", run "make include/asm". The former is no longer
supported with the 2.5.xx kernel series.


Chapter 5 - Installing glibc
----------------------------

Ahhh, here is where all the magic happens. The first step in making this monster
come to life is grabbing glibc from CVS, mainly because NPTL depends on some
features that aren't currently present in the releases. Go ahead and get it
using the following commands:

	# cvs -d:pserver:anoncvs at sources.redhat.com:/cvs/glibc -z3 co libc

Rename the directory and tar it up so we can use it later (Chapter 6).

	# mv libc glibc-`date +%Y%m%d`
	# tar -cf glibc-`date +%Y%m%d`.tar glibc-`date +%Y%m%d`
	# bzip2 glibc-`date +%Y%m%d`

Untar the NPTL sources inside the glibc directory as you would usually do with
linuxthreads. Add the following lines to the configure line:

	--enable-add-ons=nptl --without-cvs --with-tls --with-__thread \
	--enable-kernel=2.4.0

Note: the reason we specify the --enable-kernel=2.4.0 option is because with the
new NPTL-enabled glibc, you will not be able to drop back down to a kernel of
less then 2.5.47, ever, at all. However, anything higher than 2.4.0 causes the
glibc to fail to compile.

After the first "make" command completes, you may want to run it again. Glibc
CVS has a tendency to not do a complete job the first time around, and another
make never hurt anybody, so take the 5 minutes to do it again.

After the builds have completed, you probably want to make sure NPTL is
functioning the way it should be, because (in my experience) NPTL has a nasty
likeliness to make *EVERYTHING* segfault if it wasn't build correctly, so it's
definately a good idea to test it before you install it:

	# make subdirs=nptl check

All tests should pass for all of glibc. If you get an error from tst-atime, just
ignore it. That error is simply due to having the "noatime" option on one of
your filesystems. If the tst-cond2 test fails in nptl, then (for now) just
ignore it. I get the very same error, and AFAIK it's harmless.


Chapter 5 - Install GCC - Pass 2 (shared)
-----------------------------------------

Apply all the patches marked as required, as well as the no-fixincl patch. Also,
if you are planning on building C++ support, be sure to apply the C++ support
patches as well.


Chapter 5 - Install binutils - Pass 2 (shared)
----------------------------------------------

No need to patch binutils, as we use a version that already has the functions
required for the magic that is Pure LFS.


Chapter 6 - Install glibc
-------------------------

Create the following symlink so as to keep glibc contented:

	ln -s /stage1/bin/pwd /bin/pwd

Pass the following options to the configure in addition to what the hint says:

	--enable-add-ons=nptl --without-cvs --with-tls --with-__thread \
	--enable-kernel=2.4.0

See the note about --enable-kernel=2.4.0 in the Chapter 5 - Install glibc
section of this hint for more information.

After building glibc, make sure that NPTL is functioning:

	# make subdirs=nptl check

Preferrably no tests should fail, but if one of two fail (again, not because of
segmentation faults) then everything is probably fine.


Chapter 6 - Install binutils
----------------------------

No patches are required here, just as in all previous builds of binutils. The
build will fail, however, if m4 and bison are not build first. They should put
up no fight during the build, just do the standard configure, make, make install
dance, and be done with it.


Chapter 6 - Install GCC
-----------------------

Be sure to apply all the patches as you have done during the past two builds.


Chapter 6 - Install sysklogd
----------------------------

Sysklogd seems to dislike the 2.5.xx headers. For now the only solution to this
is to use a different (better) alternative: metalog. See it's hint for more
information about its installation and use.


Chapter 6 - Install LILO
------------------------

Lilo seems to dislike the 2.5.xx headers. For now the only solution to this is
to use a different (better) alternative: GRUB. See its hint for more information
about it's installation and use.

WARNING: Do not use optimizations on GRUB, it will most likely fail to build.


Chapter 6 - Install module-init-tools
-------------------------------------

Install module-init tools using the following commands:

	# ./configure --prefix=/
	# make
	# make install

Whenever you would normally use /etc/modules.conf, use /etc/modprobe.conf
instead. The configuration file is very similiar in syntax. So much so that one
can usually use the two interchangably. If for some reason you have an option
that used to work but now does not, use the generate-modprobe.conf to convert
your modules.conf to modprobe.conf.


CREDITS
=======

Ryan Oliver - Devised the whole scheme of Pure LFS, and made it work to a large
              extent. 

Greg Schafer - Wrote most of the documentation for Pure LFS. Helped refine the
               process of creating Pure LFS, and with testing and research of
               said process. Spurred Ryan into action.

Zack Winkles - Wrote NPTL hint.



More information about the hints mailing list