cvs commit: hints parallelcompiling.txt

tushar at linuxfromscratch.org tushar at linuxfromscratch.org
Fri Jan 16 19:42:29 PST 2004


tushar      04/01/16 20:42:29

  Modified:    .        parallelcompiling.txt
  Log:
  Updated parallelcompiling.txt
  
  Revision  Changes    Path
  1.5       +209 -148  hints/parallelcompiling.txt
  
  Index: parallelcompiling.txt
  ===================================================================
  RCS file: /home/cvsroot/hints/parallelcompiling.txt,v
  retrieving revision 1.4
  retrieving revision 1.5
  diff -u -u -r1.4 -r1.5
  --- parallelcompiling.txt	15 Sep 2003 13:41:33 -0000	1.4
  +++ parallelcompiling.txt	17 Jan 2004 03:42:29 -0000	1.5
  @@ -1,8 +1,8 @@
  -AUTHOR: Daniel Baumann <danielbaumann at linuxmail.org>
  +AUTHOR: Daniel Baumann <daniel.baumann at panthera-systems.net>
   
  -DATE: 2003-09-09
  +DATE: 2004-01-16
   
  -LICENSE: GNU Free Document License
  +LICENSE: GNU General Public License
   
   SYNOPSIS: How to build LFS using multiple GCC instances.
   
  @@ -11,7 +11,7 @@
   and, if available, on multiple processors.
   
   PREREQUISITES:
  -The command-replacements are applicable for LFS version 4.1. Similar changes
  +The command-replacements are applicable for LFS version 5.0. Similar changes
   will probably work fine on other versions of the book, but your mileage may
   vary.
   
  @@ -21,10 +21,8 @@
   PARALLELCOMPILING
   =================
   
  -This hint is based on the smp-hint written by Adam Greenblatt
  -<agreenbl at aol.com>. now updated and maintained by Daniel Baumann.
  -
  -I would like to thank Adam for allowing to continue his good work.
  +This hint is replaces the smp-hint written by Adam Greenblatt
  +<agreenbl at aol.com>. Now updated and maintained by Daniel Baumann.
   
   
   1. Introduction
  @@ -39,12 +37,12 @@
   
   On multi-processor machines, the GCC instances will be automatically scheduled
   between the processors. This gains speed because it makes use of the additional
  -application-processor(s) (CPU(s) number 1-?) instead of only using the
  -boot-processor (CPU number 0).
  +application-processor(s) (CPU(s) number 1-?) instead of only using the boot-
  +processor (CPU number 0).
   
  -A few packages fail to build in parallel: their makefiles contain implicit
  -dependencies that force them to build serially (arguably, this is a bug in those
  -packages). A couple other packages, notably Glibc and XFree86, use other
  +A few packages fail to build in parallel: Their makefiles contain implicit
  +depencies that force them to build serially (arguably, this is a bug in those
  +packages). A couple of other packages, notably Glibc and XFree86, use other
   makefile trickery in place of '-j'.
   
   Here is a step by step description of the changes needed.
  @@ -55,215 +53,278 @@
   
   As with $LFS, we will use the variable $CC_PARALLEL to specify how many
   instances of GCC to build with. Replace $CC_PARALLEL with a numeric value.
  -A good value seems to be 2 instances per processor. In example for a
  -dual-processor system, I suggest to use CC_PARALLEL=4.
  +A good value seems to be 2 instances per processor. In example for a dual-
  +processor system, I suggest to use CC_PARALLEL=4.
   
  -Machines with Jackson Technologie (also known as Intel Hyper-Threading) should
  -enable it. In fact, every logical processor has its own floating-point-unit
  -(FPU), only the caches are shared. Therefore, you get the nearly doubled
  -performance on FPU-calculations with only a little loss on overhead for managing
  -the shared cache. A good value here is 2 instances per logical processor. In
  -example for a dual-processor system with enabled Jackson Technologie, I suggest
  -to use CC_PARALLEL=8.
  -
  -Generally on very fast systems (2GHz and more), it is also save to use more than
  -2 instances per (logical) processor. I personally use 5 per processor, but I
  -will leave this up to you to choose.
  +Machines with Jackson Technologie (also know as Intel Hyperthreading) should
  +enable it. A good value here is 2 instances per logical processor. In example
  +for a dual-processor system with enabled Jackson Technologie, I suggest to use
  +CC_PARALLEL=8.
  +
  +Generally on fast systems (2GHz and more), it is also save to use more than 2
  +isntances per (logical) processor. I personally use 5, but I leave this up to
  +you to choose.
   
   The rest of the book assumes you have set $CC_PARALLEL as an environment
   variable, in example
   
   	export CC_PARALLEL=4
   
  -If you use the '-j'-flag without a number followed, GNU Make will start an
  +If you use the '-j' switch without a number followed, GNU Make will start an
   infinit number of GCC instances. This often crashs your system and should
   therefore not be used (especially noticed on compiling Glibc which does hang
   nearly everytime the machine).
   
   
  -2. LFS-BOOK
  +2. LFS-Book
   ===========
   
  +2.1 Chapter 2. Important information
  +===================================
  +
  +About $LFS
  +	Add:	export CC_PARALLEL=<value>
  +
  +	Notice:	Please replace <value> with a proper integer according to
  +		the introduction.
   
  -2.1 Chapter 5 - Preparing the LFS system
  -========================================
   
  -Installing Bash-2.05a
  -	No changes - bash does not like to be made in parallel.
  +2.2 Chapter 5. Constructing a temporary system
  +==============================================
   
  -Installing Binutils-2.13.2
  +Installing Binutils-2.14 - Pass 1
   	Change:	make LDFLAGS="-all-static"
   	To:	make -j $CC_PARALLEL LDFLAGS="-all-static"
   
  -Installing Bzip2-1.0.2
  -	Change:	make CC="gcc -static -s"
  -	To:	make -j $CC_PARALLEL CC="gcc -static -s"
  +	Change:	make -C ld LDFLAGS="-all-static" LIB_PATH=/tools/lib
  +	To:	make -j $CC_PARALLEL -C ld LDFLAGS="-all-static" LIB_PATH=/tools/lib
   
  -Installing Diffutils-2.8.1
  +Installing GCC-3.3.1 - Pass 1
  +	Change:	make BOOT_LDFLAGS="-static" bootstrap
  +	To:	make -j $CC_PARALLEL BOOT_LDFLAGS="-static" bootstrap
  +
  +Installing Linux-2.4.22 headers
  +	No changes - we are just copying the header files here.
  +
  +Installing Glibc-2.3.2
  +	Change:	make
  +	To:	make PARALLELMFLAGS="-j $CC_PARALLEL"
  +
  +"Locking in" Glibc
  +	Change:	make -C ld install
  +	To:	make -j $CC_PARALLEL -C ld install
  +
  +Installing Tcl-8.4.4
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Fileutils-4.1
  +Installing Expect-5.39.0
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Findutils-4.1
  +Installing DejaGnu-1.4.3
  +	Change:	make install
  +	To:	make -j $CC_PARALLEL install
  +
  +Installing GCC-3.3.1 - Pass 2
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Gawk-3.1.1
  +Installing Binutils-2.14 - Pass 2
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing GCC-3.2.1
  -	Change:	make BOOT_LDFLAGS="-static" bootstrap
  -	To:	make -j $CC_PARALLEL BOOT_LDFLAGS="-static" bootstrap
  +Installing Gawk-3.1.3
  +	Change:	make
  +	To:	make -j $CC_PARALLEL
  +
  +Installing Coreutils-5.0
  +	Change:	make
  +	To:	make -j $CC_PARALLEL
  +
  +Installing Bzip2-1.0.2
  +	Change:	make PREFIX=/tools install
  +	To:	make -j $CC_PARALLEL PREFIX=/tools install
  +
  +Installing Gzip-1.3.5
  +	Change:	make
  +	To:	make -j $CC_PARALLEL
   
  -Installing Grep-2.5
  +Installing Diffutils-2.8.1
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Gzip-1.2.4a
  -	Change:	make LDFLAGS="-static"
  -	To:	make -j $CC_PARALLEL LDFLAGS="-static"
  +Installing Findutils-4.1.20
  +	Change:	make
  +	To:	make -j $CC_PARALLEL
   
   Installing Make-3.80
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Patch-2.5.4
  +Installing Grep-2.5.1
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Sed-4.0.3
  +Installing Sed-4.0.7
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Sh-utils-2.0
  +Installing Gettext-0.12.1
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Tar-1.13
  +Installing Ncurses-5.3
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Texinfo-4.3
  +	Notice:	Although Ncurses supports parallelcompiling, it often likes to
  +		fail. If this happens, just redo the 'make' without the -j flag.
  +		Otherwise, especially if you are using buildscripts, just use
  +		the normal command.
  +
  +Installing Patch-2.5.4
  +	Change:	make
  +	To:	make -j $CC_PARALLEL
  +
  +Installing Tar-1.13.25
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Textutils-2.1
  +Installing Texinfo-4.6
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Util-linux-2.11y
  +Installing Bash-2.05b
  +	No changes - Bash does not like to be made in parallel.
  +
  +Installing Util-linux-2.12
   	Change:	make -C lib
  -		make -C mount mount umount
  +		make -C mount  mount umount
  +		make -C text-utils  more
   
   	To:	make -j $CC_PARALLEL -C lib
  -		make -j $CC_PARALLEL -C mount mount umount
  +		make -j $CC_PARALLEL -C mount  mount umount
  +		make -j $CC_PARALLEL -C text-utils  more
   
  +Installing Perl-5.8.0
  +	Change:	make perl utilities
  +	To:	make -j $CC_PARALLEL perl utilities
   
  -2.2 Chapter 6 - Installing basic system software
  -================================================
  +
  +2.3 Chapter 6. Installing basic system software
  +===============================================
   
   Entering the chroot environment
  -	Change:	chroot $LFS /static/bin/env -i \
  +	Change: chroot $LFS /tools/bin/env -i \
   			HOME=/root TERM=$TERM PS1='\u:\w\$ ' \
  -			PATH=/bin:/usr/bin:/sbin:/usr/sbin:/static/bin \
  -			/static/bin/bash --login
  +			PATH=/bin:/usr/bin:/sbin:/usr/sbin:/tools/bin \
  +			/tools/bin/bash --login
   
  -	To:	chroot $LFS /static/bin/env -i \
  +	To:	chroot $LFS /tools/bin/env -i CC_PARALLEL=<value> \
   			HOME=/root TERM=$TERM PS1='\u:\w\$ ' \
  -			PATH=/bin:/usr/bin:/sbin:/usr/sbin:/static/bin \
  -			CC_PARALLEL=$CC_PARALLEL \
  -			/static/bin/bash --login
  +			PATH=/bin:/usr/bin:/sbin:/usr/sbin:/tools/bin \
  +			/tools/bin/bash --login
   
  -Installing Linux Kernel-2.4.20
  -	No changes - we are just installing the header files here.
  +	Notice:	Please replace <value> with a proper integer according to
  +		the introduction.
   
  -Installing Man-pages-1.54
  -	No changes - we are just installing the man-pages here.
  +Creating devices (Makedev-1.7)
  +	No changes - we are just creating the device files here.
   
  -Installing Glibc-2.3.1
  -	Change: make
  +Installing Linux-2.4.22 headers
  +	No changes - we are just copying the header files here.
  +
  +Installing Man-pages-1.60
  +	No changes - we are just copying the man-pages here.
  +
  +Installing Glibc-2.3.2
  +	Change:	make
   	To:	make PARALLELMFLAGS="-j $CC_PARALLEL"
   
  -Installing GCC-3.2.1
  -	Change:	make bootstrap
  -	To:	make -j $CC_PARALLEL bootstrap
  +Re-adjusting the toolchain
  +	Change:	make -C ld INSTALL=/tools/bin/install install
  +	To:	make -j $CC_PARALLEL -C ld INSTALL=/tools/bin/install install
   
  -Installing Zlib-1.1.4
  -	Change:	make LIBS="libz.so.1.1.4 libz.a"
  -	To:	make -j $CC_PARALLEL LIBS="libz.so.1.1.4 libz.a"
  +Installing Binutils-2.14
  +	Change:	make tooldir=/usr
  +	To:	make -j $CC_PARALLEL tooldir=/usr
   
  -Installing Findutils-4.1
  -	Change:	make libexecdir=/usr/bin
  -	To:	make -j $CC_PARALLEL libexecdir=/usr/bin
  +Installing GCC-3.3.1
  +	Change:	make
  +	To:	make -j $CC_PARALLEL
   
  -Installing Gawk-3.1.1
  +Installing Coreutils-5.0
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Ncurses-5.3
  +Installing Zlib-1.1.4
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Vim-6.1
  -	Change:	make CPPFLAGS=-DSYS_VIMRC_FILE=\\\"/etc/vimrc\\\"
  -	To:	make -j $CC_PARALLEL \
  -			CPPFLAGS=-DSYS_VIMRC_FILE=\\\"/etc/vimrc\\\"
  +Installing Lfs-Utils-0.3
  +	No changes - LFS-Utils doesn't make use of the -j flag.
   
  -Installling M4-1.4
  +Installing Findutils-4.1.20
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Bison-1.875
  +Installing Gawk-3.1.3
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Less-378
  +Installing Ncurses-5.3
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Groff-1.18.1
  -	No changes - groff does not like to be made in parallel.
  +	Notice:	Although Ncurses supports parallelcompiling, it often likes to
  +		fail. If this happens, just redo the 'make' without the -j flag.
  +		Otherwise, especially if you are using buildscripts, just use
  +		the normal command.
   
  -Installing Textutils-2.1
  +Installing Vim-6.2
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Sed-4.0.3
  +Installing M4-1.4
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Flex-2.5.4a
  +Installing Bison-1.875
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Binutils-2.13.2
  -	Change:	make tooldir=/usr
  -	To:	make -j $CC_PARALLEL tooldir=/usr
  -
  -Installing Fileutils-4.1
  +Installing Less-381
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Sh-utils-2.0
  +Installing Groff-1.19
  +	No changes - Groff does not like to be made in parallel.
  +
  +Installing Sed-4.0.7
  +	Change: make
  +	To:	make -j $CC_PARALLEL
  +
  +Installing Flex-2.5.4a
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Gettext-0.11.5
  +Installing Gettext-0.12.1
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
   Installing Net-tools-1.60
  -	No changes - net-tools do not like to be made in parallel.
  +	No changes - Net-tools do not like to be made in parallel.
  +
  +Installing Inetutils-1.4.2
  +	Change:	make
  +	To:	make -j $CC_PARALLEL
   
   Installing Perl-5.8.0
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Texinfo-4.3
  +Installing Texinfo-4.6
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  @@ -271,25 +332,25 @@
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Automake-1.7.2
  -	Change:	make install
  -	To:	make -j $CC_PARALLEL install
  +Installing Automake-1.7.6
  +	Change:	make
  +	To:	make -j $CC_PARALLEL
   
  -Installing Bash-2.05a
  -	No changes - bash does not like to be made in parallel.
  +Installing Bash-2.05b
  +	No changes - Bash does not like to be made in parallel.
   
  -Installing File-3.39
  -	No changes - file does not like to be made in parallel.
  +Installing File-4.04
  +	No changes - File does not like to be made in parallel.
   
  -Installing Libtool-1.4.3
  +Installing Libtool-1.5
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Bin86-0.16.3
  +Installing Bzip2-1.0.2
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Bzip2-1.0.2
  +Installing Diffutils-2.8.1
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  @@ -301,37 +362,30 @@
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Diffutils-2.8.1
  +Installing E2fsprogs-1.34
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing E2fsprogs-1.32
  +Installing Grep-2.5.1
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Grep-2.5
  +Installing Grub-0.93
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Gzip-1.2.4a
  +Installing Gzip-1.3.5
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Man-1.5k
  -	No changes - man does not like to be made in parallel.
  -
  -Installing Lilo-22.2
  -	No changes - lilo does not like to be made in parallel.
  +Installing Man-1.5m2
  +	No changes - Man does not like to be made in parallel.
   
   Installing Make-3.80
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Modutils-2.4.22
  -	Change:	make
  -	To:	make -j $CC_PARALLEL
  -
  -Installing Netkit-base-0.17
  +Installing Modutils-2.4.25
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  @@ -343,57 +397,60 @@
   	Change:	make LDLIBS=-lncurses
   	To:	make -j $CC_PARALLEL LDLIBS=-lncurses
   
  -Installing Procps-3.1.5
  +Installing Procps-3.1.11
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Psmisc-21.2
  +Installing Psmisc-21.3
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
   Installing Shadow-4.0.3
  -	No changes - shadow does not like to be made in parallel.
  +	Change:	make
  +	To:	make -j $CC_PARALLEL
   
   Installing Sysklogd-1.4.1
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Sysvinit-2.84
  +Installing Sysvinit-2.85
   	Change:	make -C src
   	To:	make -j $CC_PARALLEL -C src
   
  -Installing Tar-1.13
  +Installing Tar-1.13.25
   	Change:	make
   	To:	make -j $CC_PARALLEL
   
  -Installing Util-linux-2.11y
  +Installing Util-linux-2.12
   	Change:	make HAVE_SLN=yes
   	To:	make -j $CC_PARALLEL HAVE_SLN=yes
   
  -Installing Glibc-2.3.1
  -	Change:	make
  -	To:	make PARALLELMFLAGS="-j $CC_PARALLEL"
  +Installing GCC-2.95.3
  +	Change:	make bootstrap
  +	To:	make -j $CC_PARALLEL bootstrap
   
  +Installing LFS-Bootscripts-1.12
  +	No changes - we are just copying the scripts here.
   
  -2.3 Chapter 8 - Making the LFS system bootable
  -==============================================
   
  -Installing Linux-2.4.20
  -	Change:	make bzImage
  -		make modules
  +2.4 Chapter 8. Making the LFS system bootable
  +=============================================
   
  -	To:	make -j $CC_PARALLEL bzImage
  -		make -j $CC_PARALLEL modules
  +Installing Linux-2.4.22
  +	Change:	make CC=/opt/gcc-2.95.3/bin/gcc bzImage
  +		make CC=/opt/gcc-2.95.3/bin/gcc modules
  +
  +	To:	make -j $CC_PARALLEL CC=/opt/gcc-2.95.3/bin/gcc bzImage
  +		make -j $CC_PARALLEL CC=/opt/gcc-2.95.3/bin/gcc modules
   
   
   3. BLFS-BOOK
   ============
   
  +3.1 Chapter 26. X Window Environment
  +====================================
   
  -3.1 Chapter 26 - Installing X
  -=============================
  -
  -Installing XFree86-4.3.0
  +Installing XFree86-4.3.0.1
   	Change: WORLDOPTS="" make World 2>&1 | tee xfree-compile.log &&
   	To:	WORLDOPTS="" make TOPPARALLELMFLAGS="-j $CC_PARALLEL" \
   			World 2>&1 | tee xfree-compile.log &&
  @@ -404,10 +461,14 @@
     * Adam Greenblatt <agreenbl at aol.com> for the smp-hint.
   
   CHANGELOG:
  +[2004-01-16]
  +  * Updated to LFS 5.0.
  +  * Minor typographic mistakes corrected.
  +  * Minor text changes.
   [2003-09-09]
     * New hint format.
   [2003-06-21]
  -  * Typographic misstakes corrected.
  +  * Typographic mistakes corrected.
     * Glibc-command revised.
     * XFree86-command (BLFS) added.
   [2003-06-07]
  
  
  



More information about the hints mailing list