suggest 'make -j2' for SMP machines?

Dan Nicholson dbn.lists at gmail.com
Tue Jun 5 06:23:32 PDT 2007


On 6/4/07, Greg Schafer <gschafer at zip.com.au> wrote:
> Deskin Miller wrote:
>
> > Is this worth adding to the book?
>
> Most definitely IMHO. Multi-core systems will soon be the norm. It's just
> crazy not to take advantage. There used to exist a "SMP" hint somewhere.
> Maybe this is it:
>
> http://www.linuxfromscratch.org/hints/downloads/files/OLD/parallelcompiling.txt
>
> A lot of seasoned SMP-building folks work on the basis of make -j X+1 ie:
> make -j3 if you have 2 cpus or 2 cores. As a person who has been building
> in parallel for a long time, I strongly disagree with a comment elsewhere
> in this thread about performance plummeting if overutilizing.

I completely agree with what you're saying and I've been using j3 for
quite a while now on a dual core. It's much faster and performance
doesn't plummet, AFAIK. Basically, just how much load do you want to
put on the cpu? If you have two cpus, eight jobs, and they each use
25% of a single CPU, I don't know if that will finish in less time
than two jobs maxing out each CPU.

I'm still not sure it deserves a spot in _this_ book, though since we
don't even point out the gcc optimizations, either. Anyone with a
whiff of experience is going to know about using parallel jobs in
make. But...

> The gotcha with this approach is *ALL* invocations of `make' are affected,
> even those for `make install'. This can lead to some interesting failures.
> But it's easy enough to work around, just add `-j1' to the "known-to-fail"
> cases.

Exactly. You're counting on the Makefile to have well constructed
dependencies. With most automaked projects, this is taken care of
unless there are also custom rules in Makefile.am. In non-automaked
projects, you can almost guarantee that make will race. It would suck
to have a bunch of noob questions about broken builds because of this.

You make a good point, though, about the simplicity of the workaround.
Possible text to appear at an unknown location in the LFS book:

"The `make' utility allows multiple jobs to be run simultaneously when
building packages. This can allow a significant speed increase on
multiple processor systems. This is controlled by passing the -j<jobs>
option to `make'. For instance, you might execute `make -j2 install'
on a dual-core system.

The problem with this approach is that it depends on a well written
Makefile to support multiple simultaneous jobs. If there is a build
error, it is important to remove the jobs option to ensure that this
is not the cause of the error."

--
Dan



More information about the lfs-dev mailing list