make: Job Slots
1
1 13.1 Sharing Job Slots with GNU 'make'
1 ======================================
1
11 GNU 'make' has the ability to run multiple recipes in parallel (⇒
Parallel Execution Parallel.) and to cap the total number of parallel
11 jobs even across recursive invocations of 'make' (⇒Communicating
Options to a Sub-'make' Options/Recursion.). Tools that 'make' invokes
1 which are also able to run multiple operations in parallel, either using
1 multiple threads or multiple processes, can be enhanced to participate
1 in GNU 'make''s job management facility to ensure that the total number
1 of active threads/processes running on the system does not exceed the
1 maximum number of slots provided to GNU 'make'.
1
1 GNU 'make' uses a method called the "jobserver" to control the number
1 of active jobs across recursive invocations. The actual implementation
1 of the jobserver varies across different operating systems, but some
1 fundamental aspects are always true.
1
1 First, only command lines that 'make' understands to be recursive
11 invocations of 'make' (⇒How the 'MAKE' Variable Works MAKE
Variable.) will have access to the jobserver. When writing makefiles
1 you must be sure to mark the command as recursive (most commonly by
11 prefixing the command line with the '+' indicator (⇒Recursive Use
of 'make' Recursion.).
1
1 Second, 'make' will provide information necessary for accessing the
1 jobserver through the environment to its children, in the 'MAKEFLAGS'
1 environment variable. Tools which want to participate in the jobserver
1 protocol will need to parse this environment variable, as described in
1 subsequent sections.
1
1 Third, every command 'make' starts has one implicit job slot reserved
1 for it before it starts. Any tool which wants to participate in the
1 jobserver protocol should assume it can always run one job without
1 having to contact the jobserver at all.
1
1 Finally, it's critical that tools that participate in the jobserver
1 protocol return the exact number of slots they obtained from the
1 jobserver back to the jobserver before they exit, even under error
1 conditions. Remember that the implicit job slot should *not* be
1 returned to the jobserver! Returning too few slots means that those
1 slots will be lost for the rest of the build process; returning too many
1 slots means that extra slots will be available. The top-level 'make'
1 command will print an error message at the end of the build if it
1 detects an incorrect number of slots available in the jobserver.
1
1 As an example, suppose you are implementing a linker which provides
1 for multithreaded operation. You would like to enhance the linker so
1 that if it is invoked by GNU 'make' it can participate in the jobserver
1 protocol to control how many threads are used during link. First you
1 will need to modify the linker to determine if the 'MAKEFLAGS'
1 environment variable is set. Next you will need to parse the value of
1 that variable to determine if the jobserver is available, and how to
1 access it. If it is available then you can access it to obtain job
1 slots controlling how much parallelism your tool can use. Once done
1 your tool must return those job slots back to the jobserver.
1
Menu