2.1. Dependency Lines

As mentioned in the introduction, in any system, there are dependencies between the files that make up the system. For instance, in a program made up of several C source files and one header file, the C files will need to be re-compiled should the header file be changed. For a document of several chapters and one macro file, the chapters will need to be reprocessed if any of the macros changes. These are dependencies and are specified by means of dependency lines in the makefile.

On a dependency line, there are targets and sources, separated by a one- or two-character operator. The targets “depend” on the sources and are usually created from them. Any number of targets and sources may be specified on a dependency line. All the targets in the line are made to depend on all the sources. Targets and sources need not be actual files, but every source must be either an actual file or another target in the makefile. If you run out of room, use a backslash at the end of the line to continue onto the next one.

Any file may be a target and any file may be a source, but the relationship between the two (or however many) is determined by the “operator” that separates them. Three types of operators exist: one specifies that the datedness of a target is determined by the state of its sources, while another specifies other files (the sources) that need to be dealt with before the target can be re-created. The third operator is very similar to the first, with the additional condition that the target is out-of-date if it has no sources. These operations are represented by the colon, the exclamation point and the double-colon, respectively, and are mutually exclusive. Their exact semantics are as follows:

Enough words, now for an example. Take that C program I mentioned earlier. Say there are three C files (a.c, b.c and c.c) each of which includes the file defs.h. The dependencies between the files could then be expressed as follows:

program         : a.o b.o c.o

a.o b.o c.o     : defs.h

a.o             : a.c

b.o             : b.c

c.o             : c.c

You may be wondering at this point, where a.o, b.o and c.o came in and why they depend on defs.h and the C files do not. The reason is quite simple: program cannot be made by linking together .c files—it must be made from .o files. Likewise, if you change defs.h, it is not the .c files that need to be re-created, it is the .o files. If you think of dependencies in these terms—which files (targets) need to be created from which files (sources)—you should have no problems.

An important thing to notice about the above example, is that all the .o files appear as targets on more than one line. This is perfectly all right: the target is made to depend on all the sources mentioned on all the dependency lines. For example, a.o depends on both defs.h and a.c.

The order of the dependency lines in the makefile is important: the first target on the first dependency line in the makefile will be the one that gets made if you do not say otherwise. That is why program comes first in the example makefile, above.

Both targets and sources may contain the standard C-Shell wildcard characters ({, }, *, ?, [, and ]), but the non-curly-brace ones may only appear in the final component (the file portion) of the target or source. The characters mean the following things:

2.2. Shell Commands

“Is not that nice,” you say to yourself, “but how are files actually ``re-created'', as he likes to spell it?” The re-creation is accomplished by commands you place in the makefile. These commands are passed to the Bourne shell (better known as /bin/sh) to be executed and are expected to do what is necessary to update the target file (PMake does not actually check to see if the target was created. It just assumes it is there).

Shell commands in a makefile look a lot like shell commands you would type at a terminal, with one important exception: each command in a makefile must be preceded by at least one tab.

Each target has associated with it a shell script made up of one or more of these shell commands. The creation script for a target should immediately follow the dependency line for that target. While any given target may appear on more than one dependency line, only one of these dependency lines may be followed by a creation script, unless the :: operator was used on the dependency line.

If the double-colon was used, each dependency line for the target may be followed by a shell script. That script will only be executed if the target on the associated dependency line is out-of-date with respect to the sources on that line, according to the rules I gave earlier. I'll give you a good example of this later on.

To expand on the earlier makefile, you might add commands as follows:

program         : a.o b.o c.o
        cc a.o b.o c.o -o program

a.o b.o c.o     : defs.h
a.o             : a.c
       cc -c a.c

b.o             : b.c
       cc -c b.c

c.o             : c.c
       cc -c c.c

Something you should remember when writing a makefile is, the commands will be executed if the target on the dependency line is out-of-date, not the sources. In this example, the command cc -c a.c will be executed if a.o is out-of-date. Because of the : operator, this means that should a.c or defs.h have been modified more recently than a.o, the command will be executed (a.o will be considered out-of-date).

Remember how I said the only difference between a makefile shell command and a regular shell command was the leading tab? I lied. There is another way in which makefile commands differ from regular ones. The first two characters after the initial whitespace are treated specially. If they are any combination of @ and -, they cause PMake to do different things.

In most cases, shell commands are printed before they are actually executed. This is to keep you informed of what is going on. If an @ appears, however, this echoing is suppressed. In the case of an echo command, say

echo Linking index

it would be rather silly to see

echo Linking index
Linking index

so PMake allows you to place an @ before the command to prevent the command from being printed:

@echo Linking index

The other special character is the -. In case you did not know, shell commands finish with a certain “exit status”. This status is made available by the operating system to whatever program invoked the command. Normally this status will be 0 if everything went ok and non-zero if something went wrong. For this reason, PMake will consider an error to have occurred if one of the shells it invokes returns a non-zero status. When it detects an error, PMake's usual action is to abort whatever it is doing and exit with a non-zero status itself (any other targets that were being created will continue being made, but nothing new will be started. PMake will exit after the last job finishes). This behavior can be altered, however, by placing a - at the front of a command (e.g. -mv index index.old), certain command-line arguments, or doing other things, to be detailed later. In such a case, the non-zero status is simply ignored and PMake keeps chugging along.

Because all the commands are given to a single shell to execute, such things as setting shell variables, changing directories, etc., last beyond the command in which they are found. This also allows shell compound commands (like for loops) to be entered in a natural manner. Since this could cause problems for some makefiles that depend on each command being executed by a single shell, PMake has a -B flag (it stands for backwards-compatible) that forces each command to be given to a separate shell. It also does several other things, all of which I discourage since they are now old-fashioned.

A target's shell script is fed to the shell on its (the shell's) input stream. This means that any commands, such as ci that need to get input from the terminal will not work right – they will get the shell's input, something they probably will not find to their liking. A simple way around this is to give a command like this:

ci $(SRCS) < /dev/tty

This would force the program's input to come from the terminal. If you cannot do this for some reason, your only other alternative is to use PMake in its fullest compatibility mode. See “Compatibility” in Chapter 4.

2.3. Variables

PMake, like Make before it, has the ability to save text in variables to be recalled later at your convenience. Variables in PMake are used much like variables in the shell and, by tradition, consist of all upper-case letters (you do not have to use all upper-case letters. In fact there is nothing to stop you from calling a variable @^&$%$. Just tradition). Variables are assigned-to using lines of the form:

VARIABLE = value

appended-to by:

VARIABLE += value

conditionally assigned-to (if the variable is not already defined) by:

VARIABLE ?= value

and assigned-to with expansion (i.e. the value is expanded (see below) before being assigned to the variable—useful for placing a value at the beginning of a variable, or other things) by:

VARIABLE := value

Any whitespace before value is stripped off. When appending, a space is placed between the old value and the stuff being appended.

The final way a variable may be assigned to is using:

VARIABLE != shell-command

In this case, shell-command has all its variables expanded (see below) and is passed off to a shell to execute. The output of the shell is then placed in the variable. Any newlines (other than the final one) are replaced by spaces before the assignment is made. This is typically used to find the current directory via a line like:

CWD             != pwd

For example, to set the variable CFLAGS to the string -I/sprite/src/lib/libc -O, you would place a line:

CFLAGS = -I/sprite/src/lib/libc -O

in the makefile and use the word $(CFLAGS) wherever you would like the string -I/sprite/src/lib/libc -O to appear. This is called variable expansion.

For compatibility, Make's style of variable expansion will be used if you invoke PMake with any of the compatibility flags (-V, -B or -M. The -V flag alters just the variable expansion). There are two different times at which variable expansion occurs: when parsing a dependency line, the expansion occurs immediately upon reading the line. If any variable used on a dependency line is undefined, PMake will print a message and exit. Variables in shell commands are expanded when the command is executed. Variables used inside another variable are expanded whenever the outer variable is expanded (the expansion of an inner variable has no effect on the outer variable. For example, if the outer variable is used on a dependency line and in a shell command, and the inner variable changes value between when the dependency line is read and the shell command is executed, two different values will be substituted for the outer variable).

Variables come in four flavors, though they are all expanded the same and all look about the same. They are (in order of expanding scope):

• Local variables.

• Command-line variables.

• Global variables.

• Environment variables.

The classification of variables does not matter much, except that the classes are searched from the top (local) to the bottom (environment) when looking up a variable. The first one found wins.

$(OBJS)         : $(.PREFIX).c

pmake "CFLAGS = -I/sprite/src/lib/libc -O"

#undef variable

OBJS            = a.o b.o c.o

program         : $(OBJS)
        cc $(.ALLSRC) -o $(.TARGET)

$(OBJS)         : defs.h

a.o             : a.c
        cc -c a.c

b.o             : b.c
        cc -c b.c

c.o             : c.c
        cc -c c.c

2.4. Comments

Comments in a makefile start with a # character and extend to the end of the line. They may appear anywhere you want them, except in a shell command (though the shell will treat it as a comment, too). If, for some reason, you need to use the # in a variable or on a dependency line, put a backslash in front of it. PMake will compress the two into a single #.

2.5. Parallelism

PMake was specifically designed to re-create several targets at once, when possible. You do not have to do anything special to cause this to happen (unless PMake was configured to not act in parallel, in which case you will have to make use of the -L and -J flags (see below)), but you do have to be careful at times.

There are several problems you are likely to encounter. One is that some makefiles (and programs) are written in such a way that it is impossible for two targets to be made at once. The program xstr, for example, always modifies the files strings and x.c. There is no way to change it. Thus you cannot run two of them at once without something being trashed. Similarly, if you have commands in the makefile that always send output to the same file, you will not be able to make more than one target at once unless you change the file you use. You can, for instance, add a $$$$ to the end of the file name to tack on the process ID of the shell executing the command (each $$ expands to a single $, thus giving you the shell variable $$). Since only one shell is used for all the commands, you will get the same file name for each command in the script.

The other problem comes from improperly-specified dependencies that worked in Make because of its sequential, depth-first way of examining them. While I do not want to go into depth on how PMake works (look in Chapter 4 if you are interested), I will warn you that files in two different levels of the dependency tree may be examined in a different order in PMake than they were in Make. For example, given the makefile:

a               :

b c b           : d

PMake will examine the targets in the order c, d, b, a. If the makefile's author expected PMake to abort before making c if an error occurred while making b, or if b needed to exist before c was made, (s)he will be sorely disappointed. The dependencies are incomplete, since in both these cases, c would depend on b. So watch out.

Another problem you may face is that, while PMake is set up to handle the output from multiple jobs in a graceful fashion, the same is not so for input. It has no way to regulate input to different jobs, so if you use the redirection from /dev/tty I mentioned earlier, you must be careful not to run two of the jobs at once.

2.6. Writing and Debugging a Makefile

Now you know most of what is in a Makefile, what do you do next? There are two choices: use one of the uncommonly-available makefile generators or write your own makefile (I leave out the third choice of ignoring PMake and doing everything by hand as being beyond the bounds of common sense).

When faced with the writing of a makefile, it is usually best to start from first principles: just what are you trying to do? What do you want the makefile finally to produce? To begin with a somewhat traditional example, let's say you need to write a makefile to create a program, expr, that takes standard infix expressions and converts them to prefix form (for no readily apparent reason). You have got three source files, in C, that make up the program: main.c, parse.c, and output.c. Harking back to my pithy advice about dependency lines, you write the first line of the file:

expr            : main.o parse.o output.o

because you remember expr is made from .o files, not .c files. Similarly for the .o files you produce the lines:

main.o          : main.c

parse.o         : parse.c

output.o        : output.c

main.o parse.o output.o : defs.h

Great. You have now got the dependencies specified. What you need now is commands. These commands, remember, must produce the target on the dependency line, usually by using the sources you have listed. You remember about local variables? Good, so it should come to you as no surprise when you write:

expr            : main.o parse.o output.o
	cc -o $(.TARGET) $(.ALLSRC)

Why use the variables? If your program grows to produce postfix expressions too (which, of course, requires a name change or two), it is one fewer place you have to change the file. You cannot do this for the object files, however, because they depend on their corresponding source files and defs.h, thus if you said:

cc -c $(.ALLSRC)

you will get (for main.o):

cc -c main.c defs.h

which is wrong. So you round out the makefile with these lines:

main.o          : main.c
	cc -c main.c

parse.o         : parse.c
	cc -c parse.c

output.o        : output.c
	cc -c output.c

The makefile is now complete and will, in fact, create the program you want it to without unnecessary compilations or excessive typing on your part. There are two things wrong with it, however (aside from it being altogether too long, something I will address in Chapter 3):

1. The string main.o parse.o output.o is repeated twice, necessitating two changes when you add postfix (you were planning on that, were not you?). This is in direct violation of de Boor's First Rule of writing makefiles:

   Anything that needs to be written more than once should be placed in a variable. I cannot emphasize this enough as being very important to the maintenance of a makefile and its program.

2. There is no way to alter the way compilations are performed short of editing the makefile and making the change in all places. This is evil and violates de Boor's Second Rule, which follows directly from the first:

   Any flags or programs used inside a makefile should be placed in a variable so they may be changed, temporarily or permanently, with the greatest ease.

The makefile should more properly read:

OBJS            = main.o parse.o output.o

expr            : $(OBJS)
	$(CC) $(CFLAGS) -o $(.TARGET) $(.ALLSRC)

main.o          : main.c
	$(CC) $(CFLAGS) -c main.c

parse.o         : parse.c
	$(CC) $(CFLAGS) -c parse.c

output.o        : output.c
	$(CC) $(CFLAGS) -c output.c

$(OBJS)         : defs.h

Alternatively, if you like the idea of dynamic sources mentioned in Section 2.3.1, you could write it like this:

OBJS            = main.o parse.o output.o

expr            : $(OBJS)
	$(CC) $(CFLAGS) -o $(.TARGET) $(.ALLSRC)

$(OBJS)         : $(.PREFIX).c defs.h
	$(CC) $(CFLAGS) -c $(.PREFIX).c

These two rules and examples lead to de Boor's First Corollary: Variables are your friends.

Once you have written the makefile comes the sometimes-difficult task of making sure the darn thing works. Your most helpful tool to make sure the makefile is at least syntactically correct is the -n flag, which allows you to see if PMake will choke on the makefile. The second thing the -n flag lets you do is see what PMake would do without it actually doing it, thus you can make sure the right commands would be executed were you to give PMake its head.

When you find your makefile is not behaving as you hoped, the first question that comes to mind (after “What time is it, anyway?”) is “Why not?” In answering this, two flags will serve you well: -d m and “-p 2”. The first causes PMake to tell you as it examines each target in the makefile and indicate why it is deciding whatever it is deciding. You can then use the information printed for other targets to see where you went wrong. The “-p 2” flag makes PMake print out its internal state when it is done, allowing you to see that you forgot to make that one chapter depend on that file of macros you just got a new version of. The output from “-p 2” is intended to resemble closely a real makefile, but with additional information provided and with variables expanded in those commands PMake actually printed or executed.

Something to be especially careful about is circular dependencies. For example:

a         : b

b         : c d

d         : a

In this case, because of how PMake works, c is the only thing PMake will examine, because d and a will effectively fall off the edge of the universe, making it impossible to examine b (or them, for that matter). PMake will tell you (if run in its normal mode) all the targets involved in any cycle it looked at (i.e. if you have two cycles in the graph (naughty, naughty), but only try to make a target in one of them, PMake will only tell you about that one. You will have to try to make the other to find the second cycle). When run as Make, it will only print the first target in the cycle.

2.7. Invoking PMake

PMake comes with a wide variety of flags to choose from. They may appear in any order, interspersed with command-line variable assignments and targets to create. The flags are as follows:

Several flags may follow a single -. Those flags that require arguments take them from successive parameters. For example:

pmake -fDnI server.mk DEBUG /chip2/X/server/include

will cause PMake to read server.mk as the input makefile, define the variable DEBUG as a global variable and look for included makefiles in the directory /chip2/X/server/include.

2.8. Summary

A makefile is made of four types of lines:

• Dependency lines

• Creation commands

• Variable assignments

• Comments, include statements and conditional directives

A dependency line is a list of one or more targets, an operator (:, ::, or !), and a list of zero or more sources. Sources may contain wildcards and certain local variables.

A creation command is a regular shell command preceded by a tab. In addition, if the first two characters after the tab (and other whitespace) are a combination of @ or -, PMake will cause the command to not be printed (if the character is @) or errors from it to be ignored (if -). A blank line, dependency line or variable assignment terminates a creation script. There may be only one creation script for each target with a : or ! operator.

Variables are places to store text. They may be unconditionally assigned-to using the = operator, appended-to using the += operator, conditionally (if the variable is undefined) assigned-to with the ?= operator, and assigned-to with variable expansion with the := operator. The output of a shell command may be assigned to a variable using the != operator. Variables may be expanded (their value inserted) by enclosing their name in parentheses or curly braces, preceded by a dollar sign. A dollar sign may be escaped with another dollar sign. Variables are not expanded if PMake does not know about them. There are seven local variables: .TARGET, .ALLSRC, .OODATE, .PREFIX, .IMPSRC, .ARCHIVE, and .MEMBER. Four of them (.TARGET, .PREFIX, .ARCHIVE, and .MEMBER) may be used to specify “dynamic sources”. Variables are good. Know them. Love them. Live them.

Debugging of makefiles is best accomplished using the -n, -d m, and -p 2 flags.

3.1. Transformation Rules

As you know, a file's name consists of two parts: a base name, which gives some hint as to the contents of the file, and a suffix, which usually indicates the format of the file. Over the years, as UNIX has developed, naming conventions, with regard to suffixes, have also developed that have become almost as incontrovertible as Law. E.g. a file ending in .c is assumed to contain C source code; one with a .o suffix is assumed to be a compiled, relocatable object file that may be linked into any program; a file with a .ms suffix is usually a text file to be processed by Troff with the -ms macro package, and so on. One of the best aspects of both Make and PMake comes from their understanding of how the suffix of a file pertains to its contents and their ability to do things with a file based solely on its suffix. This ability comes from something known as a transformation rule. A transformation rule specifies how to change a file with one suffix into a file with another suffix.

A transformation rule looks much like a dependency line, except the target is made of two known suffixes stuck together. Suffixes are made known to PMake by placing them as sources on a dependency line whose target is the special target .SUFFIXES. E.g.:

.SUFFIXES   : .o .c
.c.o        :
	$(CC) $(CFLAGS) -c $(.IMPSRC)

The creation script attached to the target is used to trans form a file with the first suffix (in this case, .c) into a file with the second suffix (here, .o). In addition, the target inherits whatever attributes have been applied to the transformation rule. The simple rule given above says that to transform a C source file into an object file, you compile it using cc with the -c flag. This rule is taken straight from the system makefile. Many transformation rules (and suffixes) are defined there, and I refer you to it for more examples (type pmake -h to find out where it is).

There are several things to note about the transformation rule given above:

1. The .IMPSRC variable. This variable is set to the “implied source” (the file from which the target is being created; the one with the first suffix), which, in this case, is the .c file.

2. The CFLAGS variable. Almost all of the transformation rules in the system makefile are set up using variables that you can alter in your makefile to tailor the rule to your needs. In this case, if you want all your C files to be compiled with the -g flag, to provide information for dbx, you would set the CFLAGS variable to contain -g (CFLAGS = -g) and PMake would take care of the rest.

To give you a quick example, the makefile in Section 2.3.4 could be changed to this:

OBJS            = a.o b.o c.o
program         : $(OBJS)
	 $(CC) -o $(.TARGET) $(.ALLSRC)
$(OBJS)         : defs.h

The transformation rule I gave above takes the place of the 6 lines [1]:

a.o             : a.c
	cc -c a.c
b.o             : b.c
	cc -c b.c
c.o             : c.c
	cc -c c.c

Now you may be wondering about the dependency between the .o and .c files – it is not mentioned anywhere in the new makefile. This is because it is not needed: one of the effects of applying a transformation rule is the target comes to depend on the implied source. That's why it is called the implied source.

For a more detailed example. Say you have a makefile like this:

a.out           : a.o b.o
	$(CC) $(.ALLSRC)

and a directory set up like this:

total 4
-rw-rw-r--  1 deboor        34 Sep  7 00:43 Makefile
-rw-rw-r--  1 deboor       119 Oct  3 19:39 a.c
-rw-rw-r--  1 deboor       201 Sep  7 00:43 a.o
-rw-rw-r--  1 deboor        69 Sep  7 00:43 b.c

While just typing pmake will do the right thing, it is much more informative to type pmake -d s. This will show you what PMake is up to as it processes the files. In this case, PMake prints the following:

Suff_FindDeps (a.out)
     using existing source a.o
     applying .o -> .out to "a.o"
Suff_FindDeps (a.o)
     trying a.c...got it
     applying .c -> .o to "a.c"
Suff_FindDeps (b.o)
     trying b.c...got it
     applying .c -> .o to "b.c"
Suff_FindDeps (a.c)
     trying a.y...not there
     trying a.l...not there
     trying a.c,v...not there
     trying a.y,v...not there
     trying a.l,v...not there
Suff_FindDeps (b.c)
     trying b.y...not there
     trying b.l...not there
     trying b.c,v...not there
     trying b.y,v...not there
     trying b.l,v...not there
--- a.o ---
cc  -c a.c
--- b.o ---
cc  -c b.c
--- a.out ---
cc a.o b.o

Suff_FindDeps is the name of a function in PMake that is called to check for implied sources for a target using transformation rules. The transformations it tries are, naturally enough, limited to the ones that have been defined (a transformation may be defined multiple times, by the way, but only the most recent one will be used). You will notice, however, that there is a definite order to the suffixes that are tried. This order is set by the relative positions of the suffixes on the .SUFFIXES line – the earlier a suffix appears, the earlier it is checked as the source of a transformation. Once a suffix has been defined, the only way to change its position in the pecking order is to remove all the suffixes (by having a .SUFFIXES dependency line with no sources) and redefine them in the order you want. (Previously-defined transformation rules will be automatically redefined as the suffixes they involve are re-entered.) Another way to affect the search order is to make the dependency explicit. In the above example, a.out depends on a.o and b.o. Since a transformation exists from .o to .out, PMake uses that, as indicated by the using existing source a.o message.

The search for a transformation starts from the suffix of the target and continues through all the defined transformations, in the order dictated by the suffix ranking, until an existing file with the same base (the target name minus the suffix and any leading directories) is found. At that point, one or more transformation rules will have been found to change the one existing file into the target.

For example, ignoring what's in the system makefile for now, say you have a makefile like this:

.SUFFIXES       : .out .o .c .y .l
.l.c            :
	lex $(.IMPSRC)
	mv lex.yy.c $(.TARGET)
.y.c            :
	yacc $(.IMPSRC)
	mv y.tab.c $(.TARGET)
.c.o            :
	cc -c $(.IMPSRC)
.o.out          :
	cc -o $(.TARGET) $(.IMPSRC)

and the single file jive.l. If you were to type pmake -rd ms jive.out, you would get the following output for jive.out:

Suff_FindDeps (jive.out)
     trying jive.o...not there
     trying jive.c...not there
     trying jive.y...not there
     trying jive.l...got it
     applying .l -> .c to "jive.l"
     applying .c -> .o to "jive.c"
     applying .o -> .out to "jive.o"

and this is why: PMake starts with the target jive.out, figures out its suffix (.out) and looks for things it can transform to a .out file. In this case, it only finds .o, so it looks for the file jive.o. It fails to find it, so it looks for transformations into a .o file. Again it has only one choice: .c. So it looks for jive.c and, as you know, fails to find it. At this point it has two choices: it can create the .c file from either a .y file or a .l file. Since .y came first on the .SUFFIXES line, it checks for jive.y first, but can not find it, so it looks for jive.l and, lo and behold, there it is. At this point, it has defined a transformation path as follows:

.l  ->  .c  ->  .o  -> .out

and applies the transformation rules accordingly. For completeness, and to give you a better idea of what PMake actually did with this three-step transformation, this is what PMake printed for the rest of the process:

Suff_FindDeps (jive.o)
     using existing source jive.c
     applying .c -> .o to "jive.c"
Suff_FindDeps (jive.c)
     using existing source jive.l
     applying .l -> .c to "jive.l"
Suff_FindDeps (jive.l)
Examining jive.l...modified 17:16:01 Oct 4, 1987...up-to-date
Examining jive.c...non-existent...out-of-date
--- jive.c ---
lex jive.l
... meaningless lex output deleted ...
mv lex.yy.c jive.c
Examining jive.o...non-existent...out-of-date
--- jive.o ---
cc -c jive.c
Examining jive.out...non-existent...out-of-date
--- jive.out ---
cc -o jive.out jive.o

One final question remains: what does PMake do with targets that have no known suffix? PMake simply pretends it actually has a known suffix and searches for transformations accordingly. The suffix it chooses is the source for the .NULL target mentioned later. In the system makefile, .out is chosen as the “null suffix” because most people use PMake to create programs. You are, however, free and welcome to change it to a suffix of your own choosing. The null suffix is ignored, however, when PMake is in compatibility mode (see Chapter 4).

3.2. Including Other Makefiles

Just as for programs, it is often useful to extract certain parts of a makefile into another file and just include it in other makefiles somehow. Many compilers allow you say something like:

#include "defs.h"

to include the contents of defs.h in the source file. PMake allows you to do the same thing for makefiles, with the added ability to use variables in the filenames. An include directive in a makefile looks either like this:

#include <file>

or this:

#include "file"

The difference between the two is where PMake searches for the file: the first way, PMake will look for the file only in the system makefile directory (or directories) (to find out what that directory is, give PMake the -h flag). The system makefile directory search path can be overridden via the -m option. For files in double-quotes, the search is more complex:

1. The directory of the makefile that's including the file.

2. The current directory (the one in which you invoked PMake).

3. The directories given by you using -I flags, in the order in which you gave them.

4. Directories given by .PATH dependency lines (see Chapter 4).

5. The system makefile directory.

in that order.

You are free to use PMake variables in the filename – PMake will expand them before searching for the file. You must specify the searching method with either angle brackets or double-quotes outside of a variable expansion. I.e. the following:

SYSTEM    = <command.mk>

#include $(SYSTEM)

will not work.

3.3. Saving Commands

There may come a time when you will want to save certain commands to be executed when everything else is done. For instance: you are making several different libraries at one time and you want to create the members in parallel. Problem is, ranlib is another one of those programs that can not be run more than once in the same directory at the same time (each one creates a file called __.SYMDEF into which it stuffs information for the linker to use. Two of them running at once will overwrite each other's file and the result will be garbage for both parties). You might want a way to save the ranlib commands til the end so they can be run one after the other, thus keeping them from trashing each other's file. PMake allows you to do this by inserting an ellipsis (“...”) as a command between commands to be run at once and those to be run later.

So for the ranlib case above, you might do this:

lib1.a          : $(LIB1OBJS)
	rm -f $(.TARGET)
	ar cr $(.TARGET) $(.ALLSRC)
	...
	ranlib $(.TARGET)

lib2.a          : $(LIB2OBJS)
	rm -f $(.TARGET)
	ar cr $(.TARGET) $(.ALLSRC)
	...
	ranlib $(.TARGET)

This would save both

ranlib $(.TARGET)

commands until the end, when they would run one after the other (using the correct value for the .TARGET variable, of course).

Commands saved in this manner are only executed if PMake manages to re-create everything without an error.

3.4. Target Attributes

PMake allows you to give attributes to targets by means of special sources. Like everything else PMake uses, these sources begin with a period and are made up of all upper-case letters. There are various reasons for using them, and I will try to give examples for most of them. Others you will have to find uses for yourself. Think of it as “an exercise for the reader”. By placing one (or more) of these as a source on a dependency line, you are “marking the target(s) with that attribute”. That is just the way I phrase it, so you know.

Any attributes given as sources for a transformation rule are applied to the target of the transformation rule when the rule is applied.

system          : init a.fasl b.fasl c.fasl
	for i in $(.ALLSRC);
	do
		echo -n '(load "' >> input
		echo -n ${i} >> input
		echo '")' >> input
	done
	echo '(dump "$(.TARGET)")' >> input
	lisp < input

a.fasl          : a.l init COMPILE
b.fasl          : b.l init COMPILE
c.fasl          : c.l init COMPILE
COMPILE         : .USE
	echo '(compile "$(.ALLSRC)")' >> input
init            : .EXEC
	echo '(load-system)' > input

prog1           : $(PROG1OBJS) prog2 MAKEINSTALL
prog2           : $(PROG2OBJS) .INVISIBLE MAKEINSTALL

program         : $(OBJS) libraries
	cc -o $(.TARGET) $(.ALLSRC)

libraries       : lib1.a lib2.a lib3.a lib4.a .JOIN
	ranlib $(.OODATE)

lib1.a          : $(LIB1OBJS) MAKELIB
lib2.a          : $(LIB2OBJS) MAKELIB

MAKELIB         : .USE
	rm -f $(.TARGET)
	ar cr $(.TARGET) $(.ALLSRC)
	...
	ranlib $(.TARGET)

3.5. Special Targets

As there were in Make, so there are certain targets that have special meaning to PMake. When you use one on a dependency line, it is the only target that may appear on the left-hand-side of the operator. As for the attributes and variables, all the special targets begin with a period and consist of upper-case letters only. I will not describe them all in detail because some of them are rather complex and I will describe them in more detail than you will want in Chapter 4. The targets are as follows:

.SUFFIXES       : .bitmap
.PATH.bitmap    : /usr/local/X/lib/bitmaps
.INCLUDES       : .bitmap

cc $(.INCLUDES) -c xprogram.c

In addition to these targets, a line of the form:

attribute : sources

applies the attribute to all the targets listed as sources.

3.6. Modifying Variable Expansion

Variables need not always be expanded verbatim. PMake defines several modifiers that may be applied to a variable's value before it is expanded. You apply a modifier by placing it after the variable name with a colon between the two, like so:

${VARIABLE:modifier}

Each modifier is a single character followed by something specific to the modifier itself. You may apply as many modifiers as you want – each one is applied to the result of the previous and is separated from the previous by another colon.

There are seven ways to modify a variable's expansion, most of which come from the C shell variable modification characters:

In addition, the System V style of substitution is also supported. This looks like:

$(VARIABLE:search-string=replacement)

It must be the last modifier in the chain. The search is anchored at the end of each word, so only suffixes or whole words may be replaced.

3.7. More Exercises

Exercise 3.1

You have got a set programs, each of which is created from its own assembly-language source file (suffix .asm). Each program can be assembled into two versions, one with error-checking code assembled in and one without. You could assemble them into files with different suffixes (.eobj and .obj, for instance), but your linker only understands files that end in .obj. To top it all off, the final executables must have the suffix .exe. How can you still use transformation rules to make your life easier (Hint: assume the errorchecking versions have ec tacked onto their prefix)?

Exercise 3.2

Assume, for a moment or two, you want to perform a sort of “indirection” by placing the name of a variable into another one, then you want to get the value of the first by expanding the second somehow. Unfortunately, PMake does not allow constructs like:

$($(FOO))

What do you do? Hint: no further variable expansion is performed after modifiers are applied, thus if you cause a $ to occur in the expansion, that is what will be in the result.

4.1. Search Paths

PMake supports the dispersal of files into multiple directories by allowing you to specify places to look for sources with .PATH targets in the makefile. The directories you give as sources for these targets make up a “search path”. Only those files used exclusively as sources are actually sought on a search path, the assumption being that anything listed as a target in the makefile can be created by the makefile and thus should be in the current directory.

There are two types of search paths in PMake: one is used for all types of files (including included makefiles) and is specified with a plain .PATH target (e.g. .PATH : RCS), while the other is specific to a certain type of file, as indicated by the file's suffix. A specific search path is indicated by immediately following the .PATH with the suffix of the file. For instance:

.PATH.h         : /sprite/lib/include /sprite/att/lib/include

would tell PMake to look in the directories /sprite/lib/include and /sprite/att/lib/include for any files whose suffix is .h.

The current directory is always consulted first to see if a file exists. Only if it cannot be found there are the directories in the specific search path, followed by those in the general search path, consulted.

A search path is also used when expanding wildcard characters. If the pattern has a recognizable suffix on it, the path for that suffix will be used for the expansion. Otherwise the default search path is employed.

When a file is found in some directory other than the current one, all local variables that would have contained the target's name (.ALLSRC, and .IMPSRC) will instead contain the path to the file, as found by PMake. Thus if you have a file ../lib/mumble.c and a makefile like this:

.PATH.c         : ../lib
mumble          : mumble.c
	$(CC) -o $(.TARGET) $(.ALLSRC)

the command executed to create mumble would be cc -o mumble ../lib/mumble.c. (as an aside, the command in this case is not strictly necessary, since it will be found using transformation rules if it is not given. This is because .out is the null suffix by default and a transformation exists from .c to .out. Just thought I would throw that in). If a file exists in two directories on the same search path, the file in the first directory on the path will be the one PMake uses. So if you have a large system spread over many directories, it would behoove you to follow a naming convention that avoids such conflicts.

Something you should know about the way search paths are implemented is that each directory is read, and its contents cached, exactly once – when it is first encountered – so any changes to the directories while PMake is running will not be noted when searching for implicit sources, nor will they be found when PMake attempts to discover when the file was last modified, unless the file was created in the current directory. While people have suggested that PMake should read the directories each time, my experience suggests that the caching seldom causes problems. In addition, not caching the directories slows things down enormously because of PMake's attempts to apply transformation rules through non-existent files – the number of extra file-system searches is truly staggering, especially if many files without suffixes are used and the null suffix is not changed from .out.

4.2. Archives and Libraries

UNIX and Sprite allow you to merge files into an archive using the ar command. Further, if the files are relocatable object files, you can run ranlib on the archive and get yourself a library that you can link into any program you want. The main problem with archives is they double the space you need to store the archived files, since there is one copy in the archive and one copy out by itself. The problem with libraries is you usually think of them as -lm rather than /usr/lib/libm.a and the linker thinks they are out-of-date if you so much as look at them.

PMake solves the problem with archives by allowing you to tell it to examine the files in the archives (so you can remove the individual files without having to regenerate them later). To handle the problem with libraries, PMake adds an additional way of deciding if a library is out-of-date: if the table of contents is older than the library, or is missing, the library is out-of-date.

A library is any target that looks like -lname or that ends in a suffix that was marked as a library using the .LIBS target. .a is so marked in the system makefile. Members of an archive are specified as archive(member[member...]). Thus libdix.a(window.o) specifies the file window.o in the archive libdix.a. You may also use wildcards to specify the members of the archive. Just remember that most the wildcard characters will only find existing files. A file that is a member of an archive is treated specially. If the file does not exist, but it is in the archive, the modification time recorded in the archive is used for the file when determining if the file is out-of-date. When figuring out how to make an archived member target (not the file itself, but the file in the archive – the archive(member) target), special care is taken with the transformation rules, as follows:

• archive(member) is made to depend on member.

• The transformation from the member's suffix to the archive's suffix is applied to the archive(member) target.

• The archive(member)'s .TARGET variable is set to the name of the member if member is actually a target, or the path to the member file if member is only a source.

• The .ARCHIVE variable for the archive(member) target is set to the name of the archive.

• The .MEMBER variable is set to the actual string inside the parentheses. In most cases, this will be the same as the .TARGET variable.

• The archive(member)'s place in the local variables of the targets that depend on it is taken by the value of its .TARGET variable.

Thus, a program library could be created with the following makefile:

.o.a            :
	...
	rm -f $(.TARGET:T)
OBJS            = obj1.o obj2.o obj3.o
libprog.a       : libprog.a($(OBJS))
	ar cru $(.TARGET) $(.OODATE)
	ranlib $(.TARGET)

This will cause the three object files to be compiled (if the corresponding source files were modified after the object file or, if that does not exist, the archived object file), the out-of-date ones archived in libprog.a, a table of contents placed in the archive and the newly-archived object files to be removed.

All this is used in the makelib.mk system makefile to create a single library with ease. This makefile looks like this:

#
# Rules for making libraries. The object files that make up the library
# are removed once they are archived.
#
# To make several libraries in parallel, you should define the variable
# "many_libraries". This will serialize the invocations of ranlib.
#
# To use, do something like this:
#
# OBJECTS = <files in the library>
#
# fish.a: fish.a($(OBJECTS)) MAKELIB
#
#

#ifndef _MAKELIB_MK
_MAKELIB_MK    =

#include  <po.mk>

.po.a .o.a     :
	...
	rm -f $(.MEMBER)

ARFLAGS        ?= crl

#
# Re-archive the out-of-date members and recreate the library's table of
# contents using ranlib. If many_libraries is defined, put the ranlib
# off til the end so many libraries can be made at once.
#
MAKELIB        : .USE .PRECIOUS
	ar $(ARFLAGS) $(.TARGET) $(.OODATE)
#ifndef no_ranlib
# ifdef many_libraries
	...
# endif many_libraries
	ranlib $(.TARGET)
#endif no_ranlib

#endif _MAKELIB_MK

4.3. On the Condition...

Like the C compiler before it, PMake allows you to configure the makefile, based on the current environment, using conditional statements. A conditional looks like this:

#if boolean expression
lines
#elif another boolean expression
more lines
#else
still more lines
#endif

They may be nested to a maximum depth of 30 and may occur anywhere (except in a comment, of course). The # must the very first character on the line.

Each boolean expression is made up of terms that look like function calls, the standard C boolean operators &&, ||, and !, and the standard relational operators ==, !=, >, >=, <, and <=, with == and != being overloaded to allow string comparisons as well. && represents logical AND; || is logical OR and ! is logical NOT. The arithmetic and string operators take precedence over all three of these operators, while NOT takes precedence over AND, which takes precedence over OR. This precedence may be overridden with parentheses, and an expression may be parenthesized to your heart's content. Each term looks like a call on one of four functions:

The arithmetic and string operators may only be used to test the value of a variable. The lefthand side must contain the variable expansion, while the righthand side contains either a string, enclosed in double-quotes, or a number. The standard C numeric conventions (except for specifying an octal number) apply to both sides. E.g.:

#if $(OS) == 4.3

#if $(MACHINE) == "sun3"

#if $(LOAD_ADDR) > 0xc000

are all valid conditionals. In addition, the numeric value of a variable can be tested as a boolean as follows:

#if $(LOAD)

would see if LOAD contains a non-zero value and:

#if !$(LOAD)

would test if LOAD contains a zero value.

In addition to the bare #if, there are other forms that apply one of the first two functions to each term. They are as follows:

There are also the “else if” forms: elif, elifdef, elifndef, elifmake, and elifnmake.

For instance, if you wish to create two versions of a program, one of which is optimized (the production version) and the other of which is for debugging (has symbols for dbx), you have two choices: you can create two makefiles, one of which uses the -g flag for the compilation, while the other uses the -O flag, or you can use another target (call it debug) to create the debug version. The construct below will take care of this for you. I have also made it so defining the variable DEBUG (say with pmake -D DEBUG) will also cause the debug version to be made.

#if defined(DEBUG) || make(debug)
CFLAGS         += -g
#else
CFLAGS         += -O
#endif

There are, of course, problems with this approach. The most glaring annoyance is that if you want to go from making a debug version to making a production version, you have to remove all the object files, or you will get some optimized and some debug versions in the same program. Another annoyance is you have to be careful not to make two targets that “conflict” because of some conditionals in the makefile. For instance:

#if make(print)
FORMATTER = ditroff -Plaser_printer
#endif
#if make(draft)
FORMATTER = nroff -Pdot_matrix_printer
#endif

would wreak havoc if you tried pmake draft print since you would use the same formatter for each target. As I said, this all gets somewhat complicated.

4.4. A Shell is a Shell is a Shell

In normal operation, the Bourne Shell (better known as sh) is used to execute the commands to re-create targets. PMake also allows you to specify a different shell for it to use when executing these commands. There are several things PMake must know about the shell you wish to use. These things are specified as the sources for the .SHELL target by keyword, as follows:

The strings that follow these keywords may be enclosed in single or double quotes (the quotes will be stripped off) and may contain the usual C backslash-characters (\n is newline, \r is return, \b is backspace, \' escapes a single-quote inside single-quotes, \" escapes a double-quote inside double-quotes). Now for an example.

This is actually the contents of the <shx.mk> system makefile, and causes PMake to use the Bourne Shell in such a way that each command is printed as it is executed. That is, if more than one command is given on a line, each will be printed separately. Similarly, each time the body of a loop is executed, the commands within that loop will be printed, etc. The specification runs like this:

#
# This is a shell specification to have the Bourne shell echo
# the commands just before executing them, rather than when it reads
# them. Useful if you want to see how variables are being expanded, etc.
#
.SHELL    : path=/bin/sh \
     quiet="set -" \
     echo="set -x" \
     filter="+ set - " \
     echoFlag=x \
     errFlag=e \
     hasErrCtl=yes \
     check="set -e" \
     ignore="set +e"

It tells PMake the following:

• The shell is located in the file /bin/sh. It need not tell PMake that the name of the shell is sh as PMake can figure that out for itself (it is the last component of the path).

• The command to stop echoing is set -.

• The command to start echoing is set -x.

• When the echo off command is executed, the shell will print + set - (The + comes from using the -x flag (rather than the -v flag PMake usually uses)). PMake will remove all occurrences of this string from the output, so you do not notice extra commands you did not put there.

• The flag the Bourne Shell will take to start echoing in this way is the -x flag. The Bourne Shell will only take its flag arguments concatenated as its first argument, so neither this nor the errFlag specification begins with a -.

• The flag to use to turn error-checking on from the start is -e.

• The shell can turn error-checking on and off, and the commands to do so are set +e and set -e, respectively.

I should note that this specification is for Bourne Shells that are not part of Berkeley UNIX, as shells from Berkeley do not do error control. You can get a similar effect, however, by changing the last three lines to be:

hasErrCtl=no \
check="echo \"+ %s\"\n" \
ignore="sh -c '%s || exit 0\n"

This will cause PMake to execute the two commands:

echo "+ cmd"
sh -c 'cmd || true'

for each command for which errors are to be ignored. (In case you are wondering, the thing for ignore tells the shell to execute another shell without error checking on and always exit 0, since the || causes the exit 0 to be executed only if the first command exited non-zero, and if the first command exited zero, the shell will also exit zero, since that is the last command it executed).

4.5. Compatibility

There are three (well, 3 1/2) levels of backwards-compatibility built into PMake. Most makefiles will need none at all. Some may need a little bit of work to operate correctly when run in parallel. Each level encompasses the previous levels (e.g. -B (one shell per command) implies -V). The three levels are described in the following three sections.

4.6. DEFCON 3 – Variable Expansion

As noted before, PMake will not expand a variable unless it knows of a value for it. This can cause problems for makefiles that expect to leave variables undefined except in special circumstances (e.g. if more flags need to be passed to the C compiler or the output from a text processor should be sent to a different printer). If the variables are enclosed in curly braces (${PRINTER}), the shell will let them pass. If they are enclosed in parentheses, however, the shell will declare a syntax error and the make will come to a grinding halt.

You have two choices: change the makefile to define the variables (their values can be overridden on the command line, since that is where they would have been set if you used Make, anyway) or always give the -V flag (this can be done with the .MAKEFLAGS target, if you want).

4.7. DEFCON 2 – The Number of the Beast

Then there are the makefiles that expect certain commands, such as changing to a different directory, to not affect other commands in a target's creation script. You can solve this is either by going back to executing one shell per command (which is what the -B flag forces PMake to do), which slows the process down a good bit and requires you to use semicolons and escaped newlines for shell constructs, or by changing the makefile to execute the offending command(s) in a subshell (by placing the line inside parentheses), like so:

install :: .MAKE
      (cd src; $(.PMAKE) install)
      (cd lib; $(.PMAKE) install)
      (cd man; $(.PMAKE) install)

This will always execute the three makes (even if the -n flag was given) because of the combination of the :: operator and the .MAKE attribute. Each command will change to the proper directory to perform the install, leaving the main shell in the directory in which it started.

4.8. DEFCON 1 – Imitation is the Not the Highest Form of Flattery

The final category of makefile is the one where every command requires input, the dependencies are incompletely specified, or you simply cannot create more than one target at a time, as mentioned earlier. In addition, you may not have the time or desire to upgrade the makefile to run smoothly with PMake. If you are the conservative sort, this is the compatibility mode for you. It is entered either by giving PMake the -M flag (for Make), or by executing PMake as make. In either case, PMake performs things exactly like Make (while still supporting most of the nice new features PMake provides). This includes:

• No parallel execution.

• Targets are made in the exact order specified by the makefile. The sources for each target are made in strict left-to-right order, etc.

• A single Bourne shell is used to execute each command, thus the shell's $$ variable is useless, changing directories does not work across command lines, etc.

• If no special characters exist in a command line, PMake will break the command into words itself and execute the command directly, without executing a shell first. The characters that cause PMake to execute a shell are: #, =, |, ^, (, ), {, }, ;, &, >, <, *, ?, [, ], :, $, `, and \. You should notice that these are all the characters that are given special meaning by the shell (except ' and , which PMake deals with all by its lonesome).

• The use of the null suffix is turned off.

4.9. The Way Things Work

When PMake reads the makefile, it parses sources and targets into nodes in a graph. The graph is directed only in the sense that PMake knows which way is up. Each node contains not only links to all its parents and children (the nodes that depend on it and those on which it depends, respectively), but also a count of the number of its children that have already been processed.

The most important thing to know about how PMake uses this graph is that the traversal is breadth-first and occurs in two passes.

After PMake has parsed the makefile, it begins with the nodes the user has told it to make (either on the command line, or via a .MAIN target, or by the target being the first in the file not labeled with the .NOTMAIN attribute) placed in a queue. It continues to take the node off the front of the queue, mark it as something that needs to be made, pass the node to Suff_FindDeps (mentioned earlier) to find any implicit sources for the node, and place all the node's children that have yet to be marked at the end of the queue. If any of the children is a .USE rule, its attributes are applied to the parent, then its commands are appended to the parent's list of commands and its children are linked to its parent. The parent's unmade children counter is then decremented (since the .USE node has been processed). You will note that this allows a .USE node to have children that are .USE nodes and the rules will be applied in sequence. If the node has no children, it is placed at the end of another queue to be examined in the second pass. This process continues until the first queue is empty.

At this point, all the leaves of the graph are in the examination queue. PMake removes the node at the head of the queue and sees if it is out-of-date. If it is, it is passed to a function that will execute the commands for the node asynchronously. When the commands have completed, all the node's parents have their unmade children counter decremented and, if the counter is then 0, they are placed on the examination queue. Likewise, if the node is up-to-date. Only those parents that were marked on the downward pass are processed in this way. Thus PMake traverses the graph back up to the nodes the user instructed it to create. When the examination queue is empty and no shells are running to create a target, PMake is finished.

Once all targets have been processed, PMake executes the commands attached to the .END target, either explicitly or through the use of an ellipsis in a shell script. If there were no errors during the entire process but there are still some targets unmade (PMake keeps a running count of how many targets are left to be made), there is a cycle in the graph. PMake does a depth-first traversal of the graph to find all the targets that were not made and prints them out one by one.