summaryrefslogtreecommitdiff
path: root/README.md
blob: e7f1d4f6aee246e6c7855a4e8c467c1760c83dc7 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
# GNU Make

Notes, idioms, and worked examples for [GNU Make](https://www.gnu.org/software/make/) — the tool for describing how files are built from other files.

Everything is explained inline below. The handful of demos worth running end‑to‑end live under [`examples/`](examples/).

## Table of Contents

- [Introduction](#introduction)
- [Rules](#rules)
- [Recipes](#recipes)
- [Variables](#variables)
- [Automatic variables](#automatic-variables)
- [Functions](#functions)
- [Conditionals](#conditionals)
- [Prerequisites and rebuild semantics](#prerequisites-and-rebuild-semantics)
- [Special targets and directives](#special-targets-and-directives)
- [Running make](#running-make)
- [Gotchas](#gotchas)
- [Visualising the dependency graph](#visualising-the-dependency-graph)
- [Worked examples](#worked-examples)

---

## Introduction

GNU `make` defines a language for describing the relationships between files. Using `make` forces you to think about each component and how the pieces fit together.

A Makefile is a list of **rules**. A rule has three parts — a **target**, its **prerequisites**, and a **recipe**:

```
target: prereq1 prereq2
<tab>command
<tab>command
```

- The **target** is usually the file to create.
- The **prerequisites** (also called dependencies) are files that must exist and be up to date before the target can be built.
- The **recipe** is the shell command(s) that build the target from its prerequisites.

If a prerequisite has its own rule, `make` builds it first. `make` only rebuilds a target when it is missing or older than one of its prerequisites — this is the whole point: it does the minimum work needed to bring things up to date.

> **The tab rule.** Every recipe line *must* start with a real tab character, not spaces. This trips up everyone at least once. If you must use something else, set [`.RECIPEPREFIX`](https://www.gnu.org/software/make/manual/html_node/Special-Variables.html) (e.g. `.RECIPEPREFIX = >`).

The examples below assume a small sample dataset:

```bash
mkdir data; for i in {1..4}; do echo ${i} > data/${i}.fa; done
```

---

## Rules

A rule defines a target and its prerequisites (if any). There are several kinds:

1. **Explicit rules** name a specific target — the most common kind.
2. **Pattern rules** use the `%` wildcard instead of explicit filenames, so the rule applies to any target matching the pattern.
3. **Implicit rules** are pattern or suffix rules built into Make's database (e.g. how to make `%.o` from `%.c`).
4. **Static pattern rules** are pattern rules restricted to a specific list of targets: `targets: pattern: prereq-pattern`.

*Suffix rules* also exist but are obsolete — pattern rules are clearer and more general.

### Explicit and pattern rules

A small C build shows both an explicit rule and a pattern rule working together:

```makefile
CC     := gcc
CFLAGS := -Wall -O2

# Pattern rule: any %.o can be built from the matching %.c.
# -c compiles without linking.
%.o: %.c
	$(CC) $(CFLAGS) -c $< -o $@

# Explicit rule: link the objects into the final binary.
hello: main.o util.o
	$(CC) $(CFLAGS) $^ -o $@
```

`make hello` builds `main.o` and `util.o` via the pattern rule, then links them. (`$<`, `$@` and `$^` are *automatic variables* — see [below](#automatic-variables).)

### The default goal

Running `make` with no target builds the **default goal**: the first *normal* target in the file. Make skips targets whose names start with a dot (special targets like `.PHONY`) and pattern rules when choosing it, so neither becomes the default. By convention the default goal is a phony `all` target that depends on everything you normally want built:

```makefile
all: hello
```

You can set it explicitly with `.DEFAULT_GOAL := some_target`.

### Multiple targets and grouped targets

Listing several targets in one rule means the recipe runs **once per target**, with `$@` set to whichever is being built:

```makefile
# `make first.txt` makes first.txt; `make second.txt` makes second.txt.
first.txt second.txt:
	@echo "Generating: $@"
	touch $@
```

When a single recipe produces *all* the listed files at once, use a **grouped target** with `&:` (GNU Make 4.3+). The recipe runs only once for the whole group:

```makefile
# Both files are produced together by one invocation.
fifth.txt sixth.txt &:
	@echo "Generating fifth.txt and sixth.txt together"
	touch fifth.txt sixth.txt
```

---

## Recipes

A **recipe** is the set of shell commands Make runs to update a target. Each line is indented with a tab.

### Each line runs in its own shell

By default Make runs every recipe line in a *separate* shell, so `cd`, shell variables, and other state do **not** carry from one line to the next:

```makefile
broken:
	cd /tmp
	pwd          # prints the original directory, NOT /tmp

working:
	cd /tmp && pwd   # prints /tmp
```

Use a trailing backslash to continue one command across lines in a single shell:

```makefile
also_working:
	cd /tmp && \
		pwd
```

### .ONESHELL

`.ONESHELL:` (GNU Make 3.82+) runs an entire recipe in one shell, so multi‑line recipes share state naturally:

```makefile
.ONESHELL:
my_target:
	cd /tmp
	pwd          # now this prints /tmp
```

Note that `.ONESHELL` is **global** — it affects every recipe in the Makefile, so don't mix it with demos that rely on separate‑shell behaviour.

### Line prefixes

| Prefix | Effect |
|--------|--------|
| `@` | Suppress echoing of the command (Make normally prints each command before running it) |
| `-` | Ignore a non‑zero exit status from the command |
| `+` | Run the command even under `make -n` (dry run) |

They combine, e.g. `-@cmd` silences output *and* ignores errors:

```makefile
errors:
	@echo "--- ignoring a failing command ---"
	-false
	@echo "Make continued past 'false' because of the - prefix"
```

### Escaping `$`

Make expands `$(VAR)` and `${VAR}` **before** the shell sees the line. To pass a literal `$` to the shell (for shell variables, `awk`, etc.) escape it as `$$`:

```makefile
demo:
	@echo "Make variable: $(MY_VAR)"
	@SHELL_VAR="hello"; echo "Shell variable: $$SHELL_VAR"
```

### Canned recipes (`define` / `endef`)

A **canned recipe** is a reusable command sequence stored in a variable. Define it once, expand it with `$(name)` wherever it's needed. Automatic variables like `$@` inside it resolve to the *calling* rule's target:

```makefile
define log_start
	@echo "=== Starting: $@ ==="
	@date
endef

define log_end
	@date
	@echo "=== Finished: $@ ==="
endef

target_a:
	$(log_start)
	@echo "Building target_a"
	$(log_end)

target_b:
	$(log_start)
	@echo "Building target_b"
	$(log_end)
```

See the manual on [recipes](https://www.gnu.org/software/make/manual/html_node/Recipes.html) and [canned recipes](https://www.gnu.org/software/make/manual/html_node/Canned-Recipes.html).

---

## Variables

### Assignment operators

| Operator | Name | When the right‑hand side is expanded |
|----------|------|--------------------------------------|
| `=` | recursive (lazy) | Every time the variable is **used** (late binding) |
| `:=` (and POSIX `::=`) | simple (immediate) | Once, **at the point of definition** (early binding) |
| `?=` | conditional | Like `=`, but the assignment happens **only if the variable is not already set** |
| `+=` | append | Appends to the current value (or sets it, if previously unset) |
| `:::=` | immediate‑with‑escape (4.4+) | Expanded immediately, then results re‑escaped; rarely needed |

> **Common confusion:** `:=` is the *immediate* operator (value fixed when defined); `?=` is the *set‑only‑if‑undefined* operator. They are not interchangeable.

```makefile
A  = $(B)      # recursive: A re-expands $(B) every time A is used
B := hello     # simple: evaluated once, here
C ?= default   # conditional: only assigned if C has no value yet
D += more      # append to D
```

### Late vs early binding

The difference between `=` and `:=` is *when* references inside the value are resolved:

```makefile
BAR  := initial
SIMP := $(BAR)   # := captures "initial" right now
REC   = $(BAR)   # = just stores the reference $(BAR)
BAR  := changed  # reassign BAR

# When expanded later:
#   SIMP -> "initial"   (snapshot taken at definition)
#   REC  -> "changed"   (resolved at use, sees the new BAR)
```

This matters most with `$(shell ...)`: a recursive variable re‑runs the command on every expansion, a simple one runs it once:

```makefile
NOW_EQ      = $(shell date +%s)   # re-runs `date` every time NOW_EQ is used
NOW_SIMPLE := $(shell date +%s)   # runs `date` once, at parse time
```

### Override precedence

When the same variable is set in more than one place, the priority is:

```
command line  >  Makefile  >  environment  >  built-in default
```

This is the *default* ordering. `make -e` (`--environment-overrides`) flips the middle two, making environment variables win over Makefile assignments.

`?=` is convenient for *overridable defaults*, because it backs off if any higher‑priority source already set the value:

```makefile
CC ?= gcc        # `make CC=clang` wins; otherwise CC is gcc
```

To make a Makefile assignment win even over the command line, use `override`:

```makefile
override USER := Mr. President   # `make USER=...` cannot change this
```

### Variable scope

A variable is global by default, but can be scoped to a target or a pattern:

```makefile
VAR = global scope

# Target-specific: VAR has this value only while building scoping_local
# (and its prerequisites).
scoping_local: VAR = local scope
scoping_local:
	@echo "In $@, VAR is $(VAR)"

# Pattern-specific: applies to any target matching the pattern p%
p%: VAR = set with wildcard
```

### Environment variables

Make imports the environment as variables. A variable that isn't set expands to the empty string:

```makefile
echo_env:
	@echo "Hello $(USER), your home is $(HOME)"
	@echo "This env var does not exist: [$(DOESNOTEXIST)]"
	@echo "Exported var PACIFIC=[$(PACIFIC)]"   # run `export PACIFIC=ocean` first
```

---

## Automatic variables

Inside a recipe, Make sets these to describe the rule being built. They are the key to writing generic rules:

| Variable | Meaning |
|----------|---------|
| `$@` | The target name |
| `$<` | The first prerequisite |
| `$^` | All prerequisites, duplicates removed |
| `$+` | All prerequisites, duplicates kept, in order |
| `$?` | Prerequisites that are newer than the target |
| `$*` | The stem matched by `%` in a pattern rule |
| `$(@D)` / `$(@F)` | The directory / file parts of `$@` |

```makefile
target: one two three four five
	@echo "$$@  target name:                 $@"
	@echo "$$<  first prereq:                 $<"
	@echo "$$^  all prereqs:                  $^"
	@echo "$$?  prereqs newer than target:    $?"
	@echo "second prereq via word:           $(word 2,$^)"
	@echo "all but the first prereq:         $(filter-out $<,$^)"
```

Note that `$(filter-out $<,$^)` removes *every* prerequisite equal to `$<` (a value match, not a positional one); because `$^` de‑duplicates, that happens to mean "all but the first" here. `$(@D)` is handy when a tool needs the *directory* containing a file rather than the file itself.

---

## Functions

Make has built‑in functions for text processing; they expand to strings before recipes run.

### Text and list functions

Assuming `BAMS := $(wildcard *.bam)`:

| Function | Purpose | Example |
|----------|---------|---------|
| `$(wildcard pat)` | Files matching a glob | `$(wildcard *.bam)` |
| `$(patsubst pat,repl,text)` | Pattern substitution over words | `$(patsubst %,%.bai,$(BAMS))` |
| `$(subst from,to,text)` | Plain (non‑pattern) string substitution | `$(subst -,_,$(BAMS))` |
| `$(addprefix p,names)` | Prepend a prefix to each word | `$(addprefix raw-,$(BAMS))` |
| `$(addsuffix s,names)` | Append a suffix to each word | `$(addsuffix .bai,$(BAMS))` |
| `$(filter pat,text)` | Keep words matching the pattern(s) | `$(filter sample1%,$(BAMS))` |
| `$(filter-out pat,text)` | Drop words matching the pattern(s) | `$(filter-out sample1%,$(BAMS))` |
| `$(sort list)` | Sort words and remove duplicates | `$(sort $(BAMS))` |
| `$(notdir names)` | Strip the directory part | `$(notdir path/to/x.bam)` → `x.bam` |
| `$(dir names)` | Keep only the directory part | `$(dir path/to/x.bam)` → `path/to/` |
| `$(basename names)` | Strip the suffix | `$(basename x.bam)` → `x` |
| `$(suffix names)` | Keep only the suffix | `$(suffix x.bam)` → `.bam` |

A common idiom — derive one file list from another:

```makefile
fastas := $(wildcard data/*.fa)
# data/1.fa data/2.fa ... -> result/1.bam result/2.bam ...
bams   := $(patsubst %,result/%.bam,$(basename $(notdir $(fastas))))
```

You can also use a **substitution reference**, a shorthand for the common `patsubst` case `$(VAR:pattern=replacement)`:

```makefile
BOOST_LIBRARIES := system filesystem regex
# -> -lboost_system -lboost_filesystem -lboost_regex
LDFLAGS += $(BOOST_LIBRARIES:%=-lboost_%)
```

### foreach

`$(foreach var,list,text)` expands `text` once per word in `list`:

```makefile
# Wrap each BAM in brackets: [a.bam] [b.bam] ...
BAMS_BRACKETS := $(foreach f,$(BAMS),[$(f)])
```

Combined with **computed variable names** (`$($(x)_SUFFIX)`), `foreach` can drive a small data model:

```makefile
BATCHES   = batch1 batch2
batch1_ID = ID1
batch2_ID = ID2
batch1_DIR = dir_batch1
batch2_DIR = dir_batch2

# Expands to: dir_batch1 dir_batch2
all: $(foreach batch,$(BATCHES),$($(batch)_DIR))

$(BATCHES):
	@echo "Processing $@ using ID $($@_ID) into $($@_DIR)"
	mkdir -p $($@_DIR)
```

(For generating whole *rules* dynamically rather than just lists, reach for `$(eval ...)`.)

### shell

`$(shell cmd)` runs a command and captures its output: embedded newlines become spaces, and the trailing newline is stripped:

```makefile
DATE := $(shell date +%F)
```

Remember the timing rule from [Variables](#variables): with `:=` the command runs once at parse time; with `=` it re‑runs on every expansion.

### Diagnostics: info, warning, error

These print during parsing (not in a recipe). `$(error ...)` aborts the build:

```makefile
$(info  CFLAGS is $(CFLAGS))      # informational message
$(warning Debugging enabled)      # warning, build continues
$(error Cannot continue)          # print the message and abort parsing
```

See [Conditionals](#conditionals) for the common `ifndef`/`$(error)` pattern used to guard against unset variables.

### User‑defined functions

Define a function as a variable; call it with `$(call name,arg1,arg2,...)`, where `$1`, `$2`, … are the arguments:

```makefile
make_date = $1-$2-$3
today    := $(call make_date,2022,09,15)   # -> 2022-09-15

# Abort the build if a required command is missing.
assert-command-present = $(if $(shell command -v $1),,$(error '$1' is missing))
$(call assert-command-present,samtools)
```

---

## Conditionals

`ifeq` / `ifneq` compare two expanded values. `ifdef` / `ifndef` test whether a variable is **defined to a non‑empty value, without expanding it first** — so `ifdef foo` is true for `foo = $(bar)` even when `$(bar)` itself expands to empty. To test the *expanded* value for emptiness, use `ifeq ($(VAR),)` (shown below). Conditionals are evaluated when the Makefile is **parsed**, not when a recipe runs.

```makefile
my_target:
ifeq ($(COND),1)
	echo Condition 1
else ifeq ($(COND),2)
	echo Condition 2
else
	echo Condition else
endif
```

A frequent pattern is the **emptiness test** — note the empty first argument:

```makefile
# True when BOOST_ROOT is set to something non-empty.
ifneq (,$(BOOST_ROOT))
  BOOST_INCLUDEDIR ?= $(BOOST_ROOT)/include
endif
```

And a guard that aborts early if a required variable is unset:

```makefile
ifndef PACIFIC
$(error Variable PACIFIC is not set)
endif
```

### Requiring a minimum Make version

Several features in these notes need a recent Make (`.ONESHELL` 3.82+, grouped `&:` targets 4.3+, `:::=` 4.4+). Guard for them at parse time using the built‑in `$(MAKE_VERSION)`:

```makefile
need := 4.3
ok   := $(filter $(need),$(firstword $(sort $(MAKE_VERSION) $(need))))
ifeq (,$(ok))
$(error GNU Make $(need)+ required, but this is $(MAKE_VERSION))
endif
```

`$(sort ...)` orders the running version against the one you need, `$(firstword ...)` takes the lower of the two, and `$(filter ...)` is non‑empty only when the running version is at least `need`.

---

## Prerequisites and rebuild semantics

Make decides whether to rebuild a target by comparing modification times (mtimes): if any **normal** prerequisite is newer than the target — or the target is missing — the recipe runs.

### Order‑only prerequisites

Sometimes a target needs another to *exist* but should not rebuild just because that other thing changed — the classic case is an output directory. List such prerequisites after a pipe `|`; their timestamps are ignored:

```makefile
# build/output.txt depends on input.txt (normal) and the build/ dir (order-only).
# A newer build/ timestamp will NOT trigger a rebuild of output.txt.
build/output.txt: input.txt | build
	cp $< $@

build:
	mkdir -p $@
```

> Beware **symlink** targets: Make checks the mtime of the file the link *resolves to*, not the link itself — which has surprising consequences. See [Gotchas](#gotchas).

---

## Special targets and directives

### .PHONY

By default a target is a *file* target. A **phony** target is one that doesn't correspond to a file — `all`, `clean`, `install`, `test`, and friends. Declaring it phony does two things: it stops Make being confused by a real file of the same name (e.g. a file literally called `clean`), and it makes the target *always* run:

```makefile
.PHONY: clean
clean:
	rm -rf *.o
```

A phony target is, in effect, always out of date, so `make clean` runs regardless of the filesystem. You can also use a phony name as a short alias for a long real target (see [Worked examples](#worked-examples)).

### .DELETE_ON_ERROR

If a recipe exits non‑zero partway through, the half‑written target file is left behind and looks up to date. `.DELETE_ON_ERROR:` tells Make to delete a target whose recipe fails:

```makefile
.DELETE_ON_ERROR:

output1.txt:
	touch $@

# Write a partial result, then fail.
output2.txt: output1.txt
	touch $@ && false
```

When the recipe exits non‑zero, the half‑written `output2.txt` is removed instead of being left behind looking up to date:

```console
$ make
touch output1.txt
touch output2.txt && false
make: *** [Makefile:8: output2.txt] Error 1
make: *** Deleting file 'output2.txt'
```

Make *also* deletes a target if you interrupt a running recipe with Ctrl‑C, but that cleanup happens regardless of `.DELETE_ON_ERROR` — this special target is specifically about non‑zero *exit status*.

Related special targets worth knowing: `.SECONDARY` and `.PRECIOUS` (keep intermediate files), `.INTERMEDIATE`, and `.NOTPARALLEL` (force serial builds).

### SHELL

Set the shell used for recipes (recipes default to `/bin/sh`). Use this when you rely on bash features:

```makefile
SHELL := /bin/bash
```

### include

`include` suspends reading the current Makefile to read another first — useful for pulling configuration out into its own file:

```makefile
include config.mk
```

This is also how you pin tool versions: keep them in a small included file (e.g. `tool_versions` with `samtools_version=1.17`) and reference `$(samtools_version)` in your rules. The [dragmap example](examples/dragmap/) uses `include config.mk` to assemble compiler flags conditionally.

---

## Running make

`make` accepts many flags; the full list is in `man make` or `make --help`. The most useful:

| Flag | Long form | What it does | When it's useful |
| --- | --- | --- | --- |
| `-j [N]` | `--jobs[=N]` | Run up to `N` recipes in parallel (no `N` = as many as possible). | Anything CPU/IO‑bound with independent targets — the single biggest speed‑up. Prerequisites must be declared correctly or parallel builds race. |
| `-O[type]` | `--output-sync[=type]` | Group each recipe's output so parallel jobs don't interleave. `type` ∈ `none`, `line`, `target` (default), `recurse`. | Whenever you use `-j`. Without it, parallel logs are unreadable. |
| `-k` | `--keep-going` | Keep building unrelated targets after a failure instead of stopping. | Long batch runs where you want to see *all* failures in one pass. |
| `-n` | `--dry-run` | Print the commands Make *would* run without executing them. | Sanity‑checking a Makefile, or generating a command list (e.g. for `make2graph`). |
| `-B` | `--always-make` | Treat every target as out of date and rebuild unconditionally. | Forcing a clean rebuild, or when you changed something Make can't detect. |
| `-d` | `--debug` | Verbose info on which rules fire, which prerequisites are considered, and why targets are (or aren't) rebuilt. | Figuring out *why* Make is rebuilding something — or refusing to. Pair with `-n`. |
| `-C dir` | `--directory=dir` | `cd` into `dir` before reading the Makefile. | Running a sub‑project's Makefile from elsewhere. |
| `-f file` | `--file=file` | Use `file` instead of the default (`Makefile`/`makefile`/`GNUmakefile`). | Keeping multiple Makefiles in one directory. |
| `-s` | `--silent` | Suppress command echoing (as if every line were prefixed with `@`). | Cleaner output when you only care about program messages. |
| `-i` | `--ignore-errors` | Ignore non‑zero exit codes from recipes. | Rarely a good idea; occasionally useful for cleanup targets. |
| `-p` | `--print-data-base` | Print Make's internal database (variables, rules, implicit rules) after parsing. | Discovering implicit rules and variables in scope; pair with `-n`. |

A useful combination for real workloads:

```bash
make -j4 -O -k all 2>&1 | tee results/run.log
```

This runs up to four recipes in parallel, keeps the interleaved output readable, doesn't stop on the first failure, and captures stdout+stderr to a log.

Notes on `-j` and `-O`:

- `-O target` (the default) prints each recipe's output as one block when it finishes — the most readable mode.
- `-O line` synchronises at line granularity, so you see progress in real time without garbled lines.
- `-O none` is the legacy interleaved behaviour — almost never what you want with `-j`.
- `-k` with `-j` skips failed targets and their dependents but keeps other branches going; Make still exits non‑zero so CI catches the failure.
- `-n` suppresses *recipe* execution only — parse‑time `$(shell ...)` functions still run.

### Passing variables on the command line

Variables given on the command line override the Makefile (unless `override` is used). This is the standard way to toggle behaviour: `make CC=clang`, `make DEBUG=1`, `make VERBOSE=1`.

### A verbosity toggle

A tidy idiom for switchable command echoing — quiet by default, full commands when asked:

```makefile
VERBOSE ?= 0
ifeq ($(VERBOSE),0)
  Q := @
else
  Q :=
endif

all:
	$(Q)printf "Hello, World!\n"
```

`make` is quiet; `make VERBOSE=1` echoes the command. The same `?=`/conditional pattern drives a debug toggle — e.g. `DEBUG ?= 0` and, when set, `CFLAGS += -g` plus a `$(warning Debugging enabled)`.

---

## Gotchas

**A target that never creates its file forces endless rebuilds.** Here `main.o` has no effective recipe (the condition is false, so the body is empty) and no file is ever produced, so Make treats it as perpetually out of date — and `hello`, which depends on it, relinks *every* run. In a large Makefile this is silent and maddening:

```makefile
hello: main.o
	touch $@

main.o:
ifeq (false,true)
	echo $@      # never runs -> main.o is never created -> hello always rebuilds
endif
```

**Symlink targets are judged by the linked file's mtime.** When a target is a symlink, Make uses the mtime of the file it *resolves to*, not the link itself. So recreating or touching the link without changing the underlying file won't trigger dependents, and a prerequisite that is newer than the underlying file keeps the step re‑running until that file is regenerated.

**`$` needs escaping in recipes.** Make consumes a single `$` for its own expansion. Use `$$` for a literal `$` (shell variables, `awk` programs, etc.). See [Recipes](#recipes).

**Don't depend on a list of unknown targets.** It's tempting to generate files whose names you don't know ahead of time and have Make skip them once they exist. This is fragile: once `$(FILES)` exists, Make won't rebuild those files even when their inputs change, and you can't tell whether you got the 5 files you expected or only 2. Prefer knowing your targets up front.

**Parallel builds expose under‑declared prerequisites.** Under `-j`, two recipes that secretly share a file (without declaring it) can race. If a parallel build is flaky but a serial one isn't, look for a missing prerequisite.

---

## Visualising the dependency graph

[`make2graph`](https://github.com/lindenb/makefile2graph) turns Make's dry‑run output into a Graphviz diagram — a quick way to see how targets connect.

```console
# Build make2graph (one-off)
git clone https://github.com/lindenb/makefile2graph.git
cd makefile2graph && make

# Install graphviz for `dot`
sudo apt update && sudo apt install -y graphviz

# Render the graph: -B force all targets, -n dry-run, -d debug output
make -Bnd | ./make2graph | dot -Tpng -o out.png
```

A runnable demo and a sample `out.png` are in [`examples/visual/`](examples/visual/). To explore Make's *internal* knowledge (implicit rules, variables) instead of just your own targets, use `make -p`.

> `-n` only suppresses *recipe* execution — parse‑time `$(shell ...)` calls still run, so `make -Bnd` is not always fully side‑effect‑free when generating the graph.

---

## Worked examples

Short patterns are inlined here; the longer, runnable ones live under [`examples/`](examples/).

### Aliasing long targets

Give long real filenames short, memorable phony aliases. The trick: targets and prerequisites depend on the *real* files (via variables); the aliases just point at them.

```makefile
.PHONY: all first
one := file_one_one_one_one_one.txt

all: $(one)

first: $(one)     # `make first` is the convenient alias...
$(one):           # ...but the real work hangs off the file target
	touch $@
```

### Versioned symlink backups

Keep one stable symlink that always points at the latest version, while retaining a timestamped backup of every version:

```makefile
my_symlink: dep.txt
	DS=$$(date +%Y%m%d_%H%M%S) && \
		cat $< > $$DS && \
		ln -fs $$DS $@
```

Editing `dep.txt` makes the prerequisite newer than the target, so the rule re‑runs — producing a new timestamped file and repointing the link. (The symlink‑mtime caveat in [Gotchas](#gotchas) applies if you instead try to trigger rebuilds via the link itself.)

### Data‑driven targets

Build a target list partly from a file, mixing static and discovered names:

```makefile
FILES      := 1.txt 2.txt 3.txt
MORE_FILES := $(shell cut -f1 -d, files_to_create.csv)

all: $(FILES) $(MORE_FILES) final.txt

%.txt:
	touch $@

final.txt: $(FILES) $(MORE_FILES)
	touch $@
```

(See the unknown‑targets caveat in [Gotchas](#gotchas) before relying on this.)

### Debugging a build's environment — [`examples/dragmap/`](examples/dragmap/)

A real case study: building [DRAGMAP](https://github.com/Illumina/DRAGMAP) against an alternate Boost install. It uses `include`, `$(info ...)` to dump variables, and `ifneq` emptiness tests to assemble `CPPFLAGS`/`LDFLAGS` from `BOOST_ROOT` / `BOOST_INCLUDEDIR` / `BOOST_LIBRARYDIR`. The `run.sh`/`test*.sh` scripts toggle the environment variables to show which combination fixes the link errors.

### Orchestrating SGE cluster jobs — [`examples/sge/`](examples/sge/)

Using Make as a dependency‑aware front end for a job scheduler: each target submits a job with `qsub` and chains them with `-hold_jid` so a job waits for its predecessor, while independent targets run in parallel. `submit.sh` runs Make itself as a parallel job (`make -j $NSLOTS`). SGE is largely legacy now; the same pattern applies to Slurm (`sbatch --dependency=afterok:<id>`).