1

ITEC400Make Utility

and UNIX Files (Part 2)George Vaughan

Franklin University

2

Topics

• What is Make?

• What is GNU Make?

• The ‘makefile’

• Make Variables

• More Makefile features

• Implicit makefile Rules

• View Pathing

3

What is Make?

• ‘Make’ is a utility that is used to assemble things.

• Such items include programs, documents or any other product that may require multiple steps.

• Make is especially useful in constructing a component that requires a multi-step construction process.

• A type of Inference Engine

4

What Is Make?

• Make can be made aware of dependencies.– If A is dependent on B and B is dependent on C, then:

• Make will not attempt to build A until B is built• Make will not attempt to build B until C is built

• Make is aware of modification dates:– If B is somehow modified, then:

• only A and B will be built,• C will not be built (A is dependent on B, but C is not

dependent on either A or B)

• Make will only build what is necessary and nothing more.

5

What is Make?

• Make, by itself, does not know how to build anything.

• Make is a rules based system (similar to an expert system).

• Make knows how and what to build based on:– the dependencies we specify– the rules we specify– modification times of build components.

6

What is Make?

• The rules needed to build ‘something’ are specified in a file called a ‘makefile’.

• In the case of building a program, rules are established in the ‘makefile’ that specify such things as:– how to construct object files from source and

header files.– how to construct an executable from object

files.

7

What is GNU Make?

• There are several make utilities available including those that are part of standard Unix distributions and from 3rd party sources (see http://directory.google.com/Top/Computers/Software/Build_Management/Make_Tools/)

• This lecture is focused on GNU Make• GNU Make comes from the Free Software Foundation and

is part of Red Hat. It is also the default make utility on einstein.

• Type ‘make -v’ to determine what version of make you have.

• GNU Make was implemented by Richard Stallman and Roland McGrath.

• This lecture will use a simple C++ program as an example of using Make.

8

Example ex0510.dir

• In our first example, we will look at a simple C++ program composed of the following files:– ClassA.h - contains class declaration– ClassA.cpp - contains class implementation– main.cpp - contains implementation of main().

• The C++ program really doesn’t do very much except print the name of the C++ class (ClassA), but it is useful for this makefile illustration

• We will look at:– The source code.– Building this program without the use of a make file.– Building this program using a make file.

9

Example ex0510.dir

• Contents of classA.h:0001 #include <stdio.h>00020003 class ClassA {0004 public:0005 void myClass();0006 };

• Contents of ClassA.cpp:0001 #include "ClassA.h"00020003 void ClassA::myClass() {0004 printf("%s\n", "ClassA");0005 }

10

Example ex0510.dir

• Contents of main.cpp0001 #include "ClassA.h"00020003 int main() {0004 ClassA objectA;00050006 objectA.myClass();0007 return 0;0008 }

11

Example ex0510.dir

• Notice that the this example is not a single file – there are 3 source files that make up this program.

• It is common for large projects to consist of many source files (in the thousands)

• Some source files depend on other source files – for example ClassA.cpp includes ClassA.h.

• This means that the object file, ClassA.o not only depdends on ClassA.cpp, but also ClassA.h

• Object files (files whose names end in ‘.o’) are not source files - they are the products of compilation.

• Therefore we need to build based on dependencies.• The next slide shows a dependency graph for this

example. Note that graph includes both source files and object files.

12

Example ex0510.dir

myProgram (executable)

ClassA.o (object file)main.o (object file)

classA.h (source)main.cpp (source) classA.cpp (source)

Note: Italicized file namesare build products (i.e. createdby running make)

13

Example ex0510.dir

• The next slide shows the commands we need to build the application manually (without using a makefile).

• In this example, we are using “g++” which is the “C and C++” complier from GNU.

• This compiler is available on einstein and on Redhat and Knoppix (your itec400 CD).

• We first use the “-c” option on g++ to compile the source files (create the object files)

• We then use the “-o” option to create the executable program from the object files.

• Please Note: The contents on the next slide is output captured from the build process. The next slide is not the contents of some source file

14

Example ex0510.dir0001: $ g++ -c ClassA.cpp0002:0003: $ g++ -c main.cpp0004:0005: $ ls0006: ClassA.cpp ClassA.h ClassA.o

main.cpp main.o0007:0008: $ g++ -o myProgram ClassA.o

main.o0009:0010: ls0011: ClassA.cpp ClassA.h ClassA.o

main.cpp main.o myProgram0012:0013: ./myProgram0014: ClassA

• Line 1: First, compile ClassA.cpp• Line 3: Second, compile main.cpp• Line 5: directory now contains object

files (files that end in “.o”)…• Line 8: link object files into an

executable whose name is “myProgram”

• line 10: directory now contains executable which is named “myProgram”

• Line 13: execute “myProgram”.• Line 14: output of program.

15

makefile

• We will now create a simple makefile to build “myProgram” for us.

• The difference between using a makefile versus a shell script is that the makefile will only rebuild what is necessary.

• The makefile contains a list of rules which describe: – targets– dependencies – actions

16

A Simple makefile• The makefile rule has the following format:

target : prerequisites (dependencies)…command_1command_2

– where:• target is typically the name of the file to be generated. A target may also

be an action to be performed • prerequisite(s) are file(s) that that this target depends on.• command_n are the SHELL command(s) necessary to produce the target

(one command per line). Each command line MUST be prefixed with a TAB.

– If you use spaces to indent instead of TABs, your make file will not work.• The commands MUST, in some way, alter the time-stamp of the target file.

– This because make uses the timestamp to determine if the file is up to date. – If the timestamp were not updated, the file always be built (even if the file is up to

date – which is a waste of time)

17

A Simple makefile

• Note that the rules in the makefile are not executed in sequential order. They are executed based on dependencies.

• The next slides illustrate how to use a makefile to build the program that was seen in example 0510.dir

18

Example 0520.dir

0001 myProgram : main.o ClassA.o

0002 g++ -o myProgram main.o \

0003 ClassA.o

0004

0005 main.o : main.cpp ClassA.h

0006 g++ -c main.cpp

0007

0008 ClassA.o : ClassA.cpp ClassA.h

0009 g++ -c ClassA.cpp

0010

0011 clean :

0012 rm myProgram *.o

NOTE: It is easier to understand this make file by referring to the dependency graph on the next page. The graph is the same graph we saw earlier

• This make file is stored in a file called “makefile”

• Line 1: executable ‘myProgram’ is dependent on object files.

• Line 2-3: executable is constructed by linking object files.

• Line 3 is a continuation of line 2 (notice the continuation character “\”)

• Line 5: main.o is dependent on main.cpp and ClassA.h

• Line 8: ClassA.o is dependent on ClassA.cpp and ClassA.h

• Line 11: ‘clean’ is an action. ‘clean’ is used to remove generated files so we can build from scratch.

19

Dependency Graph (repeated)

myProgram (executable)

ClassA.o (object file)main.o (object file)

classA.h (source)main.cpp (source) classA.cpp (source)

Note: Italicized file namesare build products (i.e. createdby running make)

20

Make Exercise0001 myProgram : main.o ClassA.o0002 g++ -o myProgram main.o \0003 ClassA.o00040005 main.o : main.cpp ClassA.h0006 g++ -c main.cpp00070008 ClassA.o : ClassA.cpp ClassA.h0009 g++ -c ClassA.cpp00100011 clean :0012 rm myProgram *.o

Try answering the questions below using the makefile from Example ex0520.dir. It might also help to refer to the dependency graph.

What needs to be built if:1. A second ‘make’ is run

immediately after a first ‘make’?2. ClassA.cpp is changed?3. ClassA.h is changed?4. main.cpp is changed?5. if clean is specified?

* The answers appear on the next few slides.

21

Answers To Make Exercise

1. What needs to be built if a second ‘make’ is run immediately after a first ‘make’? Nothing happens – Make does not make anything.

>make

make: `myProgram' is up to date.

2. What needs to be built if ClassA.cpp is changed?

>make

g++ -c ClassA.cpp

g++ -o myProgram main.o \

ClassA.o

22

Answers To Make Exercise

3. What needs to be built if ClassA.h is changed?

>make

g++ -c main.cpp

g++ -c ClassA.cpp

g++ -o myProgram main.o \

ClassA.o

23

Answers To Make Exercise

5. What needs to be built if main.cpp is changed?>makeg++ -c main.cppg++ -o myProgram main.o \ClassA.o

6. What needs to be built if if clean is specified?• Before ‘clean’

>lsClassA.cpp ClassA.o main.cpp makefileClassA.h main.o myProgram

• After ‘clean’>make cleanrm myProgram *.o>lsClassA.cpp ClassA.h main.cpp makefile

24

More on makefiles

• By default, make attempts to build the first target defined in a makefile.

• The first target is known as the default goal.• The default goal is the target that is built if no

explicit targets are given to make on the command line.

• A target is rebuilt if any of its prerequisites have a newer timestamp than the target itself.

• Thus, make performs a recursive decent through the target tree building the lowest level targets first.

25

Example ex0530.dir

• Assume that our project gets more complicated - assume we introduce classes ClassB and ClassC.

• The source code for this example (like all examples) is located in the examples directory and will not be reproduced in these lecture notes.

26

Example ex0530.dir0001 myProgram : main.o ClassA.o \

0002 ClassB.o ClassC.o

0003 g++ -o myProgram main.o \

0004 ClassA.o ClassB.o \

0005 ClassC.o

0006 main.o : main.cpp ClassA.h ClassB.h \

0007 ClassC.h

0008 g++ -c main.cpp

0009 ClassA.o : ClassA.cpp ClassA.h

0010 g++ -c ClassA.cpp

0011 ClassB.o : ClassB.cpp ClassB.h

0012 g++ -c ClassB.cpp

0013 ClassC.o : ClassC.cpp ClassC.h

0014 g++ -c ClassC.cpp

0015 clean :

0016 rm myProgram *.o

• The name of the file is makefile

• Lines 1-5: Things are getting complicated…

• if we forget an object file, especially in the prerequisites, we can have a hard time debugging.

• There has to be a better way…

27

makefile Variables• There are some things we can do to simplify the makefile on the

previous slide.• One thing we can do is to take advantage of makefile variables.• In this case, we will use a makefile variable to store the list of object

files.• We will call the variable OBJECTS• By default, make uses the Bourne shell for the command list for

each rule.• We can specify our own shell preference with the special make

variable, “SHELL”• For example, if wish to use Korn Shell we would include the

following line in our makefile: SHELL=/bin/ksh

28

Example ex0540.dir0001 SHELL=/bin/ksh

0002 OBJECTS = main.o ClassA.o \

0003 ClassB.o ClassC.o

0004 myProgram : $(OBJECTS)

0005 g++ -o myProgram $(OBJECTS)

0006 main.o : main.cpp ClassA.h \

0007 ClassB.h ClassC.h

0008 g++ -c main.cpp

0009 ClassA.o : ClassA.cpp ClassA.h

0010 g++ -c ClassA.cpp

0011 ClassB.o : ClassB.cpp ClassB.h

0012 g++ -c ClassB.cpp

0013 ClassC.o : ClassC.cpp ClassC.h

0014 g++ -c ClassC.cpp

0015 clean :

0016 rm myProgram $(OBJECTS)

• Line 1: The special make variable ‘SHELL’ is used to tell make to use Korn Shell

• Line 2: The variable ‘OBJECTS’ is defined. Also “\” is a continuation character.

• Line 4,5 and 16: The variable ‘OBJECTS’ is used.

• Notice that when a makefile variable is used, it is enclosed by parentheses and prefixed by $.

29

Implicit makefile Rules

• As mentioned previously, make may be used to build anything, as long as actions can be defined to produce the target.

• However, since make is used so often to build software products, it can use ‘implicit’ rules for building common target types.

• For example, we do not need to explicitly state that ‘file.o’ comes from or depends on ‘file.cpp’

• In other words, in this case, we can drop file.cpp from the prerequisites list and further, we do not need to explicitly define the action - make understands what we want.

30

Example ex0550.dir0001 OBJECTS = main.o ClassA.o \0002 ClassB.o ClassC.o00030004 myProgram : $(OBJECTS)0005 g++ -o myProgram $(OBJECTS)00060007 main.o : ClassA.h ClassB.h \0008 ClassC.h00090010 ClassA.o : ClassA.h00110012 ClassB.o : ClassB.h00130014 ClassC.o : ClassC.h00150016 clean :0017 rm myProgram $(OBJECTS)

Lines 7, 10, 12, 14: we have:– removed reference for *.cpp.– removed action

31

More makefile Features

• Makefiles contain one or more of the following elements:– explicit rules– implicit rules– variable definitions– directives– comments

• Explicit Rules describe how to make a target based on prerequisites and commands.

32

More makefile Features

• Implicit Rules describe how to make a class of files based on the filename (meta-rule). – Implicit Rules may also contain prerequisites and

commands.• Directives describe special actions to be taken

while the makefile is being processed.– For example, the ‘include’ directive may be used to

include the contents of one make file into another.• Makefiles can have comments:

– The syntax is the same as Bourne shell– A comment starts with a ‘#’ and continues to the end

of line.

33

Example ex0560.dir

• The next example illustrates the use of implicit rules and directives.

• In our previous example, we saw that we add a new rule for each new class.

• But in this example, all class rules have the same format:ClassA.o : ClassA.h

• If in our project, we can assume that all targets for future classes will have the same format, we can write an implicit rule to support all classes.

34

Example ex0560.dir

• We can create an implicit rule for our needs. It might look like this:

Class%.o : Class%.cpp Class%.h

g++ -c $<

where:• % is used for pattern matching (e.g. A, B, and C in

this case)• $< is an automatic variable used to represent the

name of the first perquisite (in this case, the source file).

35

Example ex0560.dir

• Furthermore, we decide that multiple makefiles need to make use of this implicit rule (and perhaps other implicit rules).

• We can address this need by putting this implicit rule in its own makefile, which we will call ‘rules.mk’ (it could be called anything).

36

Example ex0560.dir• The rules.mk file looks like this:0001: Class%.o : Class%.cpp Class%.h0002: g++ -c $<

• The makefile looks like this:0001 include rules.mk00020003 OBJECTS = main.o ClassA.o \0004 ClassB.o ClassC.o0005 myProgram : $(OBJECTS)0006 g++ -o myProgram $(OBJECTS)0007 main.o : ClassA.h ClassB.h \0008 ClassC.h0009 clean :0010 rm myProgram $(OBJECTS)

• Line 1: include ‘rules.mk’ file.• Notice that we no longer have

explicit rules describing how to build the object file for each class.

37

View Pathing

• GNU make supports the concept of view pathing.

• It is based on the make variable, VPATH.• VPATH contains a list of (parallel) directories for

make to search for prerequisites.• Each developer only needs to have the actual

files he/she is working on in the local directory structure.

• All other files can exist in a parallel directory structure.

38

View Pathing

• Make will effectively merge the directories in the view path presenting a single view to the build tools.

• When the developer is satisfied with the new changes, he/she ‘submits’ these changes so that they become part of the official view.

• Much better than creating a snapshot of all files (with the risk that the snapshot becomes stale).

• View path always the latest and greatest official version.• View pathing can also be used to separate source form

product.

39

Ant

• Ant is a relatively new build tool and is an alternative to make.

• Ant was developed by the Apache Jakarta project.

• Ant is Java based, making it platform independent.

40

Ant

• Make is extended using shell based commands.

• Ant is extended using Java classes.

• Build (make) files in Ant are written in XML.

41

More on Unix Files

• The next portion of this lecture continues the investigation into Unix Files

42

Links

• Links can be viewed as entries in a directory that reference other files.

• In Unix we can create 2 types of links:– physical (hard) links– symbolic (soft) links

43

Physical (Hard) Links

• A physical link references the exact same file (inode) as the file entry itself.

• An inode is a unique number assigned to a file by the file system (we’ll see more about inodes, later in these slides).

• In fact, a file entry in the directory can be viewed as a physical link and is no different than any other physical link.

44

Symbolic (Soft) Links

• A Symbolic Link references a “pointer file” which has its own inode.

• The pointer file points back to the directory entry that references the target file (inode)

• Therefore the inode for the symbolic link and the target file are different.

• If the target file is deleted, then the symbolic link points to nothing…

45

Example ln0001 /export/home/vaughang/trash>ln file1 phys_link0002 /export/home/vaughang/trash>ls -li0003 total 100004 1019465 -rw-r--r-- 2 1825 Feb 4 20:18 file10005 1019465 -rw-r--r-- 2 1825 Feb 4 20:18 phys_link

• Line 1: create physical link to file 1• Line 2: ls –li (long listing with inodes), notice that both file1 and

phys_link reference the same inode.

46

File Systems From The OS Perspective

• Filesystems and disks

• Directories

• inodes

47

File Systems

• Technically, a file system is a formatted disk partition containing a fixed number of blocks.

• The UNIX Directory Structure starts with root and may be composed one or more file systems.

• In a later class, we will learn:– how to create a file system– how to mount and unmount a file system from the

Unix Directory Structure

48

System V File System Layout

• A file system is composed of many, many disk blocks

• This is ‘old’ System V format• Many systems today use more

sophisticated layout (but more complex - we will see it later)

• Block 0 is the boot block• Block 1 is the super block. It

contains information about the file system such as file system size, block size, etc.

• Blocks 2 – lib: blocks containing inodes

• Blocks fdb - ldb: data blocks

Block Contents

0 Boot Block

1 Super Block

2 Start of inode blocks

lib last inode block

fdb first data block

ldb last data block

49

Directories

• Directories are structures that map the user defined file name to an inode.

• A file may be associated with one or more directory entries.

• A file is associated with exactly one inode.• This is why a directory entry referencing a file

can be viewed as yet another physical link to that file.

• To put it another way, a directory is a list of physical links.

50

inodes

• Every file is associated with 1 inode.• The inode keeps of the following file specific information:

– file mode– count of hard links– owner id– group id– time of last file access– time of last file modification– file size– file addresses

• You can see the inode of a file by typing: ls -i

51

inodes (simple view)

• Simple view of inodes• Directory maps file

names to inodes.• Each file has 1 inode• Each file may have

more than 1 directory entry.

• An inode contains a list of disk block addresses.

Dir Entry 1

myfile

Dir Entry 2

yourfile

inode 203

disk addresses

owner, etc

inode 379

disk addresses

owner, etc

Data Block 34

Data Block 379

Data Block 1023

52

inodes (hard link)

• With hard link, 2 or more directory entries point to same inode (i.e. same file).

• inode keeps track of number of hardlinks pointing to it.

• Deleting a file deletes entry from directory.

• If link count is 1 at tome of delete, the inode is also deleted.

Dir Entry 1

myfile

Dir Entry 2

yourfile

inode 379

disk addresses

owner, etc

Data Block 34

Data Block 379

Data Block 1023

53

inodes (soft or symbolic link)

• inode 379 does not know or care about symbolic links pointing to it.

• The symbolic link is not updated if “yourfile” is deleted.

Dir Entry 1

myfile

Dir Entry 2

yourfile

inode 203

pointer file pointing to yourfile

inode 379

disk addresses

owner, etc

Data Block 34

Data Block 379

Data Block 1023

54

inode structure

• An inode can only hold a fixed number of direct data block addresses (10 for Linux).

• For really big files, indirect block address may be used

• For really, really big files secondary indirect block addresses may be used

• For really, really, really big files tertiary indirect block addresses may be used.

55

inode structure• Fields in an inode:

– File/Dir = 1/0 (File = 1, Dir = 0).– Owner = login of user when file was created.– Access = owner/group/world (read/write/execute).– Size = file/directory size in bytes.– Links = Number of hard links to inode.– p0 – p9 = direct block pointer.– I1 = block pointer with one level of indirection.– I2 = block pointer with two levels of indirection.– I3 = block pointer with three levels of indirection.

• A block can contain 256 pointers.• max file block size = 10 + 256 + 256*256 + 256*256*256• (Note: information on this slide comes from:

http://www.ecst.csuchico.edu/~hilzer/csci152/inode.htm)

56

File Systems From The Sys Admin Perspective

• Creating, Mounting and Unmounting file systems will covered in a future class dedicated to disks and devices.

• chown command– allows a user (usually root) to change the owner of a

file.– root can change the owner of all files.– On some systems, the file owner an change the

owner of a file (but can’t change it back).

57

Disk Space and inodes

• A system Administrator needs to be concerned with disk space and inode usage.

• It is possible to have a file system with plenty of space but no more inodes.

• Often this requires recreating the file system

58

Disk Space and inodes

• You can use the df command to check both disk space and inode utilization.

• Example of checking disk space utilization on Linux:

df -h

Filesystem Size Used Avail Use% Mounted on

/dev/hda2 12G 3.4G 8.0G 30% /

/dev/hdb2 4.8G 244M 4.3G 6% /home

/dev/hdb1 19G 15G 4.3G 77% /mnt/msdos/d

59

Disk Space and inodes

• Example of checking inode utilization on Linux:

df -i

Filesystem Inodes IUsed IFree IUse% Mounted on

/dev/hda2 1602496 200983 1401513 13% /

/dev/hdb2 641280 4279 637001 1% /home

/dev/hdb1 0 0 0 - /mnt/msdos/d

60

References

• “GNU Make, A Program for Directed Compilation”, by Richard Stallman and Roland McGrath, 2002.

• http://www.redhat.com/developer/whitepapers/intro_dev/make.html

• http://ant.apache.org/• www.se.cuhk.edu.hk/~seg3420/download/

unix.ppt • http://www.ecst.csuchico.edu/~hilzer/csci152/

inode.htm