Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
UNIVERSIDAD TECNICA FEDERICO SANTA MARIADEPARTAMENTO DE F ISICA
UTFSM ATLAS GRID TEAM
Nonrecursive Makefile for ClasTool
Sebastian Mancilla [email protected]
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 1 / 20
Makefiles and make command
make is a utility intended to automate and optimize the construction ofprograms.
A descriptor file named Makefile describes the relationship amongsource files and provides commands for updating each file.
make invokes Makefile to automatically rebuild a programwhenever one of the source files is modified.
It only recompiles the files that were affected by changes, thussaving compiling time.
This helps reduce the likelihood of human errors when makingentries from the command line.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 2 / 20
Makefiles and make command
make is a utility intended to automate and optimize the construction ofprograms.
A descriptor file named Makefile describes the relationship amongsource files and provides commands for updating each file.
make invokes Makefile to automatically rebuild a programwhenever one of the source files is modified.
It only recompiles the files that were affected by changes, thussaving compiling time.
This helps reduce the likelihood of human errors when makingentries from the command line.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 2 / 20
Makefiles and make command
make is a utility intended to automate and optimize the construction ofprograms.
A descriptor file named Makefile describes the relationship amongsource files and provides commands for updating each file.
make invokes Makefile to automatically rebuild a programwhenever one of the source files is modified.
It only recompiles the files that were affected by changes, thussaving compiling time.
This helps reduce the likelihood of human errors when makingentries from the command line.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 2 / 20
Makefiles and make command
make is a utility intended to automate and optimize the construction ofprograms.
A descriptor file named Makefile describes the relationship amongsource files and provides commands for updating each file.
make invokes Makefile to automatically rebuild a programwhenever one of the source files is modified.
It only recompiles the files that were affected by changes, thussaving compiling time.
This helps reduce the likelihood of human errors when makingentries from the command line.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 2 / 20
Makefiles and make commandExample
The executable project1 is built from the following sources files:
project||_ main.c|_ data.h|_ data.c|_ io.h|_ io.c
The source main.c includes data.h and io.h.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 3 / 20
Makefiles and make commandExample
The dependencies between files are shown in the following tree:
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 4 / 20
Makefiles and make commandExample
And this is a simple Makefile to build the executable:
project1: main.o data.o io.ogcc main.o data.o io.o -o project1
main.o: data.h io.h main.cgcc -c main.c
data.o: data.c data.hgcc -c data.c
io.o: io.h io.ccc -c io.c
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 5 / 20
Makefiles and make commandExample
make runs the rule for first target project1 and figures itsdependencies on data.o, main.o and io.o.
make next checks if any of the three object files are listed as targets. Ifyes, it runs the rule for first prerequisite, that is, main.o, to find itsdependencies.
make checks whether the prerequisites of main.o have furtherdependencies. If not, as in the example, it checks if main.o is up todate. If not, it runs the command for main.o by compiling main.c to getthe object file.
make looks at the targets data.o and io.o and compiles these objectfiles in a similar fashion.
make returns to first target project1. Since it now has all up-to-dateobject files for the rule, it executes the command to build project1.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 6 / 20
Recursive and nonrecursive Makefiles
Traditionally the software development projects use a hierarchy of directoriescontaining source files for the modules which make up the project.
In the recursive make method each of the sub-directories contains aMakefile which describes the rules and instructions for the makeprogram.The complete project build is done by arranging for the top-levelMakefile to change directory into each of the sub-directories andrecursively invoke make.
A nonrecursive make uses one single top-level Makefile to build theproject and one make include file for each sub-directory containing filelists and module-specific rules.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 7 / 20
Recursive and nonrecursive Makefiles
Traditionally the software development projects use a hierarchy of directoriescontaining source files for the modules which make up the project.
In the recursive make method each of the sub-directories contains aMakefile which describes the rules and instructions for the makeprogram.The complete project build is done by arranging for the top-levelMakefile to change directory into each of the sub-directories andrecursively invoke make.
A nonrecursive make uses one single top-level Makefile to build theproject and one make include file for each sub-directory containing filelists and module-specific rules.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 7 / 20
Recursive Makefile issues
Recursive makefiles are bad primarily because they partition thedependency tree into several trees.
This prevents dependencies between make instances from beingexpressed correctly.
Also causes (parts of) the dependency tree to be recalculatedmultiple times, which is a performance issue in the end.
This affects both phases of the operation of make:
It causes make to construct an inaccurate dependency tree.
It forces make to traverse the tree in an inappropriate order.
Simple example of dependency problem
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 8 / 20
Recursive Makefile issues
Other problems include:
Recursive makes which take forever to work out that they need todo nothing.
Recursive makes which do too much, or too little.
Recursive makes which are overly sensitive to changes in thesource code and require constant Makefile intervention to keepthem working.
It is very hard to get the order of the recursion into thesub-directories correct.
It is often necessary to do more than one pass over thesub-directories to build the whole system.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 9 / 20
Using nonrecursive Makefiles
The use of a whole project make is not as difficult to put intopractice as it may at first appear.
It solves the problems found in the recursive make approach.
It handles dependencies correctly. Treating the whole tree as asingle entity is really the right way.Specific targets can be built (say a particular program) correctly,since make always creates a full dependency graph.It is faster.
It requires a paradigm shift for developers used to recursive make.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 10 / 20
Using nonrecursive Makefiles
The use of a whole project make is not as difficult to put intopractice as it may at first appear.
It solves the problems found in the recursive make approach.
It handles dependencies correctly. Treating the whole tree as asingle entity is really the right way.Specific targets can be built (say a particular program) correctly,since make always creates a full dependency graph.It is faster.
It requires a paradigm shift for developers used to recursive make.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 10 / 20
Using nonrecursive Makefiles
The use of a whole project make is not as difficult to put intopractice as it may at first appear.
It solves the problems found in the recursive make approach.
It handles dependencies correctly. Treating the whole tree as asingle entity is really the right way.Specific targets can be built (say a particular program) correctly,since make always creates a full dependency graph.It is faster.
It requires a paradigm shift for developers used to recursive make.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 10 / 20
Using nonrecursive Makefiles
The use of a whole project make is not as difficult to put intopractice as it may at first appear.
It solves the problems found in the recursive make approach.
It handles dependencies correctly. Treating the whole tree as asingle entity is really the right way.Specific targets can be built (say a particular program) correctly,since make always creates a full dependency graph.It is faster.
It requires a paradigm shift for developers used to recursive make.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 10 / 20
ROOT
The ROOT Makefile itself was structured as described in thepaper Recursive Make Considered Harmful.
The main philosophy is that it is better to have a single largeMakefile describing the entire project than many small Makefiles,one for each sub-project, that are recursively called from the mainMakefile. By cleverly using the include mechanism the singleMakefile solution is as modular as the recursive approach withoutthe problems of incomplete dependency graphs.
The single Makefile is fast.About 1 second to check if anything needs to be recompiled.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 11 / 20
My approach for ClasTool
I just took the old ClasTool Makefile and turned it into anonrecursive Makefile.
My purpose was to eliminate the problems associated with using arecursive Makefile.
Because that is not the right way.
All the dictionary, objects and dependency files are created insideof their source directory, so in the new approach this needs aseries of specific rules for every directory.
The automatic dependency generation was inefficient, and with acouple of issues.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 12 / 20
My approach for ClasTool
I just took the old ClasTool Makefile and turned it into anonrecursive Makefile.
My purpose was to eliminate the problems associated with using arecursive Makefile.
Because that is not the right way.
All the dictionary, objects and dependency files are created insideof their source directory, so in the new approach this needs aseries of specific rules for every directory.
The automatic dependency generation was inefficient, and with acouple of issues.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 12 / 20
My approach for ClasTool
I just took the old ClasTool Makefile and turned it into anonrecursive Makefile.
My purpose was to eliminate the problems associated with using arecursive Makefile.
Because that is not the right way.
All the dictionary, objects and dependency files are created insideof their source directory, so in the new approach this needs aseries of specific rules for every directory.
The automatic dependency generation was inefficient, and with acouple of issues.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 12 / 20
My approach for ClasTool
I just took the old ClasTool Makefile and turned it into anonrecursive Makefile.
My purpose was to eliminate the problems associated with using arecursive Makefile.
Because that is not the right way.
All the dictionary, objects and dependency files are created insideof their source directory, so in the new approach this needs aseries of specific rules for every directory.
The automatic dependency generation was inefficient, and with acouple of issues.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 12 / 20
My approach for ClasTool
I just took the old ClasTool Makefile and turned it into anonrecursive Makefile.
My purpose was to eliminate the problems associated with using arecursive Makefile.
Because that is not the right way.
All the dictionary, objects and dependency files are created insideof their source directory, so in the new approach this needs aseries of specific rules for every directory.
The automatic dependency generation was inefficient, and with acouple of issues.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 12 / 20
Implementation
Three main files in the project root directory:
Makefile system: it has all the system-dependent definitions(almost what Makefile top used to be).
Makefile: the project’s nonrecursive Makefile.
Makefile templates: it has all the templates to build thespecific rules for every directory, and the macros used by themodule.mk files.
A module.mk file in each directory for building the libraries.
All this files are included in the main Makefile.
All share the same structure.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 13 / 20
Implementation
Each module.mk file has the following lines:local_class := $(wildcard $(module)/*.cc)local_files :=lib_name := $(module)lib_deps :=extra_libs :=extra_flags :=
$(eval $(call make-module,$(module),$(lib_name),$(lib_deps)))$(eval $(call make-libs,$(module),$(lib_name),$(lib_deps), \
$(local_class),$(local_files), \$(extra_libs)))
$(eval $(call make-objs,$(module),$(extra_flags)))
For every directory the only thing that needs to be changed is thevalues of the variables.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 14 / 20
Implementation
Each module.mk file has the following lines:local_class := $(wildcard $(module)/*.cc)local_files :=lib_name := $(module)lib_deps :=extra_libs :=extra_flags :=
$(eval $(call make-module,$(module),$(lib_name),$(lib_deps)))$(eval $(call make-libs,$(module),$(lib_name),$(lib_deps), \
$(local_class),$(local_files), \$(extra_libs)))
$(eval $(call make-objs,$(module),$(extra_flags)))
For every directory the only thing that needs to be changed is thevalues of the variables.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 14 / 20
Implementation
Each module.mk file has the following lines:local_class := $(wildcard $(module)/*.cc)local_files :=lib_name := $(module)lib_deps := ClasBanks VirtualReaderextra_libs := -L$(ROOTLIBDIR) -lPhysics -lTreeextra_flags :=
$(eval $(call make-module,$(module),$(lib_name),$(lib_deps)))$(eval $(call make-libs,$(module),$(lib_name),$(lib_deps), \
$(local_class),$(local_files), \$(extra_libs)))
$(eval $(call make-objs,$(module),$(extra_flags)))
For every directory the only thing that needs to be changed is thevalues of the variables.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 15 / 20
Implementation
The macros make-module, make-libs and make-objs aredefined in Makefile templates. Example of a macro definition
By using the eval function, all the necessary rules are autogenerated by make itself according to the above macros.
If some rules must be changed, this is done in one place:Makefile templates.
Auto-dependency generation is significantly improved using theadvanced method invented by Tom Tromey1.
1http://make.paulandlesley.org/autodep.htmlSebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 16 / 20
Results
I hope all the work be useful.
As ClasTool, it is not finished, and it could be improved.
But right now it works very good.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 17 / 20
Results
I hope all the work be useful.
As ClasTool, it is not finished, and it could be improved.
But right now it works very good.
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 17 / 20
References
Recursive Make Considered Harmful, Peter Miller, 1997http://miller.emu.id.au/pmiller/books/rmch/
Managing Projects with GNU make, 3rd Edition, RobertMecklenburg, 2004http://www.makelinux.net/make3/main.html
My Git repository (here you can find the sources)http://github.com/smancill/ClasTool_Makefile
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 18 / 20
Nonrecursive Makefile dependencies problem
Suppose make is called first for ModuleA and then for ModuleB.
What happens if we modified file C2 and we call make command?
Back
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 19 / 20
make-libs macrodefine make-libsSOURCES_DEPS += $(call dep-list,$(1),$(4) $(5))SOURCES_DICTS += $(call dict-list,$(1),$(4))
$(SLIBDIR)/lib$(2).$(DllSuf): $(call obj-list,$(1),$(4) $(5)) \$(call dict-obj-list,$(1),$(4))
@echo CREATING SHARED LIBRARY $$@ifeq (,$(7))
$(LD) $(SOFLAGS) $(LDFLAGS) -o $$@ \$$ˆ $(6) $(if $3,-L$(SLIBDIR) $(addprefix -l,$3))
else$(7) $(SOFLAGS) $(LDFLAGS) -o $$@ \$$ˆ $(6) $(if $3,-L$(SLIBDIR) $(addprefix -l,$3))
endif$(POST_LINK_COMMAND)
$(LIBDIR)/lib$(2).a: $(call obj-list,$(1),$(4) $(5)) \$(call dict-obj-list,$(1),$(4))
@echo CREATING LIBRARY $$@ar r $$@ $$ˆ
endef
Back
Sebastian Mancilla (UTFSM) ClasTool Makefile 27 de abril de 2010 20 / 20