240-491 Adv. UNIX: large/131 Advanced UNIX v Objectives of these slides: –learn how to write/manage large programs consisting of multiple files, which

240-491 Adv. UNIX: large/13 1

Advanced UNIXAdvanced UNIX

Objectives of these slides:Objectives of these slides:– learn how to write/manage large programs learn how to write/manage large programs

consisting of multiple files, which may use consisting of multiple files, which may use librarieslibraries

240-491 Special Topics in Comp. Eng. 1Semester 2, 2000-2001

13. Large Programs


OverviewOverview

1.1. Independent Compilation Independent Compilation

2.2. Global Information Global Information

3.3. Header Files Header Files

4. Information Hiding4. Information Hiding

5. The 5. The makemake UtilityUtility

6.6. Libraries Libraries

7. A Static Library Example7. A Static Library Example


1. Independent Compilation1. Independent Compilation

Split a large program into two files:Split a large program into two files:

:int main(){ ... x = maxi(a, b); ...}

:

main.c:

int maxi(int x,int y){ ... return w;}

:

others.c


Compilation and LinkingCompilation and Linking

Compilation:Compilation:$ gcc -c main.c$ gcc -c main.c /* produces main.o *//* produces main.o */

$ gcc -c others.c $ gcc -c others.c /* produces others.o *//* produces others.o */

Linking:Linking:$ gcc -o foo main.o others.o$ gcc -o foo main.o others.o

/* creates /* creates foofoo */ */


Why Split the Code into Files?Why Split the Code into Files?

AdvantagesAdvantages::– reduces the complexity of the complete program by reduces the complexity of the complete program by

organizing it into separate filesorganizing it into separate files

– speeds up recompilation:speeds up recompilation: only recompile what’s neededonly recompile what’s needed linking is faster than compilationlinking is faster than compilation

continued


DisadvantagesDisadvantages::– multiple object files use more storage spacemultiple object files use more storage space

– compiling all the separate compiling all the separate “.c”“.c” files takes files takes longer than a single combined longer than a single combined “.c”“.c” file file


2. Global Information2. Global Information

externextern variables variables– declarationsdeclarations of global variables used across of global variables used across

multiple filesmultiple files

function prototypesfunction prototypes– add to every file where the function is usedadd to every file where the function is used

constants, types, macrosconstants, types, macros– repeat in every file that uses themrepeat in every file that uses them


ExampleExample

#include <stdio.h>#define SIZE 10int maxi(int x, int y) ;int flag;float f:

int main(){ int x, a, b: f = 2.0; x = maxi(a, b); while (x < SIZE)

:}

:

main.c

#include <stdio.h>#define SIZE 10int maxi(int x, int y) ; extern int flag;extern float f:

int maxi(int x, int y){ ... f = f + 1.0; while (x < SIZE) ... return w;}

:

others.c


Definition vs. DeclarationDefinition vs. Declaration

float f; int flag;float f; int flag;

– variable variable definitiondefinition: name, type, and : name, type, and storagestorage

extern float f; extern int flag;extern float f; extern int flag;

– variable variable declarationdeclaration: name, type : name, type

Each variable must have only Each variable must have only one one definitiondefinition, but can have , but can have many declarationsmany declarations..


3. Header Files3. Header Files Use a Use a “.h”“.h” file to put global information in a file to put global information in a

single place, and then use single place, and then use #include#include to to include it in all the include it in all the “.c”“.c” files. files.

Used to store:Used to store:– constant definitionsconstant definitions– type declarations, type declarations, externextern declarationsdeclarations– macrosmacros– function prototypesfunction prototypes


ExampleExample

#define SIZE 10

int maxi(int x, int y);

extern int flag;extern float f:

defs.h


UseUse

#include <stdio.h>#include “defs.h”

int flag;float f:

int main(){ int x, a, b: f = 2.0; x = maxi(a, b); while (x < SIZE)

:}

:

main.c

#include <stdio.h>#include “defs.h”

int maxi(int x, int y){ ... f = f + 1.0; while (x < SIZE) ... return w;}

:

others.c


Compilation / LinkingCompilation / Linking

As before:As before:$ gcc -c main.c$ gcc -c main.c

$ gcc -c others.c$ gcc -c others.c

$ gcc -o foo main.o others.o$ gcc -o foo main.o others.o

defs.hdefs.h must be accessible (i.e. in the same must be accessible (i.e. in the same directory).directory).


4. Information Hiding4. Information Hiding

If a global variable or function definition is If a global variable or function definition is preceded by preceded by staticstatic then it can only be accessed by then it can only be accessed by other things in its file.other things in its file.

:static int table[SIZE]; :static int search(int key){ ... }

int maxi(int a, int b){ ... }

others.c


5. The make Utility5. The make Utility

Write down the compilation dependencies Write down the compilation dependencies between files in a between files in a ““makefilemakefile””..

makemake uses the uses the makefilemakefile to decide what to to decide what to recompile when a file is changed:recompile when a file is changed:– makemake decides on recompilation, decides on recompilation, notnot you you


5.1. The makefile Format5.1. The makefile Format

target file: list of filestarget file: list of files--tab-- command line--tab-- command line

e.g.e.g.foo: main.c others.c defs.hfoo: main.c others.c defs.h

gcc -o foo main.c others.cgcc -o foo main.c others.c

Use:Use:make foomake foo

dependency line

makefile


5.2. A more efficient makefile5.2. A more efficient makefile

Split compilation process into 3 stages:Split compilation process into 3 stages:

foo: main.o others.ofoo: main.o others.ogcc -o foo main.o others.ogcc -o foo main.o others.omain.o: main.c defs.hmain.o: main.c defs.hgcc -c main.cgcc -c main.cothers.o: others.c defs.hothers.o: others.c defs.hgcc -c others.cgcc -c others.c

The compiler will only execute those command lines The compiler will only execute those command lines where the dependent files have changed.where the dependent files have changed.


5.3. Using Built-in Rules5.3. Using Built-in Rules

makemake knows the connection between knows the connection between “.c”“.c”, , “.o”“.o” and and “.h”“.h” files files– this means that the this means that the makefilemakefile can be shorter can be shorter

foo: main.o others.ofoo: main.o others.ogcc -o foo main.o others.ogcc -o foo main.o others.o

main.o: defs.hmain.o: defs.hothers.o: defs.hothers.o: defs.h

or:main.o others.o: defs.h


Changing RulesChanging Rules

makemake uses uses cc -cOcc -cO by default by default

Change by redefining the Change by redefining the CCCC and and CFLAGCFLAG values at the start of values at the start of makefilemakefile::

CC = gccCC = gccCFLAGS = -gCFLAGS = -g /* options become /* options become -cg-cg */ */


5.4. Declaring Variables5.4. Declaring Variables

Create the variable Create the variable OBJSOBJS::OBJS = main.o file1.o file2.o \OBJS = main.o file1.o file2.o \

file3.o file4.ofile3.o file4.o

Use:Use:foo: $(OBJS)foo: $(OBJS)

gcc -o foo $(OBJS)gcc -o foo $(OBJS)$(OBJS): defs.h$(OBJS): defs.h


5.5. Other Uses of makefile5.5. Other Uses of makefile

Common addtions to Common addtions to makefilemakefile::::

test: footest: foorm -f test.outrm -f test.outfoo test.in > test.outfoo test.in > test.out

clean:clean:rm -f $(OBJS)rm -f $(OBJS)rm -f test.outrm -f test.out

all: foo test clean all: foo test clean


UsesUses

make foomake foo

make testmake test

make cleanmake clean

make allmake all


5.6. Options to make5.6. Options to make

OptionsOptions MeaningMeaning-f-f Use a file other than Use a file other than makefile.makefile.

-k-k Try to continue after a command Try to continue after a command has failed.has failed.

-n-n Only print the commands.Only print the commands.

e.g.e.g.make -f bm allmake -f bm all

make -k foomake -k foo

make -n allmake -n all

good for testingthe makefile


5.7. Touching Files5.7. Touching Files

touch filetouch file

– touches the touches the filefile, updating its modification , updating its modification datedate

touch footouch foo

– updates updates foofoo’s modification date’s modification date– make foomake foo will now cause will now cause foofoo to be to be

regenerated since its modification date has regenerated since its modification date has changedchanged


6. Libraries6. Libraries

A A librarylibrary is a collection of useful functions is a collection of useful functions (e.g. for graphics manipulation) in a single (e.g. for graphics manipulation) in a single file.file.

Linux supports two types of library:Linux supports two types of library:– staticstatic libraries (files end in libraries (files end in '.a''.a'))– sharedshared libraries (files end in libraries (files end in '.so''.so'))

continued


When several programs use a When several programs use a static librarystatic library, , they each they each load a copyload a copy into memory into memory– a copy is included inside your program when it a copy is included inside your program when it

is compiledis compiled– the object file can be quite largethe object file can be quite large

– a static library is sometimes called a a static library is sometimes called a archivearchive

continued


When several programs use a When several programs use a shared libraryshared library, , only one copy is loadedonly one copy is loaded into memory into memory– it is loaded when the program is first run (if it is loaded when the program is first run (if

there isn't a copy already in memory)there isn't a copy already in memory)– less memory is usedless memory is used

– Linux is moving over to shared librariesLinux is moving over to shared libraries based on the ELF binary formatbased on the ELF binary format


6.1. The Linking Stage6.1. The Linking Stage Linking example:Linking example:

$ gcc -o foo main$ gcc -o foo main.o.o others others.o.o

The linker does The linker does twotwo main tasks: main tasks:– 1) matches the function calls in 1) matches the function calls in main.omain.o, , others.oothers.o

to the function implementations in those files;to the function implementations in those files;

– 2) matches the function calls to the standard 2) matches the function calls to the standard librarieslibraries (e.g. (e.g. printf()printf()). ).

continued


The compiler knows which libraries to use The compiler knows which libraries to use because of the because of the “.h”“.h” files that you have files that you have included.included.

The library function implementations are The library function implementations are loaded automatically for the standard C loaded automatically for the standard C library (library (libc.solibc.so))– other libraries must be specified on the other libraries must be specified on the gccgcc

command linecommand line

continued


A library consists of object filesA library consists of object files– only the object files containing the functions only the object files containing the functions

used in your program are loadedused in your program are loaded

– this means it is best to separate functions into this means it is best to separate functions into many (small) object files inside a librarymany (small) object files inside a library

– then less of the library will be loadedthen less of the library will be loaded


6.2. Library Locations6.2. Library Locations

Standard libraries are in Standard libraries are in /lib/lib and and /usr/lib/usr/lib

Other Other sharedshared library directories are given in library directories are given in /etc/ld.so.conf/etc/ld.so.conf

– on on calvincalvin: : $ cat /etc/ld.so.conf$ cat /etc/ld.so.conf/usr/X11R6/lib/Xaw3d/usr/X11R6/lib/Xaw3d/usr/X11R6/lib/usr/X11R6/lib/usr/lib/libc5-compat/usr/lib/libc5-compat/lib/libc5-compat/lib/libc5-compat


6.3. Using Libraries6.3. Using Libraries

Use the Use the -l-l option of option of gccgcc to specify a library: to specify a library:$ gcc -o examp examp.c $ gcc -o examp examp.c -lm-lm

gccgcc will try to use a shared library will try to use a shared library ((libm.solibm.so); failing that it will try to use a ); failing that it will try to use a static library (static library (libm.alibm.a))

continued


Use a library's full name:Use a library's full name:$ gcc -o examp examp.c $ gcc -o examp examp.c /usr/lib/libm.so/usr/lib/libm.so

Use a library in a non-standard directory:Use a library in a non-standard directory:$ gcc -o foobar $ gcc -o foobar -L-L /usr/foobar/lib/usr/foobar/lib

fred.c fred.c -lfb-lfb

the library must besupplied last, so allthe functions areknown


6.4. Comparing Static and Shared6.4. Comparing Static and Shared

/* sqrtest.c *//* sqrtest.c */

#include <stdio.h>#include <stdio.h>#include <math.h>#include <math.h>

int main()int main(){{ double x = 9.0; double x = 9.0;

printf("%f\n", sqrt(x)); printf("%f\n", sqrt(x)); return 0; return 0;}}

continued


gcc -o sqrtest sqrtest.c gcc -o sqrtest sqrtest.c -lm-lm$ sqrtest$ sqrtest3.0000003.000000

$ $ lddldd sqrtest sqrtest libm.so.6libm.so.6 => /lib/libm.so.6 (0x40019000) => /lib/libm.so.6 (0x40019000) libc.so.6libc.so.6 => /lib/libc.so.6 (0x40036000) => /lib/libc.so.6 (0x40036000) /lib/ld-linux.so.2/lib/ld-linux.so.2 => =>

/lib/ld-linux.so.2 (0x40000000) /lib/ld-linux.so.2 (0x40000000)

$ ls -alt sqrtest $ ls -alt sqrtest -rwxrwxr-x 1 ad users -rwxrwxr-x 1 ad users 49644964 Dec 1 14:40 sqrtest* Dec 1 14:40 sqrtest*

continued

ldd prints detailson the sharedlibraries used.


$ gcc -o sqrtest sqrtest.c $ gcc -o sqrtest sqrtest.c -static-static -lm -lm$ sqrtest$ sqrtest3.0000003.000000

$ ldd sqrtest$ ldd sqrtest statically linked (ELF) statically linked (ELF)

$ ls -alt sqrtest$ ls -alt sqrtest-rwxrwxr-x 1 ad users -rwxrwxr-x 1 ad users 246373246373 Dec 1 14:40 sqrtest* Dec 1 14:40 sqrtest*

bigger since the libraries areadded in at compile time.

use static libraries


6.5. Shared Library Links6.5. Shared Library Links

When a programmer specifies a shared When a programmer specifies a shared library (e.g. library (e.g. libm.solibm.so), it is actually a link to a ), it is actually a link to a sonamesoname file file– libm.so libm.so libm.so.5libm.so.5

At compile time, the soname filename is At compile time, the soname filename is stored in the object filestored in the object file– no library code, so the object file is smallno library code, so the object file is small

link

continued


When the object file is executed, the When the object file is executed, the soname is used to load the real shared soname is used to load the real shared librarylibrary– the soname file is often just a link to the real the soname file is often just a link to the real

librarylibrary– libm.so.5libm.so.5 libm.so.5.0.9libm.so.5.0.9

link


6.7. Some Common Shared Libraries6.7. Some Common Shared Libraries

libc.solibc.so standard C librarystandard C library libg++.solibg++.so standard C++ librarystandard C++ library libgdbm.solibgdbm.so GNU dbm databaseGNU dbm database libncurses.solibncurses.so terminal-based graphicsterminal-based graphics libtiff.solibtiff.so TIFF image TIFF image

manipulationmanipulation


6.8. Dynamic Libraries6.8. Dynamic Libraries

A dynamic library is a A dynamic library is a variantvariant of a shared of a shared librarylibrary– the library code is loaded at run time, the library code is loaded at run time, butbut under under

the control of the program itselfthe control of the program itself– the program can also unload a librarythe program can also unload a library

– useful for programs that occasionally use very useful for programs that occasionally use very big librariesbig libraries


6.9. The old '.sa' Format6.9. The old '.sa' Format

Older versions of Linux used the Older versions of Linux used the a.outa.out binary file formatbinary file format– shared library filenames in that file format shared library filenames in that file format

end with end with '.sa''.sa'

– you should not see them in new Linux'syou should not see them in new Linux's


7. A Static Library Example7. A Static Library Example

Static libraries are fine for libraries which Static libraries are fine for libraries which will not be heavily used.will not be heavily used.

Creating shared libraries still varies Creating shared libraries still varies between UNIXesbetween UNIXes– but but staticstatic library creation has been library creation has been standardizedstandardized


7.1. A Static Library for Graphics7.1. A Static Library for Graphics

graphics.hgraphics.h::typedef struct {typedef struct { int x, y, z; int x, y, z; ... ...} Image;} Image;

void display_im(Image im);void display_im(Image im);void move_im(Image im, int x, void move_im(Image im, int x,

int y, int z);int y, int z);::


Function ImplementationsFunction Implementations

idisplay.cidisplay.c::#include “graphics.h”#include “graphics.h”void display_im(Image im)void display_im(Image im){ ... }{ ... }

imove.cimove.c::#include “graphics.h”#include “graphics.h”void move_im(Image im, int x, void move_im(Image im, int x,

int y, int z)int y, int z){ ... }{ ... }


7.2. Creating the Static Library7.2. Creating the Static Library Compile:Compile:

$ gcc -c idisplay.c$ gcc -c idisplay.c

$ gcc -c imove.c$ gcc -c imove.c

Build a static library called Build a static library called glib.aglib.a::$ $ ar ruvsar ruvs glib.a idisplay.o imove.o glib.a idisplay.o imove.o

The library is made from two object files The library is made from two object files andand a a symbol definition filesymbol definition file (a list of function (a list of function names, constants used in the “.o” files).names, constants used in the “.o” files).


ar Optionsar Options

OperatorOperator MeaningMeaningrr Replace files.Replace files.uu Update files if changed.Update files if changed.vv Verbose.Verbose.ss Create symbol definition file.Create symbol definition file.


Other CommandsOther Commands

Some versions of Some versions of arar do not have an do not have an “s” “s”

option, so you must use option, so you must use ranlibranlib::

$ ar ruv glib.a idisplay.o imove.o$ ar ruv glib.a idisplay.o imove.o

$ $ ranlibranlib glib.a glib.a

Listing the object files in a library with Listing the object files in a library with “t”“t”::$ $ ar tvar tv glib.a glib.a /* verbose listing */ /* verbose listing */


7.3. Using the Library7.3. Using the Library

#include <stdio.h>#include “graphics.h” :int main(){ Image i; ... display_im(i); foop(i); move_im(i, 2,3,4): ...}

main.c

#include <stdio.h>#include “graphics.h” :void foop(Image j){ Image i; ... display_im(i); display_im(j); ...}

foop.c


Compilation / LinkingCompilation / Linking Compile and link:Compile and link:

$ gcc -c main.c$ gcc -c main.c

$ gcc -c foop.c$ gcc -c foop.c

$ gcc -o foo main.o foop.o $ gcc -o foo main.o foop.o glib.aglib.a

The The “.a”“.a” file must be at the end so that all the file must be at the end so that all the symbols uses in the object files are known.symbols uses in the object files are known.

The linker links in the The linker links in the filesfiles from the from the “.a”“.a” library library which contain functions, constants, etc. used by which contain functions, constants, etc. used by the the “.c”“.c” code. code.

Documents

240-491 Adv. UNIX: large/131 Advanced UNIX v Objectives of these slides: –learn how to write/manage large programs consisting of multiple files, which