Carnegie Mellon

1

Programming in C & Living in Unix

15-213: Introduction to Computer SystemsRecitation 6: Monday, Sept. 30, 2013Marjorie CarlsonSection A

Carnegie Mellon

2

Weekly Update Buffer Lab is due Tuesday (tomorrow), 11:59PM

This is another lab you don’t want to waste your late days on.

Cache Lab is out Tuesday (tomorrow), 11:59 PM Due Thursday October 10th

Cache Lab has two parts.1. Write a 200- to 300-line C program that simulates the

behavior of cache. (Let the coding in C begin!)2. Optimize a small matrix transpose function in order to

minimize the number of cache misses. Next recitation will address part 2. For part 1, pay close

attention in class this week.

Carnegie Mellon

3

Agenda Living in Unix

Beginner The Command Line Basic Commands Pipes & Redirects

Intermediate More Commands The Environment Shell Scripting

Programming in C Refresher Compiling Hunting Memory Bugs

GDB Valgrind

Carnegie Mellon

4

“In the Beginning was the Command Line…” Command Line Interface

“Provides a means of communication between a user and a computer that is based solely on textual input and output.”

Shell A shell is a program that reads and executes the commands

entered on the command line. It provides “an interface between the user and the internal parts of the operating system (at the very core of which is the kernel).”

The original UNIX shell is sh (the Bourne shell). Many other versions exist: bash, csh, etc.

The Linux Information Project: ShellThe Linux Information Project: Command Line

Carnegie Mellon

5

Unix – Beginner: Basic CommandsMoving Around Manipulating Files

ls List directory contents mv Move (rename) files

cd Change working directory cp Copy files (and directories with “-r”)

pwd Display present working directory rm Remove files (or directories with “-r”)

ln Make links between files/directories cat Concatenate and print files

mkdir Make directories chmod Change file permission bits

Working Remotely Looking Up Commands

ssh Secure remote login program man Interface to online reference manuals

sftp Secure remote file transfer program which Shows the full path of shell commands

scp Secure remote file copy program locate Find files by name

Managing Processes Other Important Commands

ps Report current processes status echo Display a line of text

kill Terminate a process exit Cause the shell to exit

jobs Report current shell’s job status history Display the command history list

fg (bg) Run jobs in foreground (background) who Show who is logged on the system

Carnegie Mellon

6

Quick Aside: Using the rm Command rm ./filename – deletes file “filename” in current dir. rm ./*ame – deletes all files in the current directory that

end in “ame.” rm –r ./* – deletes all files and directories inside the

current directory. rm -r ./directory – deletes all files inside the given

directory and the directory itself. sudo rm –rvf /* – deletes the entire hard drive.

DO NOT DO THIS!!! sudo runs the command as root, allowing you to delete anything. -v (verbose) flag will list all the files being deleted. -f (force) flag prevents it from asking for confirmation before

deleting important files.

Carnegie Mellon

7

Quick Aside: Man Page Sections man foo prints the manual page for the foo, where foo is

any user command, system call, C library function, etc.

However, some programs, utilities, and functions have the same name, so you have to specify which section you want (e.g., man 3 printf gets you the man page for the C library function printf, not the Unix command printf).

How do you know which section you want? whatis foo man –f foo man –k foo lists all man pages mentioning “foo.”

Carnegie Mellon

8

Unix – Beginner: Pipes & Redirects A pipe (|) between two commands sends the first

command’s output to the second command as its input. A redirect (< or >) does basically the same thing, but with a

file rather than a process as the input or output.

Remember this slide? Option 1: Pipes

$ cat exploitfile | ./hex2raw | ./bufbomb -t andrewID

Option 2: Redirects$ ./hex2raw < exploitfile > exploit-rawfile$ ./bufbomb -t andrewID < exploit-rawfile

Carnegie Mellon

9

Agenda Living in Unix

Beginner The Command Line Basic Commands Pipes & Redirects

Intermediate More Commands The Environment Shell Scripting

Programming in C Refresher Compiling Hunting Memory Bugs

GDB Valgrind

Carnegie Mellon

10

Unix – Intermediate: More CommandsTransforming Text Useful with Other Commands

cut Remove sections from each line of files (or redirected text)

screen Screen manager with terminal emulation

sed Stream editor for filtering and transforming text

sudo Execute a command as another user (typically root)

tr Translate or delete characters sleep Delay for a specified amount of time

Archiving Looking Up Commands

zip Package and compress files alias Define or display aliases

tar Tar file creation, extraction and manipulation

export (setenv)*

Exposes variables to the shell environment and its following commands

Manipulating File Attributes Searching Files and File Content

touch Change file timestamps (creates empty file, if nonexistent)

find Search for files in a directory hierarchy

umask Set file mode creation mask grep Print lines matching a pattern

* – Bash uses export. Csh uses setenv.

Carnegie Mellon

11

Unix – Intermediate: Environment Variables Your shell’s environment is a set of variables, including

information such as your username, your home directory, even your language. env lists all your environment variables. echo $VARIABLENAME prints a specific one.

$PATH is a colon-delimited list of directories to search for executables.

Variables can be set in config files like .login or .bashrc, or by scripts, in which case the script must use export (bash) or setenv (csh) to export changes to the scope of the shell.

Carnegie Mellon

12

Unix – Intermediate: Shell Scripting Shell scripting is a powerful tool that lets you integrate

command line tools quickly and easily to solve complex problems.

Shell scripts are written in a very simple, interpreted language. The simplest shells scripts are simply a list of commands to execute.

Shell scripting syntax is not at all user-friendly, but shell scripting remains popular for its real power.

Teaching shell scripting is depth is beyond the scope of this course – for more information, check out Kesden’s old 15-123 lectures 3, 4 and 5.

Carnegie Mellon

13

Unix – Intermediate: Shell Scripting To write a shell script:

Open your preferred editor. Type #!/bin/sh. When you run the script, this will tell the

OS that what follows is a shell script. Write the relevant commands. Save the file. (You may want its extension to be .sh.) Right now it’s just a text file – not an executable. You need to chmod +x foo.sh it to make it executable.

Now run it as ./foo.sh!

Carnegie Mellon

14

Unix – Intermediate: Shell Scriptinghello.sh hello.sh with variables

#!/bin/sh

# Prints “Hello, world.” to STDOUTecho “Hello, world.”

#!/bin/sh

str=“Hello, world.”echo $str # Also prints “Hello, world.”

Anything after a ‘#’ is a comment. Variables

Setting a variable takes the form ‘varName=VALUE’. There CANNOT be any spaces to the left and right of the “=“.

Evaluating a variable takes the form “$varName”. Shell scripting is syntactically subtle. (Unquoted strings,

“quoted strings”, ‘single-quoted strings’ and `back quotes` all mean different things!)

Carnegie Mellon

15

Agenda Living in Unix

Beginner The Command Line Basic Commands Pipes & Redirects

Intermediate More Commands The Environment Shell Scripting

Programming in C Refresher Compiling Hunting Memory Bugs

GDB Valgrind

Carnegie Mellon

16

C – Refresher: Things to Remember If you allocate it, free it.

int *array= malloc(5 * sizeof(int));…free(array);

If you use Standard C Library functions that involve pointers, make sure you know if you need to free it. (This will be relevant in proxylab.)

Carnegie Mellon

17

C – Refresher: Things to Remember There is no String type. Strings are just NULL-terminated

char arrays.

Setting pointers to NULL after freeing them is a good habit, so is checking if they are equal to NULL.

Global variables are evil, but if you must use make sure you use extern where appropriate.

Define functions with prototypes, preferably in a header file, for simplicity and clarity.

Carnegie Mellon

18

C – Refresher: Command Line Arguments If you want to pass arguments on the command line to

your C functions, your main function’s parameters must be int main(int argc, char **argv)

argc is the number of arguments. (Always ≥ 1)

argv is an array of char*s (more or less an array of strings) corresponding to the command line input. argv[0] is always the name of the program. The rest of the array are the arguments, parsed on

space.

Carnegie Mellon

19

C – Refresher: Command Line Arguments

$ ls –l private

argc = 3

argv = {“ls”, “-l”, “etc”}

Carnegie Mellon

20

C – Refresher: Headers & Libraries Header files make your

functions available to other source files.

Implementation in .c, and declarations in .h. forward declarations struct prototypes #defines

It should be wrapped in#ifndef LINKEDLIST_H#define LINKEDLIST_H...#endif

bits.h

int bitNor(int, int);int test_bitNor(int, int);int isNotEqual(int, int);int test_isNotEqual(int, int);int anyOddBit();int test_anyOddBit();int rotateLeft(int, int);int test_rotateLeft(int, int);int bitParity(int);int test_bitParity(int);int tmin();int test_tmin();int fitsBits(int, int);int test_fitsBits(int, int);int rempwr2(int, int);int test_rempwr2(int, int);int addOK(int, int);int test_addOK(int, int);int isNonZero(int);int test_isNonZero(int);int ilog2(int);int test_ilog2(int);unsigned float_half(unsigned);unsigned test_float_half(unsigned);unsigned float_twice(unsigned);unsigned test_float_twice(unsignned);

Carnegie Mellon

21

C – Refresher: Headers & Libraries #include <*.h> - Used for including header files found

in the C include path: standard C libraries.

#include “*.h” – Used for including header files in the same directory.

There are command-line compiler flags to control where header files are searched for, but you shouldn’t need to worry about that.

Carnegie Mellon

22

C – Refresher: “Reverse” Demo reverse takes arguments from the command line and

prints each of them backwards. It calls reverse_strdup.

To compile:gcc –Wall –reverse.c –reverse_strdup.c –o reverse

Carnegie Mellon

23

Agenda Living in Unix

Beginner The Command Line Basic Commands Pipes & Redirects

Intermediate More Commands The Environment Shell Scripting

Programming in C Refresher Compiling Hunting Memory Bugs

GDB Valgrind

Carnegie Mellon

24

C – Compiling: Command Linegcc GNU project C and C++ compiler

When compiling C code, all dependencies must be specified. Remember how we just compiled?

gcc –Wall –reverse.c –reverse_strdup.c –o reverse

This will not compile:gcc –Wall –reverse.c –o reverse

Carnegie Mellon

25

C – Compiling: Command Linegcc GNU project C and C++ compiler

gcc does not requires these flags, but they encourage people to write better C code.

Useful Flags

-Wall Enables all construction warnings

-Wextra Enables even more warnings not enabled by Wall

-Werror Treat all warnings as Errors

-pedantic Issue all mandatory diagnostics listed in C standard

-ansi Compiles code according to 1989 C standards

-g Produces debug information (GDB uses this information)

-O1 Optimize

-O2 Optimize even more

-o filename Names output binary file “filename”

Carnegie Mellon

26

C – Compiling: MakefilesMake GNU make utility to maintain groups of programs

Projects can get very complicated very fast, and it can take a long time to have GCC recompile the whole project for a small change.

Makefiles are designed to solve this problem; make recompiles only the parts of the project that have changed and links them to those that haven’t changed.

Makefiles commonly separate compilation (.c .o) and linking (.o’s executable).

Carnegie Mellon

27

C – Compiling: MakefilesMake GNU make utility to maintain groups of programs

Makefiles consist of one or more rules in the following form.

Makefile Rule Format Makefile for “gcc foo.c bar.c baz.c –o myapp”

target : source(s)[TAB]command[TAB]command

myapp: foo.o bar.o baz.o gcc foo.o bar.o baz.o –o myappfoo.o: foo.c foo.h gcc –c foo.cbar.o: bar.c bar.h gcc –c bar.cbaz.o: baz.c baz.h gcc –c baz.c

Carnegie Mellon

28

C – Compiling: Makefiles

Comments are any line beginning with ‘#’

The first line of each command must be a TAB.

If you want help identifying dependencies, you can try: gcc –MM foo.c outputs foo’s dependencies to the console.

makedepend adds dependencies to the Makefile for you, if you already have one. E.g., foo.c bar.c baz.c .

Make GNU make utility to maintain groups of programs

Carnegie Mellon

29

C – Compiling: Makefiles

Use macros – similar to shell variables – to avoid retyping the same stuff for every .o file.

For more information on Makefiles, check out Kesden’s old 15-123 lecture 16.

Make GNU make utility to maintain groups of programs

Makefile Rule Format

CC = gccCCOPT = -g –DDEBUG –DPRINT#CCOPT = -02

foo.o: foo.c foo.h $(CC) $(CCOPT) –c foo.c

Carnegie Mellon

30

Agenda Living in Unix

Beginner The Command Line Basic Commands Pipes & Redirects

Intermediate More Commands The Environment Shell Scripting

Programming in C Refresher Compiling Hunting Memory Bugs

GDB Valgrind

Carnegie Mellon

31

C – Hunting Memory Bugs: GDB Useful for debugging the occasional easy segfault (among

other things!). You’re used to using disas and stepi/nexti to look at

and step through Assembly. If you compile with –g, you can use list and step/next to look at and step through the C, too.

This even works after you’ve segfaulted! Other useful commands:

where (print function stack and lines) up/down (traverse the function stack) display/print variables (like you do now with registers).

Carnegie Mellon

32

C – Hunting Memory Bugs: Valgrind

Great for finding memory problems in C programs. Has a ton of tools: memcheck, leakfind, cachegrind.

Valgrind’s memcheck tool can: Track memory leaks & (definitely/possibly) lost blocks Track origin for uninitialized values Tell you what line of code seems to have been the problem

Syntax (including the verbose flag):valgrind --tool=memcheck -v ./myprogram argsvalgrind –-tool=leak-check=full -v ./myprogram args

valgrind A suite of tools for debugging and profiling programs

Carnegie Mellon

33

Sources and Useful Links The Linux Information Project: Command Line Definition Introduction to Linux: A Hands-On Guide (Garrels)

You should be comfortable with chapters 2, 3, 4 and 5. The On-line Manual Database Kesden’s 15-213: Effective Programming in C and Unix

Lectures 3, 4 and 5 cover the basics of Shell Scripting. Lecture 16 covers Makefiles and lecture 15 covers Valgrind.