61
Background Buffer Overflow Integer and Heap Overflow Summary CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer and Heap Overflow Zhiqiang Lin Department of Computer Science University of Texas at Dallas April 11 th , 2012

CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Embed Size (px)

Citation preview

Page 1: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

CS 6V81-05: System Security and Malicious Code Analysis

Buffer Overflow, Integer and Heap Overflow

Zhiqiang Lin

Department of Computer ScienceUniversity of Texas at Dallas

April 11th, 2012

Page 2: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Outline

1 Background

2 Buffer Overflow

3 Integer and Heap Overflow

4 Summary

Page 3: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Outline

1 Background

2 Buffer Overflow

3 Integer and Heap Overflow

4 Summary

Page 4: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

IA32 Stack

Region of memorymanaged with stackdisciplineGrows toward loweraddressesRegister %esp indicateslowest stack address

address of top element

IA32 Stack

Stack Pointer %esp

Stack “Top”

Stack “Bottom”

Page 5: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

IA32 Stack Pushing

Pushing

pushl srcFetch operand at SrcDecrement %esp by 4Write operand ataddress given by %esp

IA32 Stack Pushing

Stack “Top”

Stack “Bottom”

Stack Pointer %esp

-4

Page 6: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

IA32 Stack Popping

Popping

popl destRead operand ataddress given by %espIncrement %esp by 4Write to Dest

IA32 Stack Popping

Stack Pointer %esp

Stack “Top”

Stack “Bottom”

+4

Page 7: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Procedure Call Example

804854e: e8 3d 06 00 00 call 8048b90 <main>8048553: 50 pushl %eax

Carnegie Mellon

4

%esp

%eip

%esp

%eip 0x804854e

0x108

0x108

0x10c

0x110

0x104

0x804854e

123

Procedure Call Example

0x108

0x10c

0x110

123

0x108

804854e: e8 3d 06 00 00 call 8048b90 <main>

8048553: 50 pushl %eax

%eip is program counter

Page 8: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Procedure Call Example

804854e: e8 3d 06 00 00 call 8048b90 <main>8048553: 50 pushl %eax

Carnegie Mellon

5

%esp

%eip

%esp

%eip 0x804854e

0x108

0x108

0x10c

0x110

0x104

0x804854e

123

Procedure Call Example

0x108

0x10c

0x110

123

0x108

call 8048b90

804854e: e8 3d 06 00 00 call 8048b90 <main>

8048553: 50 pushl %eax

%eip is program counter

Page 9: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Procedure Call Example

804854e: e8 3d 06 00 00 call 8048b90 <main>8048553: 50 pushl %eax

Carnegie Mellon

6

%esp

%eip

%esp

%eip 0x804854e

0x108

0x108

0x10c

0x110

0x104

0x804854e

123

Procedure Call Example

0x108

0x10c

0x110

123

0x108

call 8048b90

804854e: e8 3d 06 00 00 call 8048b90 <main>

8048553: 50 pushl %eax

0x104

%eip is program counter

Page 10: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Procedure Call Example

804854e: e8 3d 06 00 00 call 8048b90 <main>8048553: 50 pushl %eax

Carnegie Mellon

7

%esp

%eip

%esp

%eip 0x804854e

0x108

0x108

0x10c

0x110

0x104

0x804854e

0x8048553

123

Procedure Call Example

0x108

0x10c

0x110

123

0x108

call 8048b90

804854e: e8 3d 06 00 00 call 8048b90 <main>

8048553: 50 pushl %eax

0x104

%eip is program counter

Page 11: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Procedure Call Example

804854e: e8 3d 06 00 00 call 8048b90 <main>8048553: 50 pushl %eax

Carnegie Mellon

8

%esp

%eip

%esp

%eip 0x804854e

0x108

0x108

0x10c

0x110

0x104

0x804854e

0x8048553

123

Procedure Call Example

0x108

0x10c

0x110

123

0x108

call 8048b90

804854e: e8 3d 06 00 00 call 8048b90 <main>

8048553: 50 pushl %eax

0x8048b90

0x104

%eip is program counter

Page 12: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Procedure Return Example

8048591: c3 ret

Carnegie Mellon

9

%esp

%eip

0x104

%esp

%eip 0x8048591 0x8048591

0x104 0x104

0x108

0x10c

0x110

0x8048553

123

Procedure Return Example

0x108

0x10c

0x110

123

ret

8048591: c3 ret

0x108

%eip is program counter

0x8048553

0x8048553

Page 13: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Call Chain Example Carnegie Mellon

10

Call Chain Example

Code Structure

yoo(…)

{

who();

}

who(…)

{

• • •

amI();

• • •

amI();

• • •

}

amI(…)

{

amI();

}

yoo

who

amI

amI

amI

Call Chain

Procedure amI recursive

amI

Page 14: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Call Chain Example

Contents

Local variablesReturn informationTemporary space

Management

Space allocated when enter procedure

“Set-up” code

Deallocated when return

“Finish” code

Pointers

Stack pointer %esp indicates stack topFrame pointer %ebp indicates start ofcurrent frame

yoo

who

Stack Frames

amI

Page 15: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Stack Operation

Stack Pointer %esp

yoo

• Frame

Pointer %ebp

yoo

Call Chain yoo(…)

{

who();

}

Page 16: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Stack Operation

Stack Pointer %esp

yoo

who

Frame Pointer %ebp

yoo

who

Call Chain who(…)

{

• • •

amI();

• • •

amI();

• • •

}

Page 17: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Stack Operation

Stack Pointer %esp

yoo

who

amI

Frame Pointer %ebp

yoo

who

amI

Call Chain amI(…)

{

amI();

}

Page 18: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Stack Operation

Stack Pointer %esp

yoo

who

amI

Frame Pointer %ebp

yoo

who

amI

Call Chain amI(…)

{

amI();

}

amI

amI

Page 19: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Stack Operation

Stack Pointer %esp

yoo

who

amI

Frame Pointer %ebp

yoo

who

amI

Call Chain amI(…)

{

amI();

}

amI

amI

amI

amI

Page 20: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Stack Operation

Stack Pointer %esp

yoo

who

amI

Frame Pointer %ebp

yoo

who

amI

Call Chain amI(…)

{

amI();

}

amI

amI

amI

Page 21: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Stack Operation

Stack Pointer %esp

yoo

who

amI

Frame Pointer %ebp

yoo

who

amI

Call Chain amI(…)

{

amI();

}

amI

amI

Page 22: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Stack Operation

Stack Pointer %esp

yoo

who

Frame Pointer %ebp

yoo

who

Call Chain who(…)

{

• • •

amI();

• • •

amI();

• • •

} amI

amI

amI

Page 23: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Stack Operation

Stack Pointer %esp

yoo

who

amI

Frame Pointer %ebp

yoo

who

Call Chain amI(…)

{

} amI

amI

amI

amI

Page 24: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Stack Operation

Stack Pointer %esp

yoo

who

Frame Pointer %ebp

yoo

who

Call Chain who(…)

{

• • •

amI();

• • •

amI();

• • •

} amI

amI

amI

amI

Page 25: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Stack Operation

yoo(…)

{

who();

}

Stack Pointer %esp

yoo

• Frame

Pointer %ebp

yoo

who

Call Chain

amI

amI

amI

amI

Page 26: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

IA32/Linux Stack Frame

Current Stack Frame (“Top” toBottom)

Parameters for function aboutto call

“Argument build”

Local variables

If can’t keep in registers

Saved register contextOld frame pointer

Caller Stack Frame

Return address

Pushed by callinstruction

Arguments for this call

Carnegie Mellon

23

IA32/Linux Stack Frame

Current Stack Frame (“Top” to Bottom) Parameters for function about to

call

“Argument build”

Local variables

If can’t keep in registers

Saved register context

Old frame pointer

Caller Stack Frame Return address

Pushed by call instruction

Arguments for this call Stack Pointer (%esp)

Frame Pointer (%ebp)

Return Addr

Saved

Registers

+

Local

Variables

Argument

Build

Old %ebp

Arguments

Caller

Frame

Page 27: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

IA32 Linux Memory Layout

Stack

Runtime stack (8MB limit)E. g., local variables

Heap

Dynamically allocated storageWhen call malloc(), calloc(),new()

Data

Statically allocated dataE.g., arrays & strings declaredin code

Text

Executable machineinstructionsRead-only

Upper 2 hex digits = 8 bits of address

FF

00

Stack

Text

Data

Heap

08

8MB

not drawn to scale

Page 28: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Memory Allocation Example

char big_array[1<<24]; /* 16 MB */char huge_array[1<<28]; /* 256 MB*/

int beyond;char *p1, *p2, *p3, *p4;

int useless() { return 0; }

int main(){p1 = malloc(1 <<28); /* 256 MB */p2 = malloc(1 << 8); /* 256 B */p3 = malloc(1 <<28); /* 256 MB */p4 = malloc(1 << 8); /* 256 B *//* Some print statements ... */}

Where does everything go?

FF

00

Stack

Text

Data

Heap

08

not drawn to scale

Where does everything go?

Page 29: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

IA32 Example AddressesCarnegie Mellon

26

IA32 Example Addresses

$esp 0xffffbcd0

p3 0x65586008

p1 0x55585008

p4 0x1904a110

p2 0x1904a008

&p2 0x18049760

&beyond 0x08049744

big_array 0x18049780

huge_array 0x08049760

main() 0x080483c6

useless() 0x08049744

final malloc() 0x006be166

address range ~232

FF

00

Stack

Text

Data

Heap

08

80

not drawn to scale

malloc() is dynamically linked

address determined at runtime

FF

00

Stack

Text

Data

Heap

08

80

not drawn to scale

Page 30: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Outline

1 Background

2 Buffer Overflow

3 Integer and Heap Overflow

4 Summary

Page 31: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Internet Worm and IM War

November, 1988

Internet Worm attacks thousands of Internet hosts.How did it happen?

Page 32: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Internet Worm and IM War

November, 1988Internet Worm attacks thousands of Internet hosts.How did it happen?

July, 1999Microsoft launches MSN Messenger (instant messagingsystem).Messenger clients can access popular AOL InstantMessaging Service (AIM) servers

Carnegie Mellon

29

Internet Worm and IM War

November, 1988 Internet Worm attacks thousands of Internet hosts.

How did it happen?

July, 1999 Microsoft launches MSN Messenger (instant messaging system).

Messenger clients can access popular AOL Instant Messaging Service (AIM) servers

AIM server

AIM client

AIM client

MSN client

MSN server

Page 33: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Internet Worm and IM War (cont.)

August 1999

Mysteriously, Messenger clients can no longer access AIMservers.Microsoft and AOL begin the IM war:

AOL changes server to disallow Messenger clientsMicrosoft makes changes to clients to defeat AOL changes.At least 13 such skirmishes.

How did it happen?

The Internet Worm and AOL/Microsoft War were bothbased on stack buffer overflow exploits!

many library functions do not check argument sizes.allows target buffers to overflow.

Page 34: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

String Library Code

Implementation of Unix function gets()

/* Get string from stdin */char *gets(char *dest){

int c = getchar();char *p = dest;while (c != EOF && c != ’\n’) {

*p++ = c;c = getchar();

}

*p = ’\0’;return dest;

}

No way to specify limit on number of characters to read

Similar problems with other library functions

strcpy, strcat: Copy strings of arbitrary lengthscanf, fscanf, sscanf, when given %s conversionspecification

Page 35: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Vulnerable Buffer Code

/* Echo Line */void echo(){

char buf[4]; /* Way too small! */gets(buf);puts(buf);

}

void call_echo() {echo();

}

unix>./bufdemoType a string:12345671234567

unix>./bufdemoType a string:12345678Segmentation Fault

unix>./bufdemoType a string:123456789ABCSegmentation Fault

Page 36: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Buffer Overflow Disassembly

echo80485c5: 55 push %ebp80485c6: 89 e5 mov %esp,%ebp80485c8: 53 push %ebx80485c9: 83 ec 14 sub $0x14,%esp80485cc: 8d 5d f8 lea 0xfffffff8(%ebp),%ebx80485cf: 89 1c 24 mov %ebx,(%esp)80485d2: e8 9e ff ff ff call 8048575 <gets>80485d7: 89 1c 24 mov %ebx,(%esp)80485da: e8 05 fe ff ff call 80483e4 <puts@plt>80485df: 83 c4 14 add $0x14,%esp80485e2: 5b pop %ebx80485e3: 5d pop %ebp80485e4: c3 ret

call echo80485eb: e8 d5 ff ff ff call 80485c5 <echo>80485f0: c9 leave80485f1: c3 ret

Page 37: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Buffer Overflow Stack

Buffer Overflow Stack

/* Echo Line */

void echo()

{

char buf[4]; /* Way too small! */

gets(buf);

puts(buf);

}

Return Address

Saved %ebp %ebp

Stack Frame for main

Stack Frame for echo

[3] [2] [1] [0] buf

Before call to gets

Saved %ebx

/* Echo Line */void echo(){

char buf[4]; /* Way too small! */gets(buf);puts(buf);

}

echo:pushl %ebp # Save %ebp on stackmovl %esp, %ebppushl %ebx # Save %ebxsubl $20, %esp # Allocate stack spaceleal -8(%ebp),%ebx # Compute buf as %ebp-8movl %ebx, (%esp) # Push buf on stackcall gets # Call gets. . .

Page 38: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Buffer Overflow Stack Example

unix> gdb bufdemo(gdb) break echoBreakpoint 1 at 0x80485c9(gdb) runBreakpoint 1, 0x80485c9 in echo ()(gdb) print /x $ebp$1 = 0xffffd678(gdb) print /x *(unsigned *)$ebp$2 = 0xffffd688(gdb) print /x *((unsigned *)$ebp + 1)$3 = 0x80485f0

Carnegie Mellon

35

Buffer Overflow Stack Example

unix> gdb bufdemo

(gdb) break echo

Breakpoint 1 at 0x80485c9

(gdb) run

Breakpoint 1, 0x80485c9 in echo ()

(gdb) print /x $ebp

$1 = 0xffffd678

(gdb) print /x *(unsigned *)$ebp

$2 = 0xffffd688

(gdb) print /x *((unsigned *)$ebp + 1)

$3 = 0x80485f0

80485eb: e8 d5 ff ff ff call 80485c5 <echo>

80485f0: c9 leave

0xffffd678

buf

0xffffd688

Return Address

Saved %ebp

Stack Frame for main

Stack Frame for echo

[3] [2] [1] [0]

Stack Frame for main

Stack Frame for echo

xx xx xx xx buf

ff ff d6 88

08 04 85 f0

Before call to gets Before call to gets

Saved %ebx Saved %ebx

Page 39: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Buffer Overflow Stack Example #1Carnegie Mellon

36

Buffer Overflow Example #1

Overflow buf, and corrupt %ebx, but no problem

Stack Frame for echo

xx xx xx xx buf

Stack Frame for echo

34 33 32 31 buf

00 37 36 35

Before call to gets Input 1234567

0xffffd678

0xffffd688 Stack Frame for main

ff ff d6 88

08 04 85 f0

0xffffd678

0xffffd688 Stack Frame for main

ff ff d6 88

08 04 85 f0

Saved %ebx

Page 40: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Buffer Overflow Stack Example #2Carnegie Mellon

37

Buffer Overflow Example #2

Base pointer corrupted

Stack Frame for echo

xx xx xx xx buf

Stack Frame for echo

34 33 32 31 buf

00

38 37 36 35

Before call to gets Input 12345678

. . .

80485eb: e8 d5 ff ff ff call 80485c5 <echo>

80485f0: c9 leave # Set %ebp to corrupted value

80485f1: c3 ret

0xffffd678

0xffffd688 Stack Frame for main

ff ff d6 88

08 04 85 f0

0xffffd678

0xffffd688 Stack Frame for main

ff ff d6

08 04 85 f0

Saved %ebx

Page 41: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Buffer Overflow Stack Example #3Carnegie Mellon

34

Buffer Overflow Example #3

Return address corrupted

Stack Framefor echo

xx xx xx xx buf

Stack Framefor echo

34 33 32 31 buf

43 42 41 3900

38 37 36 35

Before call to gets Input 123456789!”#

80485eb: e8 d5 ff ff ff call 80485c5 <echo>80485f0: c9 leave # Desired return point

0xffffd678

0xffffd688Stack Framefor main

ff ff d6 8808 04 85 f0

0xffffd678

0xffffd688Stack Framefor main

08 04 85

Saved %ebx

Page 42: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Buffer Overflow Stack Example #3Carnegie Mellon

39

Malicious Use of Buffer Overflow

Input string contains byte representation of executable code

Overwrite return address A with address of buffer B

When bar() executes ret, will jump to exploit code

int bar() {

char buf[64];

gets(buf);

...

return ...;

}

void foo(){

bar();

...

}

Stack after call to gets()

B

return address A

foo stack frame

bar stack frame

B

exploit code

pad data written by gets()

Input string contains byte representation of executablecodeOverwrite return address A with address of buffer BWhen bar() executes ret, will jump to exploit code

Page 43: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Exploits Based on Buffer Overflows

Buffer overflow bugs allow remote machines to executearbitrary code on victim machinesInternet worm

Early versions of the finger server (fingerd) used gets() toread the argument sent by the client:Worm attacked fingerd server by sending phony argument:

finger “exploit-code padding new-return-address”exploit code: executed a root shell on the victim machinewith a direct TCP connection to the attacker.

Page 44: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Exploits Based on Buffer Overflows

Buffer overflow bugs allow remote machines to executearbitrary code on victim machinesIM War

AOL exploited existing buffer overflow bug in AIM clientsexploit code: returned 4-byte signature (the bytes at somelocation in the AIM client) to server.When Microsoft changed code to match signature, AOLchanged signature location.

Page 45: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Code Red Exploit Code

Starts 100 threads runningSpread self

Generate random IP addresses & send attack stringBetween 1st & 19th of month

Attack www.whitehouse.govSend 98,304 packets; sleep for 4-1/2 hours; repeat

Denial of service attack

Between 21st & 27th of month

Deface server’s home pageAfter waiting 2 hours

Page 46: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Avoiding Overflow Vulnerability

/* Echo Line */void echo(){

char buf[4]; /* Way too small! */gets(buf);puts(buf);

}

Use library routines that limit string lengths

fgets instead of getsstrncpy instead of strcpyDon’t use scanf with %s conversion specification

Use fgets to read the stringOr use %ns where n is a suitable integer

Page 47: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

System-Level Protections

Randomized stack offsets

At start of program, allocaterandom amount of space on stackMakes it difficult for hacker topredict beginning of inserted code

Nonexecutable code segments

In traditional x86, can mark regionof memory as either “read-only” or“writeable”

Can execute anything readable

X86-64 added explicit “execute”permission

unix> gdb bufdemo(gdb) break echo

(gdb) run(gdb) print /x $ebp$1 = 0xffffc638

(gdb) run(gdb) print /x $ebp$2 = 0xffffbb08

(gdb) run(gdb) print /x $ebp$3 = 0xffffc6a8

Page 48: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Stack Canaries

Idea

Place special value (“canary”) on stack just beyond bufferCheck for corruption before exiting function

GCC Implementation

-fstack-protector-fstack-protector-all

unix>./bufdemo-protectedType a string:12341234

unix>./bufdemo-protectedType a string:12345

*** stack smashing detected ***

Page 49: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Protected Buffer Disassembly

echo804864d: 55 push %ebp804864e: 89 e5 mov %esp,%ebp8048650: 53 push %ebx8048651: 83 ec 14 sub $0x14,%esp8048654: 65 a1 14 00 00 00 mov %gs:0x14,%eax804865a: 89 45 f8 mov %eax,0xfffffff8(%ebp)804865d: 31 c0 xor %eax,%eax804865f: 8d 5d f4 lea 0xfffffff4(%ebp),%ebx8048662: 89 1c 24 mov %ebx,(%esp)8048665: e8 77 ff ff ff call 80485e1 <gets>804866a: 89 1c 24 mov %ebx,(%esp)804866d: e8 ca fd ff ff call 804843c <puts@plt>8048672: 8b 45 f8 mov 0xfffffff8(%ebp),%eax8048675: 65 33 05 14 00 00 00 xor %gs:0x14,%eax804867c: 74 05 je 8048683 <echo+0x36>804867e: e8 a9 fd ff ff call 804842c <FAIL>8048683: 83 c4 14 add $0x14,%esp8048686: 5b pop %ebx8048687: 5d pop %ebp8048688: c3 ret

Page 50: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Setting Up Canary

Setting Up Canary

echo:

. . .

movl %gs:20, %eax # Get canary

movl %eax, -8(%ebp) # Put on stack

xorl %eax, %eax # Erase canary

. . .

/* Echo Line */

void echo()

{

char buf[4]; /* Way too small! */

gets(buf);

puts(buf);

} Return Address

Saved %ebp %ebp

Stack Frame for main

Stack Frame for echo

[3] [2] [1] [0] buf

Before call to gets

Saved %ebx

Canary

/* Echo Line */void echo(){

char buf[4]; /* Way too small! */gets(buf);puts(buf);

}

echo:. . .movl %gs:20, %eax # Get canarymovl %eax, -8(%ebp) # Put on stackxorl %eax, %eax # Erase canary. . .

Page 51: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Checking Canary

Setting Up Canary

echo:

. . .

movl %gs:20, %eax # Get canary

movl %eax, -8(%ebp) # Put on stack

xorl %eax, %eax # Erase canary

. . .

/* Echo Line */

void echo()

{

char buf[4]; /* Way too small! */

gets(buf);

puts(buf);

} Return Address

Saved %ebp %ebp

Stack Frame for main

Stack Frame for echo

[3] [2] [1] [0] buf

Before call to gets

Saved %ebx

Canary

/* Echo Line */void echo(){

char buf[4]; /* Way too small! */gets(buf);puts(buf);

}

echo:. . .movl -8(%ebp), %eax # Retrieve from stackxorl %gs:20, %eax # Compare with Canaryje .L24 # Same: skip aheadcall __stack_chk_fail # ERROR.L24:. . .

Page 52: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Canary ExampleCarnegie Mellon

50

Canary Example

(gdb) break echo

(gdb) run

(gdb) stepi 3

(gdb) print /x *((unsigned *) $ebp - 2)

$1 = 0x3e37d00

Return Address

Saved %ebp %ebp

Stack Frame for main

Stack Frame for echo

[3] [2] [1] [0] buf

Before call to gets

Saved %ebx

03 e3 7d 00

Return Address

Saved %ebp %ebp

Stack Frame for main

Stack Frame for echo

buf

Input 1234

Saved %ebx

03 e3 7d 00

34 33 32 31

Benign corruption! (allows programmers to make silent off-by-one errors)

Page 53: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Outline

1 Background

2 Buffer Overflow

3 Integer and Heap Overflow

4 Summary

Page 54: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

What are the common features of integer overflowvulnerabilities?

Carnegie Mellon

53

What are the common features of integer overflow vulnerabilities?

unsigned int x = read_int();

if ( x > 0x7fffffff )

abort();

unsigned int s = x*sizeof(int);

char* p=malloc(s);

read_int_into_buf(p, x);

an untrusted source

an incomplete

check

an integer overflow

a sensitive

operation a heap overflow

followed

Page 55: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

CVE-2008-5238(Xine)

Carnegie Mellon

54

CVE-2008-5238(Xine)

……

if (version == 4) {

const uint16_t sps = _X_BE_16 (this->header+44) ? : 1;

this->w = _X_BE_16 (this->header+42);

this->h = _X_BE_16 (this->header+40);

this->cfs = _X_BE_32 (this->header+24);

this->frame_len = this->w * this->h;

this->frame_size = this->frame_len * sps;

this->frame_buffer = calloc(this->frame_size, 1);

……

an untrusted source

a sensitive

operation

Page 56: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

CVE-2008-1722(CUPS)

Carnegie Mellon

55

55

png_get_IHDR(pp, info, &width, &height, &bit_depth, &color_type,

&interlace_type, &compression_type, &filter_type);

{

……

if (width == 0 || width > CUPS_IMAGE_MAX_WIDTH ||

height == 0 || height > CUPS_IMAGE_MAX_HEIGHT)

{//error

return (1);

}

img->xsize = width;

img->ysize = height;

……

if (color_type == PNG_COLOR_TYPE_GRAY ||color_type ==

PNG_COLOR_TYPE_GRAY_ALPHA)

in = malloc(img->xsize * img->ysize);

else

in = malloc(img->xsize * img->ysize * 3);

……

}

an untrusted source

an incomplete

check

an integer overflow

a sensitive

operation

Page 57: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

CVE-2008-2430(VLC)Carnegie Mellon

56

……

if( ChunkFind( p_demux, "fmt ", &i_size ) )

{

msg_Err( p_demux, "cannot find 'fmt ' chunk" );

goto error;

}

if( i_size < sizeof( WAVEFORMATEX ) - 2 )

{

msg_Err( p_demux, "invalid 'fmt ' chunk" );

goto error;

}

stream_Read( p_demux->s, NULL, 8 ); /* Cannot fail */

/* load waveformatex */

p_wf_ext = malloc( __EVEN( i_size ) + 2 );

……

an untrusted source

an incomplete

check

an integer overflow

a sensitive

operation

Page 58: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

What’s the essential feature of integer overflowvulnerabilities?

Carnegie Mellon

57

What’s the essential feature of integer overflow vulnerabilities?

unsigned int x = read_int();

if ( x > 0x7fffffff )

abort();

unsigned int s = x*sizeof(int);

char* p=malloc(s);

read_int_into_buf(p, x);

an integer overflow

an untrusted source

an incomplete

check

a sensitive

operation

Page 59: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Outline

1 Background

2 Buffer Overflow

3 Integer and Heap Overflow

4 Summary

Page 60: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

Summary

Software vulnerabilities1 Cannot be avoided (software complexity)2 Memory bugs (buffer overflow, integer overflow) are

dangerous3 Tons of research has been carried out to stop memory

bugs4 Stack, and integer and heap overflow can be stopped.

Page 61: CS 6V81-05: System Security and Malicious Code Analysis ...zhiqiang.lin/spring2012/public/lec21.pdf · CS 6V81-05: System Security and Malicious Code Analysis Buffer Overflow, Integer

Background Buffer Overflow Integer and Heap Overflow Summary

References

http://en.wikipedia.org/wiki/Buffer_overflowSmashing the Stack for Fun and Profit by Aleph OneA Comparison of Buffer Overflow PreventionImplementations and Weaknesses (Blackhat 2004)http://www.cs.cmu.edu/afs/cs/academic/class/15213-f10/www/lectures/08-machine-advanced.pdf...