53
©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs. Reality Tel Aviv University, 368-4474-01, Winter 2012-2013 Lecture 7

©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

Embed Size (px)

Citation preview

Page 1: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

©2012 Check Point Software Technologies Ltd.

Introduction to Reverse Engineering

Inbar RazMalware Research Lab Manager

Information Security – Theory vs. RealityTel Aviv University, 368-4474-01, Winter 2012-2013Lecture 7

Page 2: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

22©2012 Check Point Software Technologies Ltd.

What is Reverse Engineering?

Reverse engineering is the process of discovering the technological principles of a device, object, or system through analysis of its structure, function, and operation.

aka: Reversing, RE, SRE

Page 3: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

33©2012 Check Point Software Technologies Ltd.

Why do it?

Discover Trade Secrets

FindVulnerabilities

AcademicResearch

(Yeah, right…)

Circumvent[Copy]

Protection

AnalyseProtocolsPure

CuriosityPatch Binary

andAlter Behavior

Page 4: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

44©2012 Check Point Software Technologies Ltd.

Sounds awesome,

right?

Page 5: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

55©2012 Check Point Software Technologies Ltd.

So where’s the catch?

Low-level is, well, low level…00401000 push ebp00401001 mov ebp, esp00401003 push ecx00401004 push ecx00401005 and dword ptr [ebp-4], 000401009 push esi0040100A mov esi, [ebp+8]0040100D push edi0040100E push esi0040100F call ds:[00402008h]00401015 mov edi, eax00401017 xor edx, edx00401019 test edi, edi0040101B jle 00401047h0040101D movsx ecx, byte ptr [edx+esi]00401021 add [ebp-4], ecx00401024 mov [ebp-8], ecx00401027 rol dword ptr [ebp-4], 10040102A mov eax, ecx0040102C imul eax, [ebp-4]00401030 mov [ebp-4], eax00401033 mov eax, [ebp-8]00401036 add [ebp-4], eax00401039 xor [ebp-4], ecx0040103C inc edx0040103D cmp edx, edi0040103F jl 0040101Dh00401041 cmp dword ptr [ebp-4], 000401045 jnz 00401063h00401047 push 000401049 push 40230Ch0040104E push 4022CCh00401053 push ds:[004023ACh]00401059 call ds:[00402010h]0040105F xor eax, eax00401061 jmp 0040107Fh00401063 xor dword ptr [ebp+0Ch], 01337C0DEh0040106A sub dword ptr [ebp+0Ch], 0BADC0DE5h00401071 mov eax, [ebp-4]00401074 not dword ptr [ebp+0Ch]00401077 xor eax, [ebp+0Ch]0040107A neg eax0040107C sbb eax, eax0040107E inc eax0040107F pop edi00401080 pop esi00401081 leave00401082 retn

for (Serial = 0, i = 0; i < strlen(UserName); i++) { CurChar = (int) UserName[i]; Serial += CurChar; Serial = (((Serial << 1) && 0xFFFFFFFE) || ((Serial >> 31) && 1)); Serial = (((Serial * CurChar) + CurChar) ^ CurChar);}UserSerial = ~((UserSerial ^ 0x1337C0DE) - 0xBADC0DE5);

Page 6: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

66©2012 Check Point Software Technologies Ltd.

So where’s the catch?

Low-level is, well, low level…

Needle in a haystack– Average opcode size:

3 bytes– Average executable size:

500KB (on WinXP)– There are executables,

libraries, drivers….

Page 7: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

77©2012 Check Point Software Technologies Ltd.

So where’s the catch?

Low-level is, well, low level…

Needle in a haystack

Sometimes, the code resists– Packers and compressors– Obfuscators

Page 8: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

88©2012 Check Point Software Technologies Ltd.

So where’s the catch?

Low-level is, well, low level…

Needle in a haystack

Sometimes, the code resists

Sometimes, the code fights back– Detect reversing tools– Detect VMs and emulators

Page 9: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

99©2012 Check Point Software Technologies Ltd.

A Battle of Wits

Author writes code

Reverser reverses it

Author creates an anti-reversing technique

Reverser bypasses it

And so on…

Page 10: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

1010©2012 Check Point Software Technologies Ltd.

So what do you need

in order to be a good reverser?

Page 11: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

1111©2012 Check Point Software Technologies Ltd.

We’ll come backto this…

Page 12: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

1212©2012 Check Point Software Technologies Ltd.

Tools of the Trade

Disassembler (Static code analysis)

Debugger (Dynamic code analysis)

Hex Editor

PE Analyzer

Resource Editor

Page 13: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

1313©2012 Check Point Software Technologies Ltd.

Disassemblers

Page 14: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

1414©2012 Check Point Software Technologies Ltd.

The old world: Sourcer

Page 15: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

1515©2012 Check Point Software Technologies Ltd.

The old world: Sourcer

Page 16: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

1616©2012 Check Point Software Technologies Ltd.

Old ages: Sourcer

Page 17: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

1717©2012 Check Point Software Technologies Ltd.

Old ages: Sourcer

Page 18: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

1818©2012 Check Point Software Technologies Ltd.

Welcome to Windows: W32DASM

Page 19: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

1919©2012 Check Point Software Technologies Ltd.

The Holy Grail: IDA-Pro

Started as an Interactive Dis-Assembler, enabling user interaction with the disassembler’s decisions.

Slowly evolved into an automatic RE tool:– Built-in full-control script language– Library recognition (including user-generated)– Function prototype information

– Display– Propagate throughout the code

– Support for plug-ins– Support for Python scripting– Multi-architecture, cross-platform support– Full incorporation with built-in and external debuggers

Page 20: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

2020©2012 Check Point Software Technologies Ltd.

Debuggers

באג בדיזיין –

זין בדיבאג

Page 21: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

2121©2012 Check Point Software Technologies Ltd.

First, there was DEBUG…

Page 22: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

2222©2012 Check Point Software Technologies Ltd.

GUI and much more: Turbo Debugger

Page 23: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

2323©2012 Check Point Software Technologies Ltd.

GUI and much more: Turbo Debugger

Page 24: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

2424©2012 Check Point Software Technologies Ltd.

GUI and much more: Turbo Debugger

Page 25: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

2525©2012 Check Point Software Technologies Ltd.

Next major step: Soft-ICE

Page 26: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

2626©2012 Check Point Software Technologies Ltd.

And finally: OllyDbg

Page 27: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

2727©2012 Check Point Software Technologies Ltd.

Other Tools

Page 28: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

2828©2012 Check Point Software Technologies Ltd.

Hex-Editor

Page 29: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

2929©2012 Check Point Software Technologies Ltd.

PE Analyzer

Page 30: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

3030©2012 Check Point Software Technologies Ltd.

Resource Editor

Page 31: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

3131©2012 Check Point Software Technologies Ltd.

Methodology

Page 32: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

3232©2012 Check Point Software Technologies Ltd.

How we do it?

Finding the interesting part– System calls (User mode and Kernel mode API)– Strings and constants– Dynamic loading of libraries– Provocation– Zoom-in and Zoom-out

Better and quicker (and how not to get stuck)– Iterative passes over the code– Leave no stone unturned– Macros/Scripts/Plugins– BinDiffing (manual and with tools)

Page 33: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

3333©2012 Check Point Software Technologies Ltd.

Let’s play with them tools…

Page 34: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

3434©2012 Check Point Software Technologies Ltd.

60 seconds on x86 registers

General purpose registers:32bit/16bit/8bit

Index registers:32bit/16bit

Segment registers:16bit

Flags:32bit/16bit

Page 35: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

3535©2012 Check Point Software Technologies Ltd.

Exercise 1:Static Reversing

Page 36: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

3636©2012 Check Point Software Technologies Ltd.

Exercise 1: Static Reversing

Target: a 2004 “Crack-Me”

Tools: IDA-Pro

Page 37: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

3737©2012 Check Point Software Technologies Ltd.

Exercise 2:Dynamic Reversing

Page 38: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

3838©2012 Check Point Software Technologies Ltd.

Exercise 2: Dynamic Reversing

Target: a 2004 “Crack-Me”

Tools: OllyDbg, IDA-Pro

Page 39: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

3939©2012 Check Point Software Technologies Ltd.

Exercise 3:Simple Anti-Debugging

Page 40: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

4040©2012 Check Point Software Technologies Ltd.

Exercise 3: Simple Anti Debugging

Target: a 2006 “Crack-Me”

Tools: OllyDbg

Page 41: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

4141©2012 Check Point Software Technologies Ltd.

Reversing Malware

Malware is comprised of the following building blocks:– Infection Vector– Concealment– Operation– Communications

Check Point’s Anti-Malware Software Blade sits at the gateway

Therefore, communications interest us the most

Page 42: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

4242©2012 Check Point Software Technologies Ltd.

Introducing: Spy Eye

A CrimeWare ToolKit, originating in Russia.

Used mostly for stealing financial information, but will settle for any other identity information and key logging…

Like any serious trojan, Spy Eye compresses its traffic and encrypts it

– Compression is performed using a public library (LZO)– Encryption algorithm is proprietary

Page 43: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

4343©2012 Check Point Software Technologies Ltd.

Act 1:Encryption

Page 44: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

4444©2012 Check Point Software Technologies Ltd.

Act 2:Configuration Download

Page 45: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

4545©2012 Check Point Software Technologies Ltd.

Act 3:Another Encryption

Page 46: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

4646©2012 Check Point Software Technologies Ltd.

So what do you need

in order to be a good reverser?

Page 47: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

4747©2012 Check Point Software Technologies Ltd.

What makes a good reverser?

Qualities

• Patient

• Curious

• Persistent

• Outside-the-Box Thinking

• Optional: Good lookin’

Knowledge

• Assembly Language

• Some High-Level programming• Best: origin of binary

• Operating System Internals• API• Data Structures• File Structures

• Good scripting skills

• Anti-Debugging Tricks

Page 48: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

4848©2012 Check Point Software Technologies Ltd.

Outside-the-Box Thinking

Page 49: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

4949©2012 Check Point Software Technologies Ltd.

And remember, kids:

Binary Reverse Engineer

+ =?

Page 50: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

5050©2012 Check Point Software Technologies Ltd.

Which means…

F A I

L

Page 51: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

5151©2012 Check Point Software Technologies Ltd.

Questions?

Page 52: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

5252©2012 Check Point Software Technologies Ltd.

Check yourOpportunities

We are looking for: Excellent Computer Science/Software Engineering students

for a variety of developing positions Talented software developers Security & Malware researchers

To learn more about our openings please check our website: www.checkpoint.com

Page 53: ©2012 Check Point Software Technologies Ltd. Introduction to Reverse Engineering Inbar Raz Malware Research Lab Manager Information Security – Theory vs

5353©2012 Check Point Software Technologies Ltd.

Thank you!

[email protected]