Unpacking Malicious Binaries

InfraGardSeptember 2008

- In Windows, all executables use a common format known as a portable executable- PEs are based off the Common Object File Format (COFF) used in UNIX- The PE format is also used for .dll, .obj, and .sys files in Windows

Portable Executables (PEs)

PE Format

- All PE files will follow this same format, regardless of the version of Windows

- Packing an executable essentially encapsulates one PE inside of another PE- The inner PE is our original code (in our case, the malicious code) and the outer PE is the newly packed executable that is produced- There are two categories of packers, generic packers and custom packers

Packing

- Custom packers are unique packing algorithms invented by the malware author for that specific piece (or group) of malware- Can be more difficult to unpack, because no documentation, information, or tutorials about the packer can be found, because it was custom made- Custom packers may not be as robust, or contain all of the extra features that a generic packer might include, and once you figure out how to unpack one custom packed binary, you can usually unpack all of its related binaries (ie different versions of Storm)

Custom vs Generic packers

- Generic packers can be easier to unpack than custom made packers, because information and tutorials can be found online where people have already reverse engineered these packers- Methods of unpacking stay consistent with the packer used, in other words, all malware packed with UPX can be unpacked the same way- However, many contain extra features such as antidebugging (won't run in a debugger) or anti VMware (won't run in a virtual machine)

Custom vs Generic packers

• UPX• ASpack• PECompact• Armadillo• Themida• Yoda• CryptX• MEW

Examples of generic packers:• MoleBox• Morphine• PEspin• TElock• WinUPack• VCrypt• ExECryptor• eXPressor

- Obfuscation of the malicious code. The original executable can't be analyzed or reverse engineered until it has been unpacked- Makes the malware harder to detect for AV products- Makes the executable compressed/smaller in size- Can make the same (or similar) pieces of malware appear to be different

Why bother packing?

- Peid, a program used to detect which packer was used on an executable - Ollydbg, with the Ollydump plugin, for disassembling and debugging the executable- ImpRec, for reconstructing the IAT- Strings, to pull strings from the unpacked binary- Procexplorer, to tell when you've accidentally infected yourself

Tools used for unpacking

- Unpacking allows analysis and reverse engineering of a piece of malware- Allows us to find information that we otherwise might not find, for example what compiler was used, strings that hint at what the malware does, IP addresses, etc.- Allows us to show that 2 pieces of malware are related, when they otherwise don't appear to be

Advantages of unpacking

- In order to unpack a PE, we must use a disassembler and debugger to step through the outer PE, until we reach the original entry point (OEP) of the inner binary (our malware)- From there, we can dump the binary starting at that OEP, which now gives us an unpacked version of that inner PE- If we want to execute the PE, we must repair its import address table (IAT)- A lot of unpacking is Trial and Error

The unpacking process

- Using PEID, this is a good place to start, but this method is fairly unreliable, and doesn't ID custom packers- Looking at the section headers of a PE can also give hints about which packer was used

Identifying the Packer

- Running strings on an executable, some strings can give hints about which packer was used- Ollydbg is pretty reliable on determining if a binary is packed, though not what it was packed with

Identifying the Packer

- Refers to a trick commonly used by unpackers. Can be used to unpack many generic packers- Many generic packers start with the instruction “PUSHAD”, this pushes all information in the processor's registers onto the stack, thus allowing the packer to use the registers without risk of overwriting important information in those registers (if for example, the unpacked binary will later need to reference those values)

The ESP trick

The ESP trick

- ESP refers to the register that holds the stack pointer

- After executing the PUSHAD, the next time the executable references the stack pointer (ESP) is when it is popping everything off the stack- The executable isn't going to pop everything off the stack until it is done using the registers, or in other words, when it is finished unpacking

The ESP trick

- The next JMP after the POPAD will be our OEP

- Execute the PUSHAD at the start of the packed executable- Set a breakpoint on the stack pointer ESP- Let the program run until it hits the breakpoint- Execute the program one line at a time until you hit the next JMP statement, this will be the unpacked executable's OEP

The ESP trick

- Live demo of video.exe

Video.exe screens

Video.exe screens

- Live demo of Colonial Digital Certificates

Digital cert spam screens

Digital cert spam screens

Questions?