Seong-je Cho
Spring 2018
Computer Security & Operating Systems Lab, DKU
Introduction to Software Security
Malware(Chapter 11)
- 2 -
Sources / References
Textbook
N. Vlajic, CSE 3482: Introduction to Computer Security, Yorku
Nicholas Weaver, Computer Science 161: Computer Security, Berkeley
Myrto Arapinis, Computer Security: INFRA10067, University of Edinburgh
Please do not duplicate and distribute
Computer Security & OS Lab, DKU
- 3 -
Contents
Malware
Malicious Software = Malicious Code
Types of Malware
Virus
Worm
Malware Detection
Signature/Change/Anomaly based Detection
New Trends of Malware
Do You Trust Software?
Smartphone app, Anti-malware software, DBMS, Compiler, OS/Kernel, Boot loader…
Computer Security & OS Lab, DKU
- 4 -
Malicious Software
Malware
any software program designed to cause harm to a computer system or designed to frustrate the user experience.
Fred Cohen’s initial virus work in 1980’s Used viruses to break the MLS (multi-level security) systems of the time
Types of malware (lots of overlap) Virus passive propagation
Worm active propagation
Trojan horse unexpected functionality
Trapdoor/backdoor unauthorized access
Rabbit exhaust system resources can implemented by virus, warm …
Computer Security & OS Lab, DKU
- 5 -
Malware Timeline
Preliminary work by Cohen (early 80’s)
Brain virus (1986)
Morris worm (1988)
Code Red worm (2001)
SQL Slammer worm (2004)
Future of malware?
Computer Security & OS Lab, DKU
- 6 -
Where do Viruses live?
Boot sector Take control before anything else
Memory resident Stays in memory – Rebooting system can remove the virus out
Applications, macros, data, etc.
Library routines
Compilers, debuggers, virus checker, etc. These are particularly nasty!
Computer Security & OS Lab, DKU
- 7 -
Brain Virus
First appeared in 1986 More annoying than harmful
Brain did nothing malicious Not much reaction by users
A prototype for later viruses
What it did1. Placed itself in boot sector (and other places)2. Screened disk calls to avoid detection3. Each disk read, checked boot sector to see if boot sector
infected; if not, goto 1
Screen: = check (적절한지)확인하다[거르다], 가려내다. I use my answerphone to screen my phone calls.
Computer Security & OS Lab, DKU
- 8 -
Morris Worm – 1/4
First appeared in 1988 What it tried to do
Determine where it could spread Spread its infection Remain undiscovered
Morris claimed it was a test gone bad “Flaw” in worm code it even re-infected already-infected
systems Led to resource exhaustion Adverse effect was like a so-called rabbit
The worm (consists of two parts)
a 99-line bootstrap program written in the C language, plus
a large relocatable object file that comes in VAX and Sun-3 flavors
Source: Seeley, Donn. "A Tour of the Worm.“ http://securitydigest.org/phage/resource/seely.pdf
Computer Security & OS Lab, DKU
- 9 -
Morris Worm – 2/4
How to spread its infection?
Tried to obtain access to machine by
User account password guessing
Exploited buffer overflow in fingerd
Exploited trapdoor in sendmail
Flaws in fingerd and sendmail were well-known at the time, but not widely patched
① Once access had been obtained to machine
② “Bootstrap loader” sent to victim
Consisted of 99 lines of C code
③ Victim machine compiled and executed the code
④ Bootstrap loader then fetched the rest of the worm
⑤ Victim even authenticated the sender!
Computer Security & OS Lab, DKU
- 10 -
Morris Worm – 3/4
SW flaw --- in sendmail - in debug mode –
Normally sendmail runs in the background.
It receives a ‘send’ instruction along with dest address.
However in debug mode the worm can send a command string, in place of dest address.
Then this command string may be executed.
How to remain undetected?
If transmission of the worm was interrupted, all code was deleted
Code was encrypted when downloaded
Downloaded code was deleted after decrypting and compiling
When running, the worm regularly changed its name and process identifier (PID)
Computer Security & OS Lab, DKU
- 11 -
Result of Morris Worm – 4/4
Shocked the Internet community of 1988
Internet designed to withstand nuclear war Yet it was brought down by a graduate student!
At the time, Morris’ father worked at NSA…
Could have been much worse not malicious
CERT (Computer Emergency Response Team) began, The worm increased security awareness
Though limited actions to improve security
Computer Security & OS Lab, DKU
- 12 -
Code Red Worm
Appeared in July 2001
Infected more than 250,000 systems in about 10 ~ 15 hours
In total, infected 750,000 out of 6,000,000 susceptible systems
To gain access to a system, exploited buffer overflow in Microsoft IIS server software
Then monitored traffic on port 80 looking for other susceptible servers
What it did
Day 1 to 19 of month: tried to spread infection
Day 20 to 27: distributed denial of service (DDOS) attack on www.whitehouse.gov
Later versions (several variants)
Included trapdoor for remote access to infected system
Rebooted to flush worm, leaving only trapdoor
Has been claimed that Code Red may have been “beta test for information warfare”
Computer Security & OS Lab, DKU
- 13 -
SQL Slammer Worm
Infected 250,000 systems in 10 minutes!
Code Red took 15 hours to do what Slammer did in 10 minutes
At its peak, Slammer infections doubled every 8.5 seconds
Slammer spread too fast
“Burned out” available bandwidth
Computer Security & OS Lab, DKU
Why was Slammer so successful?
Worm fit in one 376 byte UDP packet
Firewalls often let small packet thru, assuming it could do no harm by itself
Then firewall monitors the connection
Expectation was that much more data would be required for an attack
Slammer defied assumptions of “experts”
Y 축: 트래픽, 초당 1백만패킷
- 14 -
Trojan Horse – 1/2
A Trojan has unexpected function
Prototype of Trojan for the Mac
Example
File icon for freeMusic.mp3:
For a real mp3, double click on icon
iTunes opens
Music in mp3 file plays
But for freeMusic.mp3, unexpected results…
Double click on freeMusic.mp3
iTunes opens (expected)
“Wild Laugh” (probably not expected)
Message box (unexpected)
A wolf in sheep’s clothing
Computer Security & OS Lab, DKU
- 15 -
Trojan Horse – 2/2
How does freeMusic.mp3 trojan work?
This “mp3” is an application, not data!
This Trojan is harmless,
but… Trojan could have done anything user can do delete files, download files, launch apps, etc.
Computer Security & OS Lab, DKU
- 16 -
Definitions of Terms in 2003
[source] Separating Trojan Horses, Viruses, and Worms - A Proposed Taxonomy of Software Weapons, Martin Karresand
Virus:
A self-replicating program.
Some definitions also add the constraint that it has to attach itself to a host program to be able to replicate
Worm:
Also a self-replication program, which does not need another program to be able to replicate but instead is a stand alone program.
The difference between a worm and a virus is often said to be the way they replicate, worms replicate over network connections, while viruses replicate on the host computer.
Trojan horse:
A program performing for the user unknown and unwanted actions, while at the same time posing as a legitimate program.
Computer Security & OS Lab, DKU
- 17 -
Definitions of Terms in 2003
[source] Separating Trojan Horses, Viruses, and Worms - A Proposed Taxonomy of Software Weapons, Martin Karresand
Computer Security & OS Lab, DKU
- 18 -
Other Types of Malware and Anti-malware
Rootkits
Ransomware
Keyloggers
What is anti-malware?
How does anti-malware work?
Why should we know types of malware?
Why should anti-virus program distinguish the unique types of malware from each other?
Why is malware family classification important?
Computer Security & OS Lab, DKU
Malware Detection
Anti-Virus SW (AV SW):
V3, ALYac, ViRobot, Kaspersky, Norton, …
Computer Security & OS Lab, DKU
- 20 -
Malware Detection
Three common methods
Signature detection 흔적탐지 (패턴매칭)
Change detection 변경탐지
Anomaly detection 비정상탐지 (이상탐지)
For signature detection & change detection, cryptographic graphic hash functions can be used
We’ll briefly discuss each of these
And consider advantages and disadvantages of each
Computer Security & OS Lab, DKU
- 21 -
Malware Signature
Brain Virus
The virus marks the six disk sectors as faulty, so that OS may not use them.
Signature: in 5th and 6th bytes of the file, it stores 1234 ( HEX ).
Action : with every disk read, it examines the file for its signature.
If it is not there, it infects the file.
Signature of another malware
Computer Security & OS Lab, DKU
- 22 -
Malware Signature
Example of simple code obfuscation
Computer Security & OS Lab, DKU
- 23 -
Signature Detection
A signature is a string of bits found in software (or could be a hash value)
Suppose that a virus has signature 0x23956a58bd910345
We can search for this signature in all files
If we find the signature, are we sure we’ve found the virus?
No, same signature could appear in other files
But at random, chance is very small: 1/264
Software is not random, so probability is higher
Advantages Effective on “traditional” malware
Minimal burden for users/administrators
Disadvantages Signature file can be large (10,000’s)…
…making scanning slow
Signature files must be kept up to date
Cannot detect unknown viruses
Cannot detect some new types of malware
By far the most popular detection method!
Computer Security & OS Lab, DKU
- 24 -
Change Detection
Viruses must live somewhere on system
If we detect that a file has changed, it may be infected
How to detect changes? Hash files and (securely) store hash values
Re-compute hashes and compare
If hash value changes, file might be infected
Advantages
Virtually no false negatives
Can even detect previously unknown malware
Disadvantages
Many files change and often
Many false alarms (false positives)
Heavy burden on users/administrators
If suspicious change detected, then what?
Might still need signature-based system
Computer Security & OS Lab, DKU
- 25 -
Anomaly Detection – 1/2
Monitor system for anything “unusual” or “virus-like” or potentially malicious
What is unusual?
Files change in some unusual way
System misbehaves in some way
Unusual network activity
Unusual file access, etc.
But must first define “normal”
And normal can change!
Computer Security & OS Lab, DKU
- 26 -
Anomaly Detection – 2/2
Advantages Chance of detecting unknown malware
Disadvantages Unproven in practice
Attacker can make anomaly look normal
Must be combined with another method (such as signature detection)
Also popular in intrusion detection (IDS)
A difficult unsolved (unsolvable?) problem! As difficult as AI?
Computer Security & OS Lab, DKU
- 27 -
Quiz
What are the similarities and differences between virus and worm?
What kinds of executable file formats are there?
MS Windows: ?, Linux: ?, Android: ?, iOS: ?
What is malware analysis?
What is static analysis technique for malware detection?
What is dynamic analysis technique for malware detection?
What are the main differences between static analysis and dynamic analysis?
How do you create a unique signature for malware detection?
Is it OK to run a malicious code on bare machine?
What are the common evasive techniques used by malware to avoid anti-malware detection?
What do you do to be a malware analyst?Computer Security & OS Lab, DKU
- 28 -
Malware Detection using Machine Learning
Machine Learning for Malware Detection, www.Kaspersky.com
"Learning and classification of malware behavior." International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment. Springer, 2008.
"Analysis of machine learning techniques used in behavior-based malware detection." IEEE Second International Conference on Advances in Computing, Control and Telecommunication Technologies (ACT), 2010.
"Automated static code analysis for classifying android applications using machine learning." IEEE International Conference on Computational Intelligence and Security (CIS), 2010.
Automatic analysis of malware behavior using machine learning, Journal of Computer Security, 2011
"A machine learning approach to android malware detection." European Intelligence and security informatics conference (eisic), 2012.
Machine learning for Android malware detection using permission and API calls, IEEE 25th International Conference on Tools with Artificial Intelligence (ICTAI), 2013.
Evading Machine Learning Malware Detection, BlackHat USA, 2017
Computer Security & OS Lab, DKU
- 29 -
Machine Learning for Malware Detection (Kaspersky)
Computer Security & OS Lab, DKU
- 30 -
The Best Antivirus Protection of 2018- https://www.pcmag.com/article2/0,2817,2372364,00.asp
Computer Security & OS Lab, DKU
- 31 -
Detection Performance
Computer Security & OS Lab, DKU
Source: https://www.pcmag.com/article2/0,2817,2481367,00.asphttps://en.wikipedia.org/wiki/F1_score
- 32 -
Open-Source Antivirus / Free Antivirus
ClamAV
OpenAV
Armadito AV
Computer Security & OS Lab, DKU
The Best Free Antivirus Protection of 2018 by PC magazine
VirusTotal Analyze suspicious files and URL to detect types of malware
https://www.virustotal.com/#/home/upload
YARA The pattern matching swiss knife for malware researchers
https://virustotal.github.io/yara/
YaraRules Project: http://yararules.com/
New Trends of Malware(Malware with advanced technology)
You had better learn this part after encryption.
(Polymorphic malware is related to cryptography)
Computer Security & OS Lab, DKU
- 34 -
New Trends of Malware
Polymorphic 다형성 malware and
Metamorphic 변성 malware
The techniques of polymorphism and metamorphism change the form of each instance of software in order to evade “pattern matching” detection during the detection and investigative process
Fast replication/Warhol worms
Warhol worm could infect all vulnerable machines on the entire Internet in 15 minutes or less
The term is based on Andy Warhol's remark that "In the future, everyone will have 15 minutes of fame"
Flash worms, Slow worms, etc.
Future is bright for malware
Good news for the bad guys…
…bad news for the good guys
Future of malware detection? machine learning (?)
Computer Security & OS Lab, DKU
- 35 -
Polymorphic Malware
The first responses of virus writers to avoid signature detection
Polymorphism loosely means: “change the appearance of”
Polymorphic malware is (usually) encrypted Data appending / Data pre-pending can be used too
New key is used each time malware propagates
The purpose of encryption is for masking
The encryption is weak (repeated XOR)
Malware body has no fixed signature
Malware must include code to decrypt itself
Signature detection searches for decrypt code
Detectable by signature-based method
Though more challenging than non-polymorphic…
That is, there are limitations:
The decrypted code is essentially the same in each case, thus memory based signature detection is possible
Block hashing can be effective in identifying memory based remnants
Computer Security & OS Lab, DKU
- 36 -
Memory Block Hashing
Source: Chet Hosmer, Polymorphic & Metamorphic Malware, WetStone Technologies (Black Hat briefings and Training)
Computer Security & OS Lab, DKU
- 37 -
Memory Block Hashing
Computer Security & OS Lab, DKU
Memory code Snapshot
- 38 -
Metamorphic Malware
“automatically recodes itself each time it propagates or is distributed”
Simple techniques include:
Adding varying lengths of NOP instructions
Permuting use registers
Adding useless instructions and loops within the code segments
Advanced techniques include:
Function reordering
Program flow modification
Static data structure modification
Reordering structures
Inserting unused data types
Computer Security & OS Lab, DKU
- 39 -
Metamorphic Malware
Limitations
Identification of Morphing Engine
Code semantics
Behavior
Automated code identification and analysis of memory snapshots or analysis of swap space remnants
Computer Security & OS Lab, DKU
Morphing Engine ComponentsMetamorphic Structure
- 40 -
Cyber vs Biological diseases
One similarity
In nature, too few susceptible individuals and disease will die out
In the Internet, too few susceptible systems and worm might fail to take hold
One difference
In nature, diseases attack more-or-less at random
Cyber attackers select most “desirable” targets
Cyber attacks are more focused and damaging
Computer Security & OS Lab, DKU
- 41 -
Packing, Metamorphic Malware and Others …
Packing and Obfuscation
Packed and Obfuscated Malware
Malware writers often use packing or obfuscation to make their files
more difficult to detect or analyze
Please see Textbook, or
Malicious Code Analysis (악성코드분석) course will handle these topic in more detail
Computer Security & OS Lab, DKU
Miscellaneous Attacks
• Numerous attacks involve software
• Can you ever trust software?
Computer Security & OS Lab, DKU
- 43 -
Logic Bomb / Time Bomb
In 1986 Donald Gene Burleson told employer to stop withholding taxes from his paycheck
His company refused
He planned to sue his company He used company computer to prepare legal docs
Company found out and fired him
Burleson had been working on a malware…
After being fired, his software “time bomb” deleted important company data
Company was reluctant to pursue the case
So Burleson sued company for back pay!
Then company finally sued Burleson
In 1988 Burleson fined $11,800
Took years to prosecute
Cost thousands of dollars to prosecute
Resulted in a slap on the wrist
One of the first computer crime cases
Many cases since follow a similar pattern
Companies often reluctant to prosecute
Computer Security & OS Lab, DKU
- 44 -
Trusting Software
Can you ever trust software? See Reflections on Trusting Trust
Consider the following thought experiment
Suppose C compiler has a virus When compiling login program, virus creates backdoor (account with known
password)
When recompiling the C compiler, virus incorporates itself into new C compiler
Difficult to get rid of this virus!
Suppose you notice something is wrong
So you start over from scratch
First, you recompile the C compiler
Then you recompile the OS Including login program…
You have not gotten rid of the problem!
In the real world Attackers try to hide viruses in virus scanner
Imagine damage that would be done by attack on virus signature updates
Computer Security & OS Lab, DKU
- 45 -
Summary, Q & A
Malware
Malware family classification
Malware Detection
Signature/Anomaly-based Detection
Anti-malware software, Vaccine = Anti virus,
Most Common Evasive Techniques used by Malware
Antivirus evasion technique,
50 Open Source Tools to Replace Popular Security Software https://www.datamation.com/osrc/article.php/3882711/50-Open-Source-Tools-To-Replace-Popular-Security-Software.htm
Computer Security & OS Lab, DKU
• What types of threat are related to this lecture?
‒ Consider the STRIDE model !