Analyzing Kernel Security and Approaches for Improving it

Analyzing Kernel Security and Approaches for Improving It

Milan Rajpara

IT Systems and Network Security

Gujarat Technological University Ahmedabad

C DAC Pune

Agenda

• Kernel Introduction

• Necessity for Kernel Security

• Kernel breach

• Analyzing Kernel Security

• Improving Approaches

• Future Work

October 8, 2013Milan Rajpara 2

What is Kernel ?

• A computer program that manages input/output requests from software and translates them into data processing instructions for the central processing unit and other electronic components of a computer. [Wikipedia]

• The kernel is a fundamental part of a modern computer's operating system.

• OS rests on a outer ring, and application above that.


Fig: Privilege rings for the x86 available in protected mode [Source: Wikipedia]

Necessity for Kernel Security

• Kernel, a vary basic (core) part of the Operating Systems

• Single vulnerability will be exposes large number of systems

• Increasing of Cloud Usage with Virtual Systems

• Smartphones now is in every hand


We talk on ..

• Kernels for General Purpose Operating System

• Some Linux flavor gives Server Optimized Kernel

• Ex. Ubuntu older then 12.04, were gave this option. Since 12.04, linux-image-server is merged into linux-image-generic, there is no difference between Generic and Server kernel. [4]

• Windows do not disclose.

• Kernels which Constructed in C language

• Almost kernels are in C

• Improvement for Monolithic kernels

• All work performed in Virtual environment

• The Xen, and VMware used


How Kernel Affected ?

• By Kernel level rootkits

• Manipulating pointers

• Manipulating data

• Direct Kernel Object Manipulation (DKOM)

• By Boot-kits

• Via hooking techniques

• Direct Hardware or Firmware injection


Effect of this Attacks

• Escalate a process’ privileges by overwriting the process’ credentials

• Hide itself by illicitly removing data structures identifying their presence from loaded drivers

• Eliding task structures for the processes from the kernel’s process accounting list

• Alter the overall behavior of OS without injecting any malicious code into the kernel address space, by just pointer manipulating.


How to analyze the Kernel Security

• Find the most critical objects of the kernel, without prior knowledge of the OS kernel data layout in memory

• Identifying OS Kernel Objects for Run-time Security Analysis

• Sort-out objects which are vulnerable to hijack

• Do Kernel Data Disambiguation

• This will make the system easy to analyze


Most critical objects in Kernel

• Windows and Linux, the core kernel part are mostly written in C

• 40% inter-data structure relations are Pointer based

• 35% of these are Generic Pointers

• Pointers which defines at run time, no initial value or data type is associated

• 28% kernel data structure are well known objects


Generic Pointer Problem

• It is the weak link in kernel security

• Use of void pointers *, assists hackers to point somewhere else

• Use of NULL pointers (to implements linklist), helps hackers to hide / change runtime objects.

• Use of Casting in C

• Enables the hackers to exploit data structure layout in physical memory


To Find Critical Objects

1. Memory Mapping techniques

• Travers address space from global variables via pointer dereferencing until reaching running object.

• according to a predefined kernel data definition for each kernel version.

2. Value Invariant Approaches

• Use the value invariants of certain fields or of a whole data structure as a signature to scan the memory for matching running instances. Ex. DeepScanner, DIMSIM

• Drawbacks of this approaches

- Not very accurate

- Require a predefined definition of the kernel data layout

- Not effective when memory mapping and object reachability information is not available.

- High performance overhead


To Find Critical Objects

3. DIGGER [1]

• Uncover all system runtime objects without any prior knowledge of the OS kernel data layout in memory.

• First it performs offline and constructs type-graph (which is used to enable systematic memory traversal of the object details).

• Then it uses the 4-byte pool memory tagging schema (to uncover kernel runtime objects from the kernel address space.)

• (+)

• Accurate result

• Low performance overhead

• Fast and nearly complete coverage


DIGGER & KDD

• DIGGER uses the KDD (Kernel Data Disambiguator) to precisely models the direct and indirect relations between data structures.

• KDD is a static analysis tool that operates offline on an OS kernel’s source code

• Generates a type-graph for the kernel data with direct and indirect relations between structures, models data structures [2]

• KDD disambiguates pointer-based relations (including generic pointers)

• by performing static points-to analysis on the kernel’s source code.

• Points-to analysis is the problem of determining statically a set of locations to which a given variable may point to at runtime.


KDD Operation


Source: Ref [2]

AST: Abstract Syntax Tree (high-level intermediate representation for the source code )

KDD Operation

• Interprocedural Analysis 1: Takes AST and differentiate it

• Gets: Variables, Procedure definition, Procedure call, etc.. .

• Interprocedural Analysis 2: Do points-to analysis across different files to perform whole-program analysis.

• Context Sensitive Analysis:

• It uses Procedure Dependency Graph (PDG) consists of nodes representing the statements of the data dependency in the program.

• context-sensitive analysis solves two problems: the calling context and the indirect (implicit) relations between nodes.


Soundness and Precision of KDD

• The points-to analysis algorithm is sound if the points-to set for each variable contains all its actual runtime targets, and is imprecise if the inferred set is larger than necessary.

• Check on C programs from the SPEC2000 and SPEC2006 benchmark suites.

• Achieved a high level of precision and 100% of soundness.

• And 96% precision on Windows (WRK*, Vista) and Linux kernel (v3.0.22). [2]

*WRK – Windows Research Kernel, the only available code from windows [6]


DIGGER Approach


Source: Ref [1]

DIGGER Approach

• Static Analysis Component: from KDD

• Signature Extraction Component:

• When the object manager allocates a memory pool block, it associates with a pool tag (pool tag is a unique four-byte tag for each object type.) Uses this tag to uncover the kernel objects running instances, and they are static and cannot be changed during object runtime.

• Dynamic Memory Analysis Component: Extract the object details,

• From Pool Tag, it gets the pool block start memory address and the object’s start address.


Analyzing Kernel through DIGGER Gives …

• Disambiguate the points-to relations between data structures, all without any prior knowledge of the OS kernel data layout.

• Robust and quite small signature size to uncover runtime objects, enhancing performance

• Able to keep track of all critical objects of kernel


Protection of Kernel

• Protect the generic pointers.

• Microsoft added a feature PatchGuard, which blocks kernel mode drivers from altering sensitive parts of the Windows kernel.

• But TDL (rootkit) manages to circumvent this protection as well, by altering a machine's MBR so that it can intercept Windows startup routines. [7]

• One approach is use of “Object Partitioning” to protect kernel data structure. [3]

• Uses Sentry, that creates access control protections for security-critical kernel data.


Sentry Architecture

• Sentry protects critical data and enforces data access restrictions based upon the origin of the access within the code of the kernel and its modules or drivers. [3]

• The data integrity model is straightforward and matches that of the Biba ring policy [9]

• The malicious code that modifies privileges by directly writing to memory is in a loaded module and not in the core kernel code, so Sentry will prevent the write


Kernel Memory Access Control

• Protect data structure from DCOM

• Sentry’s design uses a hypervisor to remain isolated from an untrusted kernel

• To keep the overhead low, Sentry uses memory partitioning to lay out sensitive data on separate memory pages and protects those pages using the hypervisor

• The policy enforcer mediates attempted writes to protected data and uses the policy to determine when writes should be permitted.


Working of Sentry

• Identifying Security-Critical Members

• Activation of mediated access

• Instruction emulation

• Secure execution history extraction


Evaluation of Sentry

• Performance

• Low performance overhead

• more performance van be achieved by memory layout optimization

• False Positive Analysis

• There were no instances when security-critical kernel data protected by Sentry was directly modified by a benign driver.

• Sentry provided a 100% detection rate for DKOM rootkits


Future Work

• Detect all kernel data structures automatically, beyond the kernel version

• The DIGGER can only be used to analyze Windows Kernels.

• The current prototype of Sentry only protects two key structures.

• Other kernel data structures may also require similar protection.

• This may gives versatile performance of Sentry, (if more data structure included)


References

[1] Amani S. Ibrahim, James Hamlyn-Harris, John Grundy, Mohamed Almorsy, "Identifying OS Kernel Objects for Run-Time Security Analysis", DOI: 10.1007/978-3-642-34601-9_6

[2] Amani S. Ibrahim, John Grundy, James Hamlyn-Harris, Mohamed Almorsy, "Operating System Kernel Data Disambiguation to Support Security Analysis", DOI: 10.1007/978-3-642-34601-9_20

[3] Abhinav Srivastava, Jonathon Giffin, "Efficient Protection of Kernel Data Structures via Object Partitioning", DOI: 10.1145/2420950.2421012

[4] RFC: Linux kernel merging. https://lists.ubuntu.com/archives/kernel-team/2011-October/017471.html

[5] Rootkits detail by Symantec http://www.symantec.com/avcenter/reference/windows.rootkit.overview.pdf

[6] Windows Research Kernel https://www.facultyresourcecenter.com/curriculum/pfv.aspx?ID=7366&c1=en-us&c2=0

[7] TDL Rootkit: http://www.theregister.co.uk/2010/11/16/tdl_rootkit_does_64_bit_windows

[8] Windows hooks: http://msdn.microsoft.com/en-us/library/ms644959(v=vs.85).aspx

[9] K. J. Biba. Integrity considerations for secure computer systems. Technical Report MTR-3153, Mitre, Apr. 1977


Questions __________________________

- Milan Rajpara

Thank you

27

Education

Analyzing Kernel Security and Approaches for Improving it