Shuttle: Intrusion Recovery in Paas

Recovery from Intrusions in PaaS

Dário Nascimento Miguel Correia

INESC-ID Lisboa

Instituto Superior Técnico

University of Lisbon

Portugal

Recovery from Intrusions in PaaS.

Recovery from Intrusions in PaaS.

Compromise:

• Integrity

• Availability

• Confidentiality

Due to:• Software Flaws (e.g. Shellshock)

• Configuration and usage mistakes (malicious or accidental)

• Corrupted legitimate requests (e.g. SQL-Injection)

Prevent is NOT enough.

Intrusions will Happen.

Wake-Up

Shower

Skip breakfast

Wake the rest of team

Connect laptop to VPN

Determine intrusion causes

Verify database consistency

Fix exploited vulnerabilities

Remove intrusion effects

Recover application’s Integrity

Recover application’s Availability

Redeploy the application

Contact angry customers

Explain the reasons to your boss

Listen your angry boss

Blame security team

Hope it never happens again

Return to bed.

Wake-Up

Shower

Skip breakfast







Recover application’s Integrity

Recover application’s Availability

Redeploy the application




Blame security team


Return to bed.

Recover Application’s Integrity

Wake-Up

Shower

Skip breakfast









Blame security team


Return to bed.


Make the Application Available

HOW?

Recovery Procedure.

does Not Exist

is Unknown.

is Not Tested.

Alternatives?



Make the Application Available

H1 : Terminate all instances and restart


H1 : Terminate all instances and restart

Lose all your data?!

Time consuming, downtime



H2: Replication


H2: Replication

Intrusion effects are also replicated


H3: Backup


H3: Backup

You will lose your customers data after intrusion


H4: Remote Site Recovery Mechanism


H4: Remote Site Recovery Mechanism

Remote Site is tampered or outdated

Best day ever to stay in BED.

Intrusion Recovery / Undo Systems.

Intrusion Recovery System

Remove unwanted actions.

Keep effect of legitimate actions.

time

Backup A Backup B

User ActionMalicious

Action

Current Solutions.

Operating Systems: Taser, Retro

Databases: ITDB, Phoenix

Web applications: Goel et. al, Warp, Aire

Others (Email): Undo for Operators

Limitations:

• Max. complexity: 1 app server, 1 database instance

• All require setup and configuration

• Cause application downtime during recovery

Shuttle.

Intrusion Recovery System for Cloud Computing:• Remove the intrusion effects

• Recover application’s integrity

• Support applications deployed in various instances• Available without setup

• Avoid application downtime

• Cost efficient

• Recover timely

Software as a Service (SaaS)

Platform as a Service (PaaS)

Infrastructure as a Service (Iaas)


Infrastructure as a Service (Iaas)

Software as a Service (SaaS)

DevOps?Micro-services?

Containers?Automated Deployment?


Cloud service to run applications

Consumer develops application to run in that environment, using supported

• Languages: e.g., Java, Python, Go, PHP

• Components: e.g., SQL/NoSQL databases, load balancers

User Request

Proxy

Load Balancer

Application Server

Application Server

Database Instance

Database Instance

Integrating intrusion recovery mechanisms.

1. Record all user requests

2. Load a database snapshot

3. Replay all legitimate requests




User Request

Proxy

Load Balancer

Application Server

Application Server

Database Instance

Database Instance

Storage

Request A

Write A=1

Request B

Read A

Request Dependency

User Request

Proxy

Load Balancer

Application Server

Application Server

Database Instance

Database Instance

Manager

Storage

DB Proxy DB Proxy

Interceptor Interceptor




Consistent Snapshot on NoSQL.

Request Consistent.

Copy-on-write.

No downtime.

More Details @ Paper

Replay/Recovery Process.

1. Identify the malicious actions (or intrusion instant)

2. Launch new application and database instances

1. Identify the malicious actions and intrusion instant

Recovery Process

2. Launch new application and database instances1. Identify the malicious actions (or intrusion instant)

3. Load a snapshot previous to intrusion instantCreate a new branch

Recovery Process

Branching

Users Shuttle



4. Replay requestsDatabase operations shall replay in same order as original

Recovery Process

User Request

Proxy

Load Balancer

Application Server

Application Server

Database Instance

Database Instance

Manager

Storage

DB Proxy DB Proxy

Replay Instances

Interceptor Interceptor




5. Block incoming requests; replay last requests

Recovery Process





6. Change branch

Recovery Process





6. Change branch

Recovery Process

Full-Replay

Replay every

operation after snapshot

Selective-Replay

Replay only

affected (tainted) operations

Serial

Replay all requests

sequentially

Clustered

Independent clusters are

replayed concurrently

Evaluation.

Amazon EC2, c3.xlarge instances, Gigabit Ethernet

Ask Q&A application; data from Stack Exchange

WildFly (formely JBoss) application servers

Voldemort database

Performance overheadin normal execution

50% Read 50% Insert 95% Read 5% Insert

ops/sec latency (ms) ops/sec latency (ms)

Shuttle 6325 5.78 15 346 3.62

No Shuttle 7148 5.07 17 821 3.01overhead 13% 14% 16% 20%

Accuracy: Intrusion Scenarios:1. Malicious requests2. Software vulnerabilities3. External channels (e.g. SSH due to Shellshock)

# Intrusion # tainted # Selective Replay

# Full Replay

1a 106 0 < 605 > 38 620

1b 58 14 < 379 > 38 620

1c 48 52 < 253 > 38 620

2a 4 338 0 - > 38 620

2b 18 286 1 278 - > 38 620

3 > 2 000 - - > 38 620

Recovery Time1 million requests

Restrain Duration

Need it faster?

Scalability.

Scalability

Storage Bill.

# objects Size (MB)

Shuttle Storage:

Requests 1 million 212

Response 1 million 8 767

Start/End timestamps 2 million 16

Keys 137 million 488

Total 9 648 MB

Database node:

Version List 14 593 1.4

Operation List 9 million 277

Total 282 MB

Manager

Graph 1 million 718 MB

Storage Overhead1 million requests

$47 per month if 20 Million requests per day.

Conclusion.

New intrusion recovery service be integrated in PaaS

Supports applications running in various instances backed by distributed databases

Leverages resource elasticity and pay-per-use model to reduce the recovery time and costs

Accomplishing intrusion recovery without service downtime using a branching mechanism

Globally transaction-consistent snapshot for NoSQL databases

Remove intrusions by redeploying the applications

Future Work.

Prevent intrusions from spreading

Client side applications

Handle database replication and fault-tolerance

Deliver as a commercial solution

Integrate with Micro-services Architecture

References[Taser] A. Goel, K. Po, K. Farhadi, Z. Li, and E. de Lara, “The taser intrusion recovery system,” in SOSP. ACM, 2005.

[Retro] T. Kim, X. Wang, N. Zeldovich, and M. F. Kaashoek, “Intrusion recovery using selective re- execution.” USENIX, 2010.

[ITDB] P.Liu, J.Jing, P.Luenam and Y.Wang, “The design and implementation of a self healing database system,” JIIS 2004.

[Goel] I. Akkus and A. Goel, “Data recovery for web applications,” in DSN. IEEE, Jun. 2010, pp. 81–90

[Warp] R. Chandra, T. Kim, and M. Shah, “Intrusion recovery for database-backed web applications,” in SOSP. ACM, 2011.

[Aire] R.Chandra, T.Kim and N.Zeldovich, “Asynchronous intrusion recovery for interconnected web services,” in SOSP. ACM, 2013.

[UndoForOperators] A. B. Brown and D. A. Patterson, “Undo for operators : Building an undoable e-mail store,” in USENIX ATC, 2003.

More details

MsC Thesis on Distributed, Cloud and Mobile Applications

Recovery from Security Intrusions in Cloud Computing

Dario Nascimento, Miguel Pupo Correia, Instituto Superior Tecnico, 2015

Thank you for your attention

[email protected]

[email protected]

Shuttle: Intrusion Recovery for PaaS

Technology

Shuttle: Intrusion Recovery in Paas