Learning to Live with an Advanced Persistent Threat PPT Only

PowerPoint Presentation

Learning to Live with an Advanced Persistent ThreatJohn DenuneIT Security DirectorUniversity of California, San [email protected]

ACT Infrastructure services

E-mailActive DirectoryNetworkingID ManagementSecurityTelecomData CenterDatabase AdministrationUNIX and Windows Support

Decentralized 100 OUs 800 OU Admins2

ACT Security

9 StaffFirewallIntrusion DetectionVulnerability AssessmentForensicsAnti-virus and FDEPatch ManagementSSL CertsIncident ResponsePolicy and ComplianceVPN

Decentralized 100 OUs 800 OU Admins3

What is an APT?

Its not Opportunistic

Low level criminal activitySpam, phishing, WAREZOff the shelf attacks, higher likelihood of AV detection4

APTTargetedPatientSkilledTechnicalSocial EngineeringVaried AttacksPhysical threatsEspionageCorporateState-SponsoredTheftHacktivism

You have something they want and they will spend a lot of time trying to get itMonths or yearsOff the shelf, but also custom malware including zero-day Extremely low detection ratesTechnical, phishing, phone callsDropping infected USB drives in parking lot or keystroke loggers on lab keyboardsOften tied so some sort of espionage, either corporate to get insider information or state sponsored to get military infoHacktivism to expose information or long term DDOS to make a pointGood old fashioned theft5

APT LifecycleComplete Mission

External Recon

Initial Compromise

Establish Foothold

Escalate Privileges

Expand

Internal Recon

Initial DetectionJune 2012

Got luckyAV alert on ACT server.OU Admins compromised through unrelated staff account on VPNOnly 1 of 4 pieces of malware detectedChanged password, rebuild serversHappened again the following night with another unrelated VPN accountAlso found several other computers in unrelated departments, also OU admins compromisedThird time password changes and re-used

7

Initial DetectionJune 2012

Got luckyAV alert on ACT server.OU Admins compromised through unrelated staff account on VPNOnly 1 of 4 pieces of malware detectedChanged password, rebuild serversHappened again the following night with another unrelated VPN accountAlso found several other computers in unrelated departments, also OU admins compromisedThird time password changes and re-used

11

Lesson #1

Pay attention to anti-virus alerts

Too many sysadmins view a detection as AV doing its job IF they even monitor at allModern malware loves company and almost always brings friends

12

Lesson #2

Dont (completely) rely on your anti-virus product

Low detection rates, especially for custom malware13

Lesson #3

Where possible, track IPs instead of blocking them

Only had IP blocks 14

Initial ReconFebruary 2012Initial CompromiseApril 2012

Going through org charts, reading about projects

15

Gh0st RAT

Lesson #4

Make your local FBI agent your new best friend

Insight into goalsAny others being attacked from same groupAssistance analyzing malwareHelp with managementThis attack is different. Not a patch, rebuild and youre doneThere are those who are hacked and know it, and those who are hacked and dontIRPS and Dali Lama17

Lesson #5

Have a secure communications plan in place

Security staff had PGP keys but most sysadmins did notVoice mail unreliable due to unified messaging

Attackers were definitely reading e-mail18

Lesson #6Log everything, especially authentication,netflow and DNS

AD logs are ugly and chatty HUGEInformation spread out over several lines using different infor (IP, system name, etc) so context is difficult

Netflow to understand where they are going within the network. VPN netflow added

DNS is HUGE but can provide a lot of insight, especially when connected through VPN.

Tremendous amount of data

19

Dynamic DNS Beaconing$ nslookup host.somehackedsite.com** server can't find host.somehackedsite.com: NXDOMAIN

$ nslookup host.somehackedsite.comhost.somehackedsite.com has address 10.2.3.4

Attack timing

All attacks took place Sunday Thursday between the hours of 6pm and 3am Pacific

This was somebodys jobInsight on when we could make system changes when the attackers werent active21

Attack Path

Malware ObservationsYou dont need to rely on a lot of malware when youve already got a long list of credentials

You dont need to crack passwords when you can just pass a hash

NTLM Authentication

User provides username and password. Client computes hash, stores it in memory and throws away the plaintext password.

Client sends username to server.

Server sends a challenge to the client.Client encrypts the challenge with the user hash and sends it back to the server. Server sends the username, challenge and encrypted response to the DC.

DC retrieves user hash, encrypts the challenge and compares to the client encrypted response. If they match, authentication is successful.

LSASS Local Security Authority Subsystem Service24

Interactive Authentication

Client computes LM and NTLM hash and stores them in memory. Plaintext password is reversibly encrypted and stored in memory.Password hash is salted with username and stored in registry.


NTLM Authentication

Client sends username to server.


NTLM Authentication

Server sends a challenge to the client.


NTLM Authentication

Client encrypts the challenge with the user hash and sends it back to the server.


NTLM Authentication

Server sends the username, challenge and encrypted response to the DC.


NTLM Authentication

DC retrieves user hash, encrypts the challenge and compares to the client encrypted response. If they match, authentication is successful.


Administrator Hash

So, lets say the domain administrator RDPs to the client

Domain Admin NTLM hash now stored in client memory.

Pass the Hash

Attacker compromises clientSteals hashes from memoryAccesses both server and domain controller

Pass the Hash

Attacker compromises clientSteals hashes from memoryAccesses both server and domain controller

GAME OVER

MitigationsChange passwords multiple times per dayFast track two factor authenticationCompartmentalized passwordsSeparate user and admin credentialsMinimize lateral trustScan entire domain for scheduled tasksRebuild Domain Controlers

Authentications that used a hash but didnt have a corresponding interactive login34

Lesson #7

Reconsider traditional password best practices

How often do you change your password?A lot of best practice is based on outdated informationKeystroke loggers and phishing have invalidated most of that thinking

How long do you want the attackers to have access to your systems before kick them a=out and force them to reacquire creds?35

Good passwords?*tecno9654postgresA Matt Hale Tribute CD would be cool..Access-Control-Allow-OriginAbundance4me2dayBulletformyvalentine123ElementarymydearwatsonPutin is nothing but commie scum.Video killed the radio star?antcolonyoptimization

Emergency ActionSeptember 2012

Tried to capture e-mail of upper org chart of one of the targeted departmentsWebmail to check cred, POP to download e-mail

Swatting flies So many compromised credentials Reset the playing field37

Lesson #8

Effectively and securely communicating a password change is hard

Met with campus sysadmins to spread the message internallyhelpdeskCampus announcementsProminent notices on official campus web pages

Just before quarter startedFac, staff, priv role accountsAvoid Sept 11.35000 accounts, Many disabled outright as not been used5 day rolling disable

Huge phishing in the following weeksTry a few, and back off38

We are not alone

Not just a windows problemSome backlash on whether AD could be trustedAs we starting protecting more and more AD creds, attackers tried local accounts in an attempt to hide their activity39

ReengagementJuly 2013

ACT

Parting ThoughtsDetection can be subtle and an artHave a good AD TeamLogging visibility is essentialRegular password changes are a MUSTBe prepared to re-image any systemFirewalls to prevent lateral movementSeparation of user and admin credentialsRequire two-factor for OU Admins

FBI has now confirmed other activity from this particular group

42

A New Hope

A New HopeStrengthened LSASS to prevent credential dumpsMany processes no longer store credentials in memoryBetter ways to restrict local account use over the networkRDP use without putting the credentials on the remote computerAddition of a new Protected Users group, whose members' credentials cannot be used in remote PtH attacks

Further ReadingKnow Your Digital Enemy Anatomy of a Gh0st RAThttp://www.mcafee.com/us/resources/white-papers/foundstone/wp-know-your-digital-enemy.pdf

Mitigating Pass-the-Hash (PtH) Attacks and Other Credential Theft Techniqueshttp://www.microsoft.com/en-us/download/details.aspx?id=36036

APT1: Exposing One of China's Cyber Espionage Unitshttp://intelreport.mandiant.com/Mandiant_APT1_Report.pdf

45

If ignorant both of your enemy and yourself, you are certain to be in peril. Sun Tzu, The Art of War

1462.8567

** Student*** Temp**** Volunteer

Last Modified 10/15/2013

UniversityofCalifornia,SanDiegoAdministrativeComputing&Telecommunications(ACT)

Assistant Vice ChancellorMin Yao

PMO, Communications & QALynn Underwood

Benjamin HodsonCharlie Chen**

Telecom PlanningEddie Mardon

Joy GacuyaRafael GonzalesGordon HammanKen Jossenberger

Jim VallettaBuck Wilmerding

Finance, Administration, and HelpdeskSheryl Gerbracht

Helpdesk , Directory Services & Desktop SupportParrish Nnambi

Desktop SupportDan AudishRichie Cruz

Loren Kim ****Mike Schwartz

Directory ServicesFema Grable

Mayra Anaya **Cherie Doria **Carolyn MasonJudy J. Sohn**

Help DeskChristopher Alarcon

Justin ClowDarryl Evangelista

Ron GalacgacGeraldine Powell

Robert RiosIsaac SerranoTom Twomey

Michael Henick**Doris Liu **

Nam Rayabphand **

HR & Administration SupportRashmi Umdekar

Linda BolesJennifer ComfortLinda KnarreborgRenae Moua **

Wendy Salazar ***Eunice Scotto

PurchasingBryan Hurley

FinanceAlison Kibble Koshi

Cathie BarneyVirginia BogtongMonica Bunnay**Cyndi HumphreyNicole Kumabe**

Theresa Pham

StorekeeperJason Traynum

IT Infrastructure / CISP OfficerCharlotte Klock

Telecom OperationsDon McLaughlin

Data CommunicationsJames Seddon

Jason CrisostomoJanet Keith

Nicole LewisDavid RamirezMalerie SamadiRobert Serocki

Lorenzo SugasteRyan A. SullivanPatrick Swiontek

Will ThomasLuis Tirado

Customer SupportVeronica Garcia

Annette ParkerLisa Schwartz

Installation and RepairYnez Hicks

James Leo-CastilloMatthew Chavez

Andrew MackCecil Nolan

Ethan Olvera ***Ronnie Ramierez

Josh SalazarDale Serpas

Randy Totanes

Voice and VideoAlf Bloxham

Damon LittleMicheline Stuyck

Network Architecture/Research LiaisonJim Madden

Valerie PolicharDavid Rapp

System Administrator OutreachJessica Hilt

Data CenterRebecca Sipili

Data Center OperationsStephen Mariani

Corey BuhrerElmer JonesFelice Lococo

Burleigh MannathSteven Pasinski

Jay RogersLedo Sayson

Production ControlDoug Meserve

Perlina BabcockDaron ChockNick Cowell

Estrella ErminoVang Michael Her

Joe Loranca

Mainframe Technical SupportLev Gilik

Bruce HecklerTom Redgrave

Enterprise Infrastructure Data ManagementJerry Singer

Kevin GaoLing He

Leandro HerreraMarian Lambkin-Motter

James PallaRoque Obusan

Chun QianJulieta Uriarte

Hiep VuMay Xu

External Affairs DWLori Barry

Anjelica BakerGreg BoyerYing Chang

Russia MaddenMonica MarLynn Sikora

IT Network SecurityJohn Denune

Network Security OperationsLowell Kinzer

Patrick BurkeFerdie EscuderoCooper NelsonJoe PomianekIsaiah Schisler

Network Security InfrastructureStephen FigueroaEverett Stauffer

Enterprise Infrastructure Systems & ServicesErik Strahm

PostMaster & Email SupportRonise Zenon

Teresa CrespinoAime Inman-Cox

Janice GabrielKaren Przywara

AD/Messaging and Windows SystemsNguyen Trieu

Ted ChiouJames Dotson

Robert FabianoTom Guptill

Claudio LombardoThomas MaddockNick MarangellaCarlos MendozaJaren Thorsen

Network ApplicationsCrys Harris

UNIX SupportJuliet Ensign

Nathalie GholmiehSoo Hom

Alan MoxleyKamlesh Mungekar

Network ApplicationsRyan AustinKevin Bowen

Michael CribbsSteven Misrack

Dan NguyenHector Ordorica Jr.

IT ApplicationsEmily Deere

Academic ApplicationsJohn Baker

Ron BlockDee ChilcoatJendy DennisMed DiramiAnna RuanBeth Surrell

Academic ApplicationsChristina Ivany

Roy HermerSunshine SunBarbara Spear

Academic Personnel Online (APOL)Viet Truong

Lin ChenDavid LittleGail Mikels

Financial ApplicationsKian Colestock

David EdmondsonMarilyn Jewett

Jennifer KramerMatthew Koral **

Mike McGillTim Morse

Nelson PenalosaYvonne Radsmikham **

Marina RusakoffLuis Valdez

James Woodruff

FinancialLinkBill Sweetman

Adrianne AntikollJianJun "JJ" LiDouglas ShiehLong Ton-That

Technical ProjectsSusan Eng

External Affairs/Research ApplicationsKevin Chou

Robert DiasHamed Foroozanfard

Siwei KuangTejas Parikh

Ge Zhang

External Affairs ApplicationsKaniel Meas

Jeff SmithThuan Vo

Campus Web OfficeBrett Pollak

Allisa BeckerAndrew Oh

Tariku TessemaJeremy Wiles

Justin Wright**

Campus Web TechnologiesAlex Wu

Suki ChuiCristian Horta Gonzalez

Middleware & Id Management ServicesMojgan Amini

John GunvaldsonAlan Kim

Ashish PanditRoger PhillipsDavid SolenoLouis Zelus

Documents

Learning to Live with an Advanced Persistent Threat PPT Only