Upload
david-tibbs
View
349
Download
1
Tags:
Embed Size (px)
Citation preview
Security in the face of adversityBecause damnit, security is important
www.brightpearl.com
My brain right now
Who the hell am I?
Dave Tibbs@LowlySysadm1n
l Systems Administrator at Brightpearl Incl Started at Brightpearl UK in October 2010l Back then, only about 20 people in the company
– I was the only Systems Administrator/General IT Dogsbody
l ~7 years experience as Sysadmin working with various flavours of Linux
Security – everyone knows it's important, right?
l Wrong!l In my experience, faced with “more
important” priorities (production issues, delivery deadlines), security is one of the first things to fall by the wayside
l But left unchecked, it has one of the worst damage potentials.
Time for a story...
My thought process...
● First – loads of instances spun up in a very short space of time?
● Very unlikely that someone could manage it that quickly through the UI – even with scripts e.g. Greasemonkey, AWS UI isn't exactly quick
● Got to have been done through the API● What is API auth controlled by? AWS
Keypairs.● ARGGGHHHH● DISABLE ALL THE KEYPAIRS!
Speedy resolution
l Disable all the keypairs on our AWS account. Even if it breaks something in production, with the keypairs (that likely had root privileges), an attacker could do far worse:● Spin up expensive instances for nefarious purposes (they did this bit)● Delete RDS instances● Delete snapshots● Terminate EC2 instances● A whole lot more...l Phone AWS Support and find out what keypair had been used to spin up the
instances so we could keep it disabledl Keep everything else disabled until we could get everything else in IAM (more
about this later)
So how the hell did it happen?
l Genuine badness in our setup:● Our AWS infrastructure and usage of it was likely
set up in a hurry● Our AWS keys in use all over our infrastructure
were attached to the root AWS account (all teh privilegez!)
● Once these were being used and working, no impetus to change them - “Fuck it, it works”
● These keys could well have been used outside the organisation – external services using AWS keys to monitor spend or test Cloudwatch integration, etc
So how did they get that keypair?...
l Unfortunately due to using the same AWS key everywhere (cue more facepalming) it's hard to know. Possibilities:
● Nefarious or even careless ex-employee. Keypairs had never been rotated/regenerated
● The keypair was stored in an environment variable for easy access by everything (as well as other places). Bug found to expose these?
● Keypair used with external service which has been compromised.
Steps for fixage
l Secure AWS account● IAM, IAM, IAM!● The more keypairs the better. Don't share
keypairs between functions – allows you to restrict access per keypair/function
● MFA for user privileged accounts.● Completely remove use of any root account
anywherel Avoid using keypairs outside of AWS
infrastructure (e.g. for external reporting sites)l ROTATE YOUR KEYPAIRS REGULARLY
Leaked keypairs - so you think that's bad?
root
This is always happening to the best of us.
l There are tales of bad security practice everywhere you look.● Adobe hack – hashed but not salted
passwords – 150 million affected● PSN hack – 77 million accounts leaked● Random hackers in Russia appear to be
leaking email/password combos every week, it seems. This means there are STILL companies storing them unencrypted
● #thefappening
Dem blockers though...
l Remember the whole “falling by the wayside” thing? Yeeeaaaahh....● Security is “boring” - often a focus, especially in
startups, of concentrating on the shiny-shiny● People often don't have the time to implement
things “properly”● “Fuck it, it works” - bad security practice is
mostly a temporary test never intended to be permanent, but always ends up that way – root password!
● Change (even to harden security) is seen as “risky”
So what to do?
l FIGHT BACK!● Nobody recognises the value of securing
systems and focusing on security when you're not being hacked
● There's always a higher priority – releases, features, other bugfix
● If you are hacked, it's suddenly TOP PRIORITY and you're strung up for not having done it sooner
l Proactive rather than reactivel Black Ops
● JFDI
Getting people to care about security is a good thing
l If people care, they understand the importance of spending time to implement good security setupsl Get rid of the “But it works now, and X is more
important” mentality. Security isn't only important when it's breached.l More work now to avoid major, major pain later
A final note...
MS14-066 - Vulnerability in SChannel Could Allow Remote Code Execution (2992611)
● Critical vulnerability in Microsoft SChannel – similar to Heartbleed but allows pushing and execution of code
● Patched yesterday● Affects Windows Server 2012, Windows Server 2008 R2 and Windows Server
2003, as well as Windows Vista, 7 and 8● This means every major TLS stack – OpenSSL, GNUTLS, NSS, MS SChannel
and Apple Secure Transport – has had a severe vulnerability this year.