Iterative Security: Secrets when you're not ready for Vault

[email protected] @tmclaughbos

Iterative Security:Secrets Management

When You’re Not Ready For Vault


mailto:[email protected]



Who is this guy up here?His headshot is lasers and a cat



I’m Tom!• Community Engineer at CloudZero

• Previously infrastructure engineer with a focus on automation.



Background & BiasesWhere I’m coming from



Background & Biases: I like startups



Background & Biases: I like startups



Background & Biases: Engineering is just a title



Does this look like you?



There is a lot of work to be done each day



For many of us, this ends up as reality…



We have all this technology! So why does these problems still exist?



How we present security



Security ParalysisI don’t know what to do



Security things you in ops might end up responsible for

• Access controls • How much is too much access?

• Password policies • How often should I force password rotation? • Wait, NIST has changed their recommendation? Don’t force

rotation? • Patching • Do I patch immediately on every vendor release or test first?

• and more…





I am not a “security person”



Iterative SecurityStarting and progressively improving your security stance



Why is security hard for us?



OMG SECURITY!!!



Things we get excited about with security




0-Days!!!




Hash collisions




What the CIA/NSA/DoD has



Things we get excited about in security

Logos



Things we don’t get excited about with security



Patching!



Not leaving MongoDB exposed to the internet with weak credentials



Not leaving Elasticsearch exposed to the internet with weak credentials



Learning from our mistakes



Many of us focus on the wrong things

http://www.littlebobbycomic.com/projects/week-115/


http://www.littlebobbycomic.com/projects/week-115/


How we present security



What we should be teaching

•What are you trying to do? •Where do you start? • How do you progress



There’s a lot of info at the extremes



But this is where many of us are



We know not to…

Put passwords, API keys, tokens, etc. in code.



But it still happens…



Where do you go from here?



Developing a threat model!



Be Realistic



USB sticks are a bigger threat than the man in the ceiling



A breach probably won’t end business



What are we trying to protect?• Intellectual property • Customer data (data about who our customers are) • Customer’s data (data from our customers) • etc.



What does our architecture look like?



Decompose the system



Decompose system: Perimeters



Decompose system: Data pipeline



Decompose system: Data pipeline



Identify threats

• Exposed network ports (network) • Unpatched EC2 instances (host) • Weak secrets management(application) • User submitted data (application) • etc.



Document Threats• Weak password management • At two points in our infrastructure we’re not managing

passwords • Both points involve highly valuable assets • Breach would be bad • Reputation loss -> customer loss • Data could be leveraged against our customers



Rate Threats

Risk = Probability * Damage Potential



Rate Threats• D: Damage potential • R: Reproducibility • E: Exploitability • A: Affected users • D: Discoverability

https://en.wikipedia.org/wiki/DREAD_(risk_assessment_model)



Rate Threats• D: Disaster if this is found (high) • R: Easy to reproduce (high) • E: Easy to exploit; requires existing access (medium) • A: Affects all users (high) • D: Not easy to find; users are hops away from issue (medium)



Putting a response into action

Let’s manage those secrets



Constraints

Time Complexity Risk



Constraints: Time



Constraints: complexity



Constraints: Risk of failure

“It’s all code… We monitor it using Nagios.”



Constraints• Time: Few days to a few weeks • The faster we get this done the more likely we will finish.

• Complexity: We’re going to go with what we know. • Less surprises. • Less to learn and get wrong.

• Risk: Taking only as much risk as we’re ready for. • We’re moving fast! • Let’s limit the failure blast radius



Approaches to get you started

Finally we secure some secrets



git-crypt• Encrypts secrets directly in your repository • Find secrets, rotate, and store them

https://github.com/AGWA/git-crypt


https://github.com/AGWA/git-crypt


git-crypt• Pro • You’ve done the exercise of auditing your code base

• Cons • Symmetric encryption • Everyone needs the master password

• TODO • Prevent key proliferation • Move to new secrets management when ready



Configuration Management: Puppet• Hiera-eyaml: Encrypt values in your Hiera hierarchy • Can use public key encryption • Multiple backends

https://github.com/voxpupuli/hiera-eyaml


https://github.com/voxpupuli/hiera-eyaml


Configuration Management: Puppet• Pros • You’ve centralized your secrets in one repo. • Public key encryption support

• Cons • May require manual intervention when rolling Puppetmasters. • May need to cleanup your Puppet code if you haven’t already

moved to Hiera. • TODO: • Figure out master rekey strategy



Configuration Management: Ansible• Ansible Vault: Encrypts entire var files in playbook

http://docs.ansible.com/ansible/playbooks_vault.html


http://docs.ansible.com/ansible/playbooks_vault.html


Configuration Management: Ansible• Pros • You’ve done the exercise of auditing your code base

• Cons • Symmetric encryption • Everyone needs the shared password • key proliferation

• TODO • Preventing the proliferation of the Vault key • rekeying and rolling secrets.



S3 Buckets• Sneaker • Encrypt, store, and retrieve secrets from S3.

https://github.com/codahale/sneaker




S3 Buckets• Pros • Secrets no longer live in repos • reduced secret proliferation

• Secrets encrypted in S3. • Cons • How are you managing S3 buckets?

• TODO • Manage your S3 buckets with CloudFormation, Terraform, etc.





What should we have gotten out of all this?



Less this…



…More this!



YOU CAN DO THIS!



Thank You!http://strayc.at/feedback


http://strayc.at/feedback




Threat Modeling: startup edition

https://twitter.com/CommitStrip/status/876830310780071936


https://twitter.com/CommitStrip/status/876830310780071936


Threat Modeling: startup edition response

https://twitter.com/ErrataRob/status/876963608076439556


https://twitter.com/ErrataRob/status/876963608076439556


We know what not to do. We (think) we know where we want to be.

But we don’t know how to get there.


Internet

Iterative Security: Secrets when you're not ready for Vault