Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
NBDE:How I could have slept better at night
Chuck MatternPrincipal Solution ArchitectRed Hat
Red Hat Customer 18 years
Linux User and Admin (TAMU, Slackware, Red Hat (& Enterprise), SuSE, Yggdrasil, Mandrake, Debian, CentOS, Scientific, Fedora)
26 years
Unix User and Admin (Coherent, UNIXWare, DG-UX, HP-UX, AT&T B3, Solaris, AIX, OpenBSD, Dynix/PTX, DEC Unix, Ultix, SCO, PrimeOS)
27 years
VMWare ESX (Engineer & Architect) 5 years
Indus International (Unix Admin, Certified Solaris Admin (OS, Networking and Storage) 1998)
1.5 years
Home Depot (Loss Prevention Supervisor, Programmer, Sys Admin, Architect, Principal Engineer, Red Hat Certified Engineer (RHEL 4 (2005), 6 (2011))
27 years
The Paradies Shops (Sr. Manager: Server, Network, Telephony, Desktop ) 1.5 years
Red Hat (Solution Architect, Red Hat Certified Engineer (RHEL 7 (2016)) ~5 years
My old intro
I’m Irish, Italian and Sysadmin-ish● We tend to talk with our hands● We get excited● We are passionate● We like to share stories
Preface: Some core concepts
“Only in self discipline will you ever find freedom”-Hon. James A. Walsh et al
“Nobody is so horrible that he can’t be the perfect bad example.”-John Kelly
“There but for the grace of God go I.”-Hon. James A. Walsh et al
My Cautionary Tale
➢ 2,000+ sites across the US (including Puerto Rico, Hawaii and Guam)➢ 4,000+ ESX hosts➢ 2,000+ iSCSI storage units➢ 2,000+ Windows 2003 VMs➢ 10,000+ RHEL VMs➢ Global deduplicating compressing backup/recovery solution living on
the same storage unit as the other VMs and replicating to a central site
➢ Fractional T1 to each location sharing credit auth and VOIP➢ No local technical staff➢ What could possibly go wrong?➢ ...oh yeah, my support team was 5 Engineers...
image via Peakpxhttp://www.peakpx.com/571722/man-in-black-wet-suit-on-sea
Enter the PRS
Portable Recovery Server➢ Run! Don’t walk…
➢ Grab the best castoff desktop you can find in the basement➢ Snag two 1TB SAS disks a spare NIC and a gig of RAM from Microcenter➢ Base install of RHEL4, mirroring the disks
➢ Encrypt the root volume with luks and use something tough like K&tx#vQ2*HW@9ucB!➢ Remember, it’s a $50-$100M a year business, in a box!
➢ Expose all spare disk via NFS➢ Mount that up to your ESX host via primary NIC➢ Build out a temporary recovery VM via ESX on the NFS share➢ Replicate backup data➢ Munge through and rename, re-IP everything under the covers➢ Slap the remote location IP on the secondary NIC ‘cause DHCP lived on one of the dead VMs (can
you say down hard?)➢ Shutdown and pack it in a box you found in the basement with styro-peanuts you stole^H^H^H
borrowed from the shipping folks➢ Drive like a maniac to Delta Dash then…➢ ...wait...
What is LUKS?
➢ Linux Unified Key Setup ➢ from Clemens Fruhwirth in 2004➢ Originally for Linux, now there are
➢ Android (yeah I know it’s Linux under there)➢ Windows➢ maybe elsewhere?
https://en.wikipedia.org/wiki/Linux_Unified_Key_Setuphttps://gitlab.com/cryptsetup/cryptestup/
Translating: It’ll be OK, I promise
By Dallastechline, Inc. [CC BY-SA 3.0 (https://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons
By Servershop24 [CC BY-SA 3.0 (https://creativecommons.org/licenses/by-sa/3.0)],from Wikimedia Commons
➢ With a DR solution based on a scavenged desktop I had difficulty establishing credibility with my end customer even though I had a well thought out technical solution to the issue at hand.
➢ Talking a non-technical user through decrypting the root volume with a password such as K&tx#vQ2*HW@9ucB! Did not make things any easier
image via Peakpxhttp://www.peakpx.com/571722/man-in-black-wet-suit-on-sea
What is NBDE?
Network Bound Disk Encryption➢ Linux systems can decrypt volumes, even root volumes, over the
network➢ Based on clevis and tang
➢ clevis framework for the client side➢ inserts into dracut➢ has several “pins”➢ https://github.com/latchset/clevis
➢ tang for the server side➢ one of the clevis “pins”➢ https://github.com/latchset/tang
License: CC0 Public Domain Robust Clevis On Vehicle
Where can I use NBDE?
● Laptops (duh…)● Workstations● Servers
○ yep, even portable ones….
Logical View of Clevis and Tang
Architectural View
Server Installation
[root@tang3 ~]# yum install -y tang[omitted]Installed: tang.x86_64 0:6-1.el7
Dependency Installed: http-parser.x86_64 0:2.7.1-5.el7_4 jose.x86_64 0:10-1.el7 libjose.x86_64 0:10-1.el7
Complete![root@tang3 ~]# systemctl enable tangd.socket --nowCreated symlink from /etc/systemd/system/multi-user.target.wants/tangd.socket to /usr/lib/systemd/system/tangd.socket.[root@tang3 ~]# systemctl status tangd.socket
● tangd.socket - Tang Server socket Loaded: loaded (/usr/lib/systemd/system/tangd.socket; enabled; vendor preset: disabled) Active: active (listening) since Tue 2018-10-16 06:01:23 UTC; 11s ago Listen: [::]:80 (Stream) Accepted: 0; Connected: 0
Oct 16 06:01:23 tang3.mobile.roninprinciples.com systemd[1]: Listening on Tan...Oct 16 06:01:23 tang3.mobile.roninprinciples.com systemd[1]: Starting Tang Se...Hint: Some lines were ellipsized, use -l to show in full.[root@tang3 ~]# firewall-cmd --add-service=httpsuccess[root@tang3 ~]# firewall-cmd --add-service=http --permanentsuccess[root@tang3 ~]#
Server Installation and Configuration
Client Installation
[root@clevis ~]# yum install -y clevis-dracut[omitted]Installed: clevis-dracut.x86_64 0:7-8.el7
Dependency Installed: clevis.x86_64 0:7-8.el7 clevis-luks.x86_64 0:7-8.el7 clevis-systemd.x86_64 0:7-8.el7 jose.x86_64 0:10-1.el7 libjose.x86_64 0:10-1.el7 libluksmeta.x86_64 0:8-1.el7 libpcap.x86_64 14:1.5.3-11.el7 luksmeta.x86_64 0:8-1.el7 nmap-ncat.x86_64 2:6.40-16.el7 tpm2-abrmd.x86_64 0:1.1.0-9.el7 tpm2-tools.x86_64 0:3.0.4-1.el7 tpm2-tss.x86_64 0:1.4.0-1.el7 tpm2-tss-devel.x86_64 0:1.4.0-1.el7
Complete![root@clevis ~]#
Client Installation: Software
[root@clevis ~]# cryptsetup luksDump /dev/vda2LUKS header information for /dev/vda2
Version: 1Cipher name: aesCipher mode: xts-plain64Hash spec: sha256Payload offset: 4096MK bits: 512MK digest: 58 e6 af 4c 89 a8 05 f1 f9 fc 8d 11 52 d8 44 60 c0 1c d7 43 MK salt: d8 c2 51 ae cd e7 3b d5 f7 9b 11 24 dd 20 b9 3f 10 49 43 5e 11 79 16 f0 c1 35 6a 62 27 0e b3 96 MK iterations: 13000UUID: 80e99979-147b-45fd-88cd-7e8ec6b195c2
Key Slot 0: ENABLEDIterations: 98308Salt: a6 6a 9f 45 a0 fb 11 f2 a4 e0 a8 02 58 25 a7 b6 0a 54 04 51 c8 5a ce 5f 5a 7f c4 0e 87 e4 fc 68 Key material offset: 8AF stripes: 4000
Key Slot 1: DISABLEDKey Slot 2: DISABLEDKey Slot 3: DISABLEDKey Slot 4: DISABLEDKey Slot 5: DISABLEDKey Slot 6: DISABLEDKey Slot 7: DISABLED[root@clevis ~]#
Client Installation: luks Status
[root@clevis ~]# clevis luks bind -d /dev/vda2 sss '{
"t": 2, "pins":
{"tang": [{"url": "http://tang1.mobile.roninprinciples.com"}, {"url": "http://tang2.mobile.roninprinciples.com"}, {"url": "http://tang3.mobile.roninprinciples.com"}
]}
}'The advertisement contains the following signing keys:
TepHUGV79tG8Cs0L9XPQh2s0f8A
Do you wish to trust these keys? [ynYN] yThe advertisement contains the following signing keys:
_tE0s8Q9oMn7gF4Hqhehl9irSac
Do you wish to trust these keys? [ynYN] yThe advertisement contains the following signing keys:
LdsB17ihj8MhRCaM8OiHEKkw2q8
Do you wish to trust these keys? [ynYN] yEnter existing LUKS password: [root@clevis ~]#
Client Installation: Configure clevis
Note: This example assumes a single block devise supporting an LVM volume group. Configurations with multiple block devices will require additional configuration.
[root@clevis ~]# cryptsetup luksDump /dev/vda2LUKS header information for /dev/vda2
Version: 1Cipher name: aesCipher mode: xts-plain64Hash spec: sha256Payload offset: 4096MK bits: 512MK digest: 58 e6 af 4c 89 a8 05 f1 f9 fc 8d 11 52 d8 44 60 c0 1c d7 43 MK salt: d8 c2 51 ae cd e7 3b d5 f7 9b 11 24 dd 20 b9 3f 10 49 43 5e 11 79 16 f0 c1 35 6a 62 27 0e b3 96 MK iterations: 13000UUID: 80e99979-147b-45fd-88cd-7e8ec6b195c2
Key Slot 0: ENABLEDIterations: 98308Salt: a6 6a 9f 45 a0 fb 11 f2 a4 e0 a8 02 58 25 a7 b6 0a 54 04 51 c8 5a ce 5f 5a 7f c4 0e 87 e4 fc 68 Key material offset: 8AF stripes: 4000
Key Slot 1: ENABLEDIterations: 176884Salt: 12 8b 7e cd d8 79 b3 44 19 fd 4c bd 82 84 5d 1f ec aa 60 72 1a 14 8b 65 b1 e1 95 a2 de 3c cc eb Key material offset: 1016AF stripes: 4000
Key Slot 2: DISABLEDKey Slot 3: DISABLEDKey Slot 4: DISABLEDKey Slot 5: DISABLEDKey Slot 6: DISABLEDKey Slot 7: DISABLED[root@clevis ~]#
Client Installation: luks Status
[[root@clevis ~]# luksmeta show -d /dev/vda20 active empty1 active cb6e8904-81ff-40da-a84a-07ab9ab5715e2 inactive empty3 inactive empty4 inactive empty5 inactive empty6 inactive empty7 inactive empty[root@clevis ~]#
Client Installation: luksmeta status
Delivering: It’ll be OK, I promise
By Dallastechline, Inc. [CC BY-SA 3.0 (https://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons
By Servershop24 [CC BY-SA 3.0 (https://creativecommons.org/licenses/by-sa/3.0)], from Wikimedia Commons
License: CC0 Public Domain Jeff Rowleyhttps://www.flickr.com/photos/jeffrowley/6675136983/in/photostream/
A few of the finer points
➢ No encryption needed in flight➢ Luks key is never transmitted➢ Only the encrypting key is transferred over the wire➢ Encrypted paraphrase is stored in luks header
Encryption Walk Through
[root@clevis ~]# echo 'Good Morning Columbus, Ohio!' | clevis encrypt sss '{ "t": 2, "pins": {"tang": [ {"url": "http://tang1.mobile.roninprinciples.com"}, {"url": "http://tang2.mobile.roninprinciples.com"}, {"url": "http://tang3.mobile.roninprinciples.com"} ] } }' >gmco.jweThe advertisement contains the following signing keys:
TepHUGV79tG8Cs0L9XPQh2s0f8A
Do you wish to trust these keys? [ynYN] yThe advertisement contains the following signing keys:
_tE0s8Q9oMn7gF4Hqhehl9irSac
Do you wish to trust these keys? [ynYN] yThe advertisement contains the following signing keys:
LdsB17ihj8MhRCaM8OiHEKkw2q8
Do you wish to trust these keys? [ynYN] y[root@clevis ~]#
Encrypting a sample passphrase
With two servers down the threshold of 2 out of 3 tang servers cannot be met:
[root@clevis ~]# clevis decrypt <gmco.jweError communicating with the server!Error communicating with the server![root@clevis ~]#
Once at least 2 of the 3 servers are online we can decrypt the passphrase:
[root@clevis ~]# clevis decrypt <gmco.jweGood Morning Columbus, Ohio![root@clevis ~]#
Decrypting a sample passphrase
It’s not just tang for breakfast anymore
Shamir’s Secret Sharing➢ from Adi Shamir➢ Allows for combinations of multiple kinds of pins
➢ tang➢ tpm2➢ http
➢ math too painful for mere mortals➢ think of it as the intersection of RAID and cryptography for now➢ see the Wikipedia link below if you’re a cryptographer,
mathematician or just like pain
https://en.wikipedia.org/wiki/Shamir's_Secret_Sharing
Magical things you can do with SSS
Magical things you can do with SSS
Magical things you can do with SSS
Magical things you can do with SSS
Magical things you can do with SSS
Magical things you can do with SSS
Magical things you can do with SSS
Magical things you can do with SSS
Magical things you can do with SSS
clevis luks bind -d /dev/vda2 sss '{"t": 2,
"pins": {"tang": [
{"url": "http://tang1.mobile.roninprinciples.com"},{"url": "http://tang2.mobile.roninprinciples.com"},{"url": "http://tang3.mobile.roninprinciples.com"}]
}}'
Quick sample incantation (human readable)
Thank you for attending Ohio Linux Fest!
Portions of the content were based on presentation from:➢ Nathaniel McCallum➢ Brian Atkisson➢ Jim WildmanTechnical references:➢ luks: https://en.wikipedia.org/wiki/Linux_Unified_Key_Setup➢ cryptsetup https://gitlab.com/cryptsetup/cryptestup/➢ Samir’s Secret Sharing:
https://en.wikipedia.org/wiki/Shamir's_Secret_Sharing➢ clevis: https://github.com/latchset/clevis➢ tang: https://github.com/latchset/tang
Resources & Credits