88
Package Management & Chef Joe Damato packagecloud.io

Chef Conf 2015: Package Management & Chef

  • Upload
    ice799

  • View
    446

  • Download
    1

Embed Size (px)

Citation preview

Page 1: Chef Conf 2015: Package Management & Chef

Package Management &

ChefJoe Damato

packagecloud.io

Page 2: Chef Conf 2015: Package Management & Chef

slides available at:

blog.packagecloud.io

Page 3: Chef Conf 2015: Package Management & Chef

hi, I’m joe!• i think these things are cool:

• computer programs

• reproducible builds / infrastructure

• automation

• configuration management

• tahdig** an rice food

Page 4: Chef Conf 2015: Package Management & Chef

packagecloud.io• I work on packagecloud.io

• packagecloud makes it easy to upload, download, store, and delete software packages

• you should use it, it’s cool.

• it’s a perfect companion to Chef Delivery

Page 5: Chef Conf 2015: Package Management & Chef

enterprise.packagecloud.io

Page 6: Chef Conf 2015: Package Management & Chef

“packagecloud:enterprise has solved the problem of distributing public and private package repositories. We’re extremely satisfied with the support and we trust it with all the GitLab Omnibus package downloads of more than 1TB per week."

Sytse Sijbrandij, CEO & Founder

Page 7: Chef Conf 2015: Package Management & Chef

marc falardeau, https://flic.kr/p/8gKeGS

Page 8: Chef Conf 2015: Package Management & Chef

Wade M, https://flic.kr/p/5aghr9

Page 9: Chef Conf 2015: Package Management & Chef

Why?• Central to maintaining, building, and testing

infrastructure.

• Packages are a primitive in Chef.

• Understanding where packages come from, and how to store them properly is a requirement for infrastructure of any size.

• Packages and packaging are much trickier than they seem!

Page 10: Chef Conf 2015: Package Management & Chef

Overview

• what is a package?

• what is a package manager?

• ./configure && make && make install pattern

• open source tools for package repositories

• HOWTO manage repos in your infra with Chef

Page 11: Chef Conf 2015: Package Management & Chef

What is a package?

Beck Gusler, https://flic.kr/p/4A15jm

Page 12: Chef Conf 2015: Package Management & Chef

What is a package?

• A package generally consists of:

• metadata (version, architecture, deps, etc)

• files to be written to the filesystem (/usr/sbin/nginx, etc)

Page 13: Chef Conf 2015: Package Management & Chef

Common package types

Page 14: Chef Conf 2015: Package Management & Chef

Common package types• RPM packages

• Used on CentOS, RHEL, Scientific Linux, Fedora, …

• files typically have the “.rpm” file extension

• can be inspected, installed, and removed with rpm

• are actually a:

• header structure (binary data)

• CPIO archive

Page 15: Chef Conf 2015: Package Management & Chef

man 8 rpm

Page 16: Chef Conf 2015: Package Management & Chef

Common package types

Page 17: Chef Conf 2015: Package Management & Chef

Common package types• Deb packages:

• Used on Ubuntu, Debian, Knoppix, …

• files typically have the “.deb” file extension

• can be inspected, installed, and removed with dpkg

Page 18: Chef Conf 2015: Package Management & Chef

Common package types• Deb packages:

• are actually an AR archive with:

• version file: the debian format version

• data.tar.gz: the actual files to write to the filesystem

• control.tar.gz: the package metadata

• Can be GPG signed, but signatures are never checked!

Page 19: Chef Conf 2015: Package Management & Chef

man 1 dpkg

Page 20: Chef Conf 2015: Package Management & Chef

Common package types

• There are lots more! (ruby gems, npm, java, python, …)

• Some packaging systems also have source packages.

Page 21: Chef Conf 2015: Package Management & Chef

What is a source package?• A source package consists of:

• metadata (version, architecture(s), build deps, etc).

• source files (C source, C++ source, py scripts, etc).

• Allows you to rebuild a binary package easily.

Page 22: Chef Conf 2015: Package Management & Chef

Install packages with chefUse the ‘package’ resource to install packages:

package "zlib1g" do action :install end

Page 23: Chef Conf 2015: Package Management & Chef

Install packages with chefSpecify the version you want by setting ‘version’:

package "zlib1g" do version "1:1.2.8-1" action :install end

Page 24: Chef Conf 2015: Package Management & Chef

Summary• Packages are a collection of files with metadata.

• The metadata usually has info like:

• architecture

• version

• dependency info

• and more.

• Installation is easy if you don’t have dependencies.

Page 25: Chef Conf 2015: Package Management & Chef

Dependencies

Nick Sieger, https://flic.kr/p/qQu1e

Page 26: Chef Conf 2015: Package Management & Chef

Dependencies• Installing 1 package is as easy as:

• dpkg -i filename.deb

• rpm -ivh filename.rpm

• Of course, you should use chef instead :D

• But what if your program needs other programs?

• For example: nginx depends on libssl, zlib, …

Page 27: Chef Conf 2015: Package Management & Chef

r-hol, https://flic.kr/p/6UZb98

Page 28: Chef Conf 2015: Package Management & Chef

So, what’s a package manager?

Page 29: Chef Conf 2015: Package Management & Chef

Package manager

• A package manager is a collection of software that allows you to:

• install, upgrade, remove packages

• query package info from local system or repos

• Some tools include more advanced features like mirroring or more advanced caching features.

Page 30: Chef Conf 2015: Package Management & Chef

Common package managers

http://en.wikipedia.org/wiki/Yellowdog_Updater,_Modified#mediaviewer/File:Yum.png

Page 31: Chef Conf 2015: Package Management & Chef

• yum (Yellowdog Updater, Modified)

• Common on RHEL, CentOS, Fedora, …

• Used for installing, removing, configuring, and querying RPM packages and dependencies.

Common package managers

Page 32: Chef Conf 2015: Package Management & Chef

Common package managers

APT

Page 33: Chef Conf 2015: Package Management & Chef

Common package managers

• APT (Advanced Package Tool)

• Common on Debian, Ubuntu, KNOPPIX, …

• Used for installing, removing, configuring, and querying Debian packages and dependencies.

Page 34: Chef Conf 2015: Package Management & Chef

Install packages with chef• When you install packages with chef, chef will

automatically detect which package manager to use.

• You won’t need to worry about which command to run, or what options to pass; chef will take care of that for you!

Page 35: Chef Conf 2015: Package Management & Chef

Summary• package managers help you install software and

associated dependencies

• easily remove, upgrade, and query packages

• Chef will automatically detect the system’s package manager when you install a package.

Page 36: Chef Conf 2015: Package Management & Chef

Kellie Parker, https://flic.kr/p/mtNMb

Page 37: Chef Conf 2015: Package Management & Chef

A problem

• You run Ubuntu 10.04 LTS

• You want to install redis

• Ubuntu 10.04 comes with redis-server 1.2.0-1

• That’s too old! You need 2.8.19!

• So, now what?

Page 38: Chef Conf 2015: Package Management & Chef

Common (not great) solution

• A common solution to this sort of problem is building redis (or ruby, or …) from source in your chef cookbook

• Like this….

Page 39: Chef Conf 2015: Package Management & Chef

execute ‘compile redis' do cwd ‘/tmp/redis’ command ‘make clean && make’ end !

execute ‘install redis' do cwd ‘/tmp/redis’ command ‘make install’ end

Common (not great) solution

Page 40: Chef Conf 2015: Package Management & Chef

Why?• It’s easy!

• ./configure && make && make install

• It works!

• I’m using chef so it’s reproducible!

Page 41: Chef Conf 2015: Package Management & Chef

But…• What happens if you need to:

• completely remove Redis?

• install a security update?

• install a new version?

• install the same exact Redis on 200 machines?

Page 42: Chef Conf 2015: Package Management & Chef

The not-so great side• Not all Makefiles have uninstall targets, so you

have to remove files manually

• Leaving artifacts on the filesystem can cause really, really hard to debug problems later

• If the build process changes version to version, it can be painful to rollback

Page 43: Chef Conf 2015: Package Management & Chef

The not-so great side• Rebuilding the same source does not necessarily

get you the same byte-for-byte binary

• If the binaries aren’t identical, you can end up with bugs in some of the compiled binaries but not others

• Painful to recreate source builds inside of chef

• Makes writing tests for cookbooks painful

Page 44: Chef Conf 2015: Package Management & Chef

Make a package• Install the same binary on every machine

• When the package is removed, all installed files are removed

• Versioning of build process built in (with most tools)

• Keep your chef cookbooks about config management

• Your build steps are “factored out” into the package

Page 45: Chef Conf 2015: Package Management & Chef

Your new chef recipe

package "redis" do action :install end

Page 46: Chef Conf 2015: Package Management & Chef

Your package• Your build steps get encapsulated in the package

itself

• Makes iterating on the build more straight forward

• Don’t need to run (potentially) a huge number of cookbooks every time you do a build

Page 47: Chef Conf 2015: Package Management & Chef

Duncan Hull, https://flic.kr/p/iVLZt

Page 48: Chef Conf 2015: Package Management & Chef

“How do I make a package?”

Page 49: Chef Conf 2015: Package Management & Chef

OZinOH, https://flic.kr/p/bRHn2v

Page 50: Chef Conf 2015: Package Management & Chef

Use tools!• debbuild

• rpmbuild

• git-buildpackage

• fpm

• omnibus

• mock and pbuilder (more advanced)

Page 51: Chef Conf 2015: Package Management & Chef

Tradeoffs

• Takes time to learn new tools

• Takes time to understand packaging

• No one ever has enough time

Page 52: Chef Conf 2015: Package Management & Chef

BUT…

Page 53: Chef Conf 2015: Package Management & Chef

Tradeoffs

• Once you learn how to make packages you can build reproducible infrastructure much more easily

• You can use your prod environment in dev and test

• You can more easily build tests for your infrastructure with kitchen.ci

Page 54: Chef Conf 2015: Package Management & Chef

Duncan Hull, https://flic.kr/p/iVLZt

Page 55: Chef Conf 2015: Package Management & Chef

“How do I store and organize my packages?”

Page 56: Chef Conf 2015: Package Management & Chef

Package repositories

• Major linux distributions keep repositories of packages for users:

• EPEL

• Ubuntu / Debian official repositories

• You can store a package and its dependencies to make it easy to install them all on your infrastructure

Page 57: Chef Conf 2015: Package Management & Chef

OZinOH, https://flic.kr/p/bRHn2v

Page 58: Chef Conf 2015: Package Management & Chef

Package repositories• createrepo: creates yum repositories

• reprepro: creates apt repositories

• Many other free tools available!

• Read the documentation carefully. Lots of tricky options.

• I’ll show some examples to get you started!

Page 59: Chef Conf 2015: Package Management & Chef

createrepo

http://en.wikipedia.org/wiki/Yellowdog_Updater,_Modified#mediaviewer/File:Yum.png

Page 60: Chef Conf 2015: Package Management & Chef

createrepo• mkdir /var/www/myrepo

• cp /path/to/rpms/*.rpm /var/www/myrepo

• createrepo /var/www/myrepo

• gpg --detach-sign --armor /var/www/my/repo/repomd.xml

Page 61: Chef Conf 2015: Package Management & Chef

GPG is important• Using GPG to sign the generated repository

guarantees that you generated the repository.

• This is important.

• This means that no one else modified, removed, or inserted a package other than you.

• GPG signing the repository is not a very well known security measure, but it is incredibly important!

• This is NOT the same as using rpmsign/rpm --sign.

Page 62: Chef Conf 2015: Package Management & Chef

Secure YUM repos

• Sign repository metadata with GPG

• Sign packages with GPG (use rpmsign)

• Serve repositories over SSL

• Enable all the right options for SSL verification, repository GPG checking, AND package GPG checking.

Page 63: Chef Conf 2015: Package Management & Chef

Wouldn’t it be cool to do all that with Chef instead?

Good news: you can!

Page 64: Chef Conf 2015: Package Management & Chef

createrepo via chef

Chef can create YUM repositories for you!

$ knife cookbook site install yumrepo_server

Page 65: Chef Conf 2015: Package Management & Chef

yumrepo_server 'creates my yum repo' do action :create dir 'relative/yum/repo/path' remote_source "http://upstream.com/path" packages %w(pkg1.rpm pkg2.rpm pkg3.rpm) end

createrepo via chef

Page 66: Chef Conf 2015: Package Management & Chef

You still need to GPG sign the repository yourself :(

execute ‘gpg sign yum metadata' do cwd ‘relative/yum/repo/path/repodata’ command ‘gpg --detach-sign —armor repomd.xml’ end

Page 67: Chef Conf 2015: Package Management & Chef

Once the repository is created, it must be added to the client machines.

Page 68: Chef Conf 2015: Package Management & Chef

Add YUM repos with chef

most people never turn on repo_gpgcheck or sslverify, or set the ssl certificate path, but you should!!

yum_repository ‘my_repo' do description “packagecloud.io is better than this” baseurl “https://myurl.com/repo“ gpgkey ‘http://myurl.com/gpg.pub.key' gpgcheck true repo_gpgcheck true sslverify true sslcacert “/etc/pki/tls/certs/ca-bundle.crt” action :create end

Page 69: Chef Conf 2015: Package Management & Chef

But that’s not all!• You MUST have the ‘pygpgme’ package

installed on the system that will verify the signatures.

• Without pygpgme, yum will not be able to verify signatures!

• Some versions of CentOS / RHEL do not automatically install pygpgme with yum!!

Page 70: Chef Conf 2015: Package Management & Chef

Make sure to install pygpgme

package "pygpgme" do action :install end

Page 71: Chef Conf 2015: Package Management & Chef

reprepro

APT

Page 72: Chef Conf 2015: Package Management & Chef

reprepro

• mkdir /var/www/myrepo

• mkdir /var/www/myrepo/conf

• Create a file named “distributions” in the conf directory

Page 73: Chef Conf 2015: Package Management & Chef

reprepro

Codename: precise Components: main Architectures: i386 amd64 SignWith: 7ABDB001

/var/www/myrepo/conf/distributions:

Page 74: Chef Conf 2015: Package Management & Chef

reprepro• You can add more sections if you need more code

names (lucid, trusty, etc).

• SignWith specifies which GPG key to use for signing repository metadata

• You can get your gpg key ID by looking at the output of gpg —list-keys

• This is not the same as using debsigs/debsign !!!

Page 75: Chef Conf 2015: Package Management & Chef

reprepro

import your Ubuntu 12.04 packages:

reprepro -b /var/www/myrepo/ includedeb precise filename.deb

Page 76: Chef Conf 2015: Package Management & Chef

Wouldn’t it be cool to do all that with Chef instead?

Good news: you can!

Page 77: Chef Conf 2015: Package Management & Chef

reprepro via chef

Chef can create APT repositories for you!

$ knife cookbook site install reprepro

Page 78: Chef Conf 2015: Package Management & Chef

reprepro via chef{ "id": "main", "fqdn": "apt.example.com", "repo_dir": "/srv/apt", "incoming": "/srv/apt_incoming", "description": "APT Repository for our packages.", "codenames": [ "lucid", "hardy", "sid", "squeeze", "lenny" ], "allow": [ "unstable>sid", "stable>squeeze" ], "pgp": { "email": "[email protected]", "fingerprint": "PGP Fingerprint for the key", "public": "-----BEGIN PGP PUBLIC KEY BLOCK-----\n-----END PGP PUBLIC KEY BLOCK-----\n", "private": "-----BEGIN PGP PRIVATE KEY BLOCK-----\n-----END PGP PRIVATE KEY BLOCK-----\n" }, "pulls": { "name": "sid", "from": "sid", "component": "main" }, "architectures": [ "amd64","i386","all","source" ] }

Page 79: Chef Conf 2015: Package Management & Chef

Once the repository is created, it must be added to the client machines.

Page 80: Chef Conf 2015: Package Management & Chef

Add APT repos with chef

apt_repository 'repo' do uri ‘http://repo.com/ubuntu/' arch 'amd64' distribution 'precise' components ['main'] key ‘http://repo.com/ubuntu/archive.key' end

$ knife cookbook site install apt

Page 81: Chef Conf 2015: Package Management & Chef

But that’s not all!• You MUST have the ‘apt-transport-https’ package

installed on the system if your repository is served over HTTPS!

• Without apt-transport-https, you can’t install packages over HTTPS.

• You definitely want this.

Page 82: Chef Conf 2015: Package Management & Chef

Make sure to install apt-transport-https

package “apt-transport-https“ do action :install end

Page 83: Chef Conf 2015: Package Management & Chef

Alosh Bennett, https://flic.kr/p/WJ7rE

Page 84: Chef Conf 2015: Package Management & Chef

Success• You can now use kitchen.ci to test your

infrastructure.

• Determine if the packages you need are actually installed after your cookbooks have run.

• Determine if the repositories you added are actually added after your cookbooks have run.

• Don’t need to wait forever for Ruby, redis, et al to build during a test run.

Page 85: Chef Conf 2015: Package Management & Chef

BEST OF ALL !!!!• You can now run Chef on your development VM

using the same cookbooks you use in production

• The cookbooks are applied and you are running the same exact binaries you run in production

• Won’t catch ALL production bugs, but getting closer to production during development is super useful

Page 86: Chef Conf 2015: Package Management & Chef

Summary• Creating package repositories can be tricky. Make

sure to GPG sign repository metadata.

• 99% of package repositories get this wrong.

• Carefully read the documentation of createrepo and reprepro.

• Make sure to install necessary libraries for verifying signatures and accessing repositories via HTTPS.

• Always serve up your repositories over HTTPS.

Page 87: Chef Conf 2015: Package Management & Chef

Use chef to automate your infrastructure.

Use packagecloud.io to deliver software.

Page 88: Chef Conf 2015: Package Management & Chef

?@packagecloudio

https://packagecloud.io [email protected]