Replacing Simple Puppet Modules with Providers

Replacing simple modules with custom Types and ProvidersOr Stop managing templates, and start managing your configs

2

Greg Swift

Linux Admin/Engineer ~ 12 yrs Red Hat Certified Engineer ~ 6 yrsAugeas user ~6 yrsPuppet user ~ 3 yrs

greg.swift@{rackspace.com,nytefyre.net}gplus.to/gregswiftlinkedin.com/gregoryswiftgithub.com/{gregswift,rackergs}

xaeth on Fedora, FreeNode, Twitter, and Ingress

3

Bit of time travel...

• Past–An unpleasant reminder of configs past

• Present–Tools available today that help

• Future–What's next?

4

Stroll down memory lane

5

systl.conf

# Controls the default maximum size of a message queue

kernel.msgmnb = 65536

6

Lets change that value

sed ­i 's/^\(kernel.msgmnb = \)\([0­9]*\)$/## Changing for db configuration. Was:\n## \1\2\n\199999/' sysctl.conf

7

Looks good..

# Controls the default maximum size of a message queue

## Changing for db configuration. Was:

## kernel.msgmnb = 65536

kernel.msgmnb = 99999

8

But the next run?

# Controls the default maximum size of a message queue

## Changing for db configuration. Was:

## ## Changing for db configuration. Was:

## kernel.msgmnb = 65536

kernel.msgmnb = 99999

## Changing for db configuration. Was:

## kernel.msgmnb = 99999

kernel.msgmnb = 99999

9

That was then...

10

Templates... yay?

• Great for 1 type of system... maybe even a couple• Supporting multiple OS releases or distributions?

11

Wouldn't it be nice?

• Safe • Repeatable• Extensible• Multi-language

12

But that is a herculean task...

13

Meet team Hercules

David Lutterkort(Now @ PuppetLabs)

Raphaël Pinson

Dominic Cleal

Francis Giraldeau

14

and Augeas

15

What is it?

• An API provided by a C library• A domain-specific language to describe configuration file formats, presented as lenses

• Canonical tree representations of configuration files• A command line tool to manipulate configuration from the shell and shell scripts

• Language bindings to do the same from your favorite scripting language

16

Lense all the things!

17

Just to name a few....

access activemq_conf activemq_xml aliases anacron approx aptcacherngsecurity aptconf aptpreferences aptsources apt_update_manager authorized_keys automaster

automounter avahi backuppchosts bbhosts bootconf build cachefilesd carbon cgconfig cgrules channels cobblermodules cobblersettings collectd cron crypttab cups cyrus_imapd

darkice debctrl desktop device_map dhclient dhcpd dnsmasq dovecot dpkg dput erlang ethers exports fai_diskconfig fonts fstab fuse gdm group grub gtkbookmarks host_conf

hostname hosts_access hosts htpasswd httpd inetd inifile inittab inputrc interfaces iproute2 iptables jaas jettyrealm jmxaccess jmxpassword json kdump keepalived krb5 ldif ldso

lightdm limits login_defs logrotate logwatch lokkit lvm mcollective mdadm_conf memcached mke2fs modprobe modules modules_conf mongodbserver monit multipath mysql nagioscfg nagiosobjects netmasks networkmanager networks nginx nrpe nsswitch

ntp ntpd odbc openshift_config openshift_http openshift_quickstarts openvpn pam pamconf passwd pbuilder pg_hba php phpvars postfix_access postfix_main postfix_master

postfix_transport postfix_virtual postgresql properties protocols puppet puppet_auth puppetfileserver pythonpaste qpid quote rabbitmq redis reprepro_uploaders resolv rsyncd rsyslog rx samba schroot securetty sep services shells shellvars shellvars_list simplelines

simplevars sip_conf slapd smbusers solaris_system soma spacevars splunk squid ssh sshd sssd stunnel subversion sudoers sysconfig sysctl syslog systemd thttpd up2date util

vfstab vmware_config vsftpd webmin wine xendconfsxp xinetd xml xorg xymon yum

18

Don't see your favorite config?

• Build• IniFile• Rx• Sep• Shellvars• Shellvars_list• Simplelines• Simplevars• Util

19

Our earlier example.. on Augeas

augeas { 'set kernel.msgmnb per db vendor':

  context => '/files/etc/sysctl.conf',

  onlyif  => 'kernel.msgmnb != 99999',

  changes => 'set kernel.msgmnb 99999',

}

20

Making it re-usable

define sysctl ($value) {

  augeas { “set ${title} in sysctl.conf”:

    context => '/files/etc/sysctl.conf',

    onlyif  => “${title} != ${value}”,

    changes => “set ${title} ${value}”,

  }

}

sysctl { 'kernel.msgmnb':

  value   => '99999',

}

21

A more complex example..

define ssh_allowgroup ($ensure) {

  if $ensure == present {

      $match = '=='

      $change = “set AllowGroups/01 ${title}”

  } else {

      $match = '!='

      $change = 'rm AllowGroups/[.=${title}]”

  }

  augeas { “sshd_config/AllowGroups ${title}”:

    context => '/files/etc/sshd_config',

    onlyif  => “match AllowGroups/[.=${title}] size $match 0”,

    changes => $change,

  }

}

$sshd_default_groups = ['engineers', 'admins']

$sshd_allowed_groups = $::env ? {

    /prod/    => $sshd_default_groups,

    default   => concat($sshd_default_groups, ['devs']),

}

ssh_allowgroup { $sshd_allowed_groups:

  ensure => present,

}

22

Well I tried it once, but...

• Lenses are hard to write• Xpathing is hard• Its just hard!

23

Make it easier!

24

Introducing AugeasProviders

• Collection of custom types and providers• Written in native Ruby rather than Puppet's DSL• Utilizes bindings directly for flexibility• Heavily tested

25

Introducing AugeasProviders

• Collection of custom types and providers• Written in native Ruby rather than Puppet's DSL• Utilizes bindings directly for flexibility• Heavily tested

26

And that example on AugeasProviders

sysctl { 'kernel.msgmnb':

  value   => '99999',

  comment => 'recommended by db vendor'

}

27

And the more complex example

  $sshd_default_groups = ['engineers', 'admins']

  $sshd_allowed_groups = $::env ? {

    /prod/    => $sshd_default_groups,

    default   => concat($sshd_default_groups, ['devs']),

  }

  sshd_config { 'AllowGroups':

    value  => $sshd_allowed_groups,

    notify => Service['sshd'],

  }

28

What's it got?

• host• mailalias• sshd_config• shellvars /etc/{defaults,sysconfig}/*• puppet's auth.conf (puppet_auth)• syslog.conf entries (rsyslog and sysklog!)• Grub and Grub2 kernel_parameter• And more!

29

Give it to me!

30

Load it up

puppet module install domcleal/augeasproviders

or

git clone https://github.com/hercules­team/augeasproviders

31

What about the future??

32

AugeasProviders next

33

What's changing?

• Minimized duplication of most common patterns• Solid generic library for reuse-ability• Enables Augeas based providers in your modules

34

Contribute

35

What can you do?

• Use it• Report bugs• Create new providers!

–resolv.conf–systemd unit files–etc

36

Educate me!

37

Augeas training

• Provided by camptocamp • http://camptocamp.com

– Solutions->Infrastructure->Training• Fundamentals

–Using augtool, XPath Augeas language, Augeas type in Puppet

• Advanced– Develop using augeas libraries and advanced tree

manipulation• Extending Augeas

–Writing lenses and providers

38

Info and Help

• http://augeas.net• http://augeasproviders.com• #augeas on FreeNode• augeas@lists.redhat.com

39