75
1 Managing Managing PostgreSQL with PostgreSQL with Ansible Ansible Gülçin Yıldırım Gülçin Yıldırım PGConf EU, 2015, Vienna

Managing Postgres with Ansible

Embed Size (px)

Citation preview

Page 1: Managing Postgres with Ansible

1

ManagingManagingPostgreSQL withPostgreSQL with

AnsibleAnsibleGülçin YıldırımGülçin Yıldırım

PGConf EU, 2015, Vienna

Page 2: Managing Postgres with Ansible

2

select * from me;select * from me;Postgres DBA @

Studying MSc Comp. & Systems Eng. @

Studied BSc Maths Eng. @

Writes blog on 2ndQuadrant Does some childish

Loves independent films

2ndQuadrant

Tallinn University of TechnologyYildiz Technical

Universityblog

paintings@Skype: gulcin2ndq

Github:

apatheticmagpie

gulcin

Page 3: Managing Postgres with Ansible

3

AgendaAgenda

Page 4: Managing Postgres with Ansible

4

What is Ansible?What is Ansible?“ Simple, agentless and powerful open

source IT automation tool

ProvisioningConfiguration ManagementApplication DeploymentContinuous DeliverySecurity & ComplianceOrchestration

Page 5: Managing Postgres with Ansible

5

Why so popular?Why so popular?

Why now?Why now?The technology: APIs, Cloud ManagementNearly every major service has an API these days.

The need: More and more products are now composed ofdistributed components. More moving parts. They need to bemanaged. The practise: It is now common to automate builds, tests,deployments, integration. Infrastructure is the next thing toautomate.

Page 6: Managing Postgres with Ansible

6

What are the options?What are the options?

Page 7: Managing Postgres with Ansible

7

What are the options?What are the options?

Page 8: Managing Postgres with Ansible

8

Why Ansible?Why Ansible?

Agent-less architecture (no agent is required, everything is done byusing SSH, ssh for communication)No centralised server, no client-side agentsSSH based, no Configuration as data, not code (YAML files)Batteries included (community, playbooks, modules, roles, tasks,documentation)Full configuration management, deployment and orchestrationCustom modules can be written in any programming language.JSON input/output is sufficient to integrate a module into Ansible.

PKI

Page 9: Managing Postgres with Ansible

9

Life before AnsibleLife before AnsibleMultiple ssh, control panels, editing config files manually.

Page 10: Managing Postgres with Ansible

10

Let's install AnsibleLet's install Ansible

Page 11: Managing Postgres with Ansible

11

Let's install AnsibleLet's install Ansible

Debian or Ubuntu:Debian or Ubuntu:

apt-get install python-dev python-setuptoolseasy_install pippip install ansible boto

Mac OS X:Mac OS X:

sudo easy_install pipsudo pip install ansible boto

Page 12: Managing Postgres with Ansible

12

Code of this talkCode of this talkBefore we start discussing Ansible in more detail,please check the repo that I created for this talk:

In this example, we will:

Provision Amazon VPC and EC2 instancesInstall latest PostgreSQL packagesSetup a streaming replication with 1 master and 2 standbysAdd a new standby server

github.com/gulcin/pgconfeu2015

Page 13: Managing Postgres with Ansible

13

Configuration managementConfiguration management

Why do we need a better & modern approach?

Manual tasks are error proneVery difficult to setup test environmentsbefore changing/implementing something bigToo many servers require too many tasks to be repeated on eachof themSince it is a manual process, some servers may go out of sync,they may become outdated or even misconfiguredLeads to heterogenous environment instead of a homogenous &more managable one

Page 14: Managing Postgres with Ansible

14

ProvisioningProvisioningAutomated:Simplifies system administrators job by freeing them from mostof manual interventions. Idempotent:Ensures that the system is in the state we expect it to be. Agentless:Every resource is managed by SSH or APIs (like the Amazon API). Feature-rich:Provides lots of modules from file handling to virtual machineprovisioning.

Page 15: Managing Postgres with Ansible

15

Application deploymentApplication deploymentTime & Bandwidth Efficient:Synchronizes only new or changed files. Full Process Handling:Ansible can remove servers from load-balancers and stopservices before transfers. Then it can restore them all back. Zero-Downtime Deployment:It can deploy on a subset of servers and then move on to the nextset to ensure that the system is always up and running. Same solution can be run on developer machines, productionenvironment, staging environment, testing environment, etc...

Page 16: Managing Postgres with Ansible

16

Continuous deliveryContinuous delivery

“ Continuous delivery means adopting a fast and simple approach forreleases.

Ansible helps us with two main aspects:

1. Enables frequent releases without downtimes2. Requires as little human intervention as possible

It can automate any task from provisioning servers to configuringservices to become production ready.Works very well in-front of a continuous integration system.

Page 17: Managing Postgres with Ansible

17

Security and complianceSecurity and complianceYou need to be sure about security of your:

Systems / Services / SoftwareDataCustomer's data

Once the security policy has been set, Ansible can be used to turnthe system back into the compliance mode, instantly.Ansible is easy-to-use whether it’s about:

Setting up firewall rulesLocking down users and groupsApplying custom security policies

Page 18: Managing Postgres with Ansible

18

OrchestrationOrchestrationAnsible ensures that all tasks are executed in the proper order.This makes orchestration of complex multi-tier deploymentseasier. For example, considering the deployment of a software stack,Ansible makes sure that:

The networking is setup before resources are communicatingPostgreSQL servers are up and running before app deploymentContainers are launched before the services are taken onlineApplication is deployed before the load balancers are activated

We will see this in action with our demo.

Page 19: Managing Postgres with Ansible

19

Building BlocksBuilding BlocksAn Ansible solution is composed of one or more itemslisted on the left side. Typically, our solutions executes tasks for aninventory, utilizing some modules, using or populatingsome variables, processing some file templates, in aplaybook, which can be organized in roles. Let's see each of them in detail.

Page 20: Managing Postgres with Ansible

20

Building Blocks - InventoryBuilding Blocks - InventoryTells Ansible about hosts it should manage

Hostnames, IPs, ports, SSH parametersServer specific variables

2 common ways to provide Ansible an inventory:Static inventory: A flat INI file (e.g., )Dynamic inventory: An application returning aJSON data (e.g.,: for Amazon EC2)

Hosts are grouped. They can belong to multiplegroups Groups can also belong to multiple groups

hosts.ini

ec2.py

Page 21: Managing Postgres with Ansible

21

Building Blocks - InventoryBuilding Blocks - InventoryBelow inventory file in INI format contains 3 hosts under2 groups.There is also a 3rd group which contains other groupsand all of the hosts.# "master" group with 1 host[master]postgresql-master ansible_ssh_host=10.0.0.5 ansible_ssh_user=ubuntu

# "standby" group with 2 hosts[standbys]postgresql-standby-01 ansible_ssh_host=10.0.0.10 ansible_ssh_user=ubuntupostgresql-standby-02 ansible_ssh_host=10.0.0.11 ansible_ssh_user=ubuntu

# the "replication" group contains both "master" and "standbys" groups[replication:children]masterstandbys

Page 22: Managing Postgres with Ansible

22

Building Blocks - ModuleBuilding Blocks - ModuleModules provide Ansible means to control or manageresources on local or remote servers. They perform a variety of functions. For example amodule may be responsible for rebooting a machineor it can simply display a message on the screen. Ansible allows users to write their own modules andalso provides out-of-the-box core or extras modules.

Core: Maintained by the core Ansible team andalways shipped with Ansible.Extras: Maintained by the community. Might beshipped separately in the future.

Page 23: Managing Postgres with Ansible

23

Building Blocks - ModuleBuilding Blocks - ModuleSome of the most commonly used modules are:

File handling: file, stat, copy, template Remote execution: command, shell Service management: service Package management: apt, yum, bsd, ports Source control systems: git, subversion

Page 24: Managing Postgres with Ansible

24

Building Blocks - TaskBuilding Blocks - TaskTasks are responsible for calling a module with a specificset of parameters. Each Ansible task contains:

a descriptive name [optional]a module to be calledmodule parameterspre/post-conditions [optional]processing directives [optional]

They allow us to call Ansible modules and passinformation to consecutive tasks.

Page 25: Managing Postgres with Ansible

25

Building Blocks - TaskBuilding Blocks - TaskBelow task invokes the "file" module by providing 4parameters. It ensures 3 conditions are true:

/var/lib/postgresql exists as a directoryowner of /var/lib/postgresql is "postgres"group of /var/lib/postgresql is "postgres"

If it doesn't exist, Ansible creates the directory andassigns owner & group. If only the owner is different,Ansible makes it "postgres".

- name: Ensure the data folder has right ownership file: path="/var/lib/postgresql" state=directory owner=postgres group=postgres

Page 26: Managing Postgres with Ansible

26

Building Blocks - TaskBuilding Blocks - TaskFollowing example shows relationships between tasks.The first task checks if a device exists and thesecond task mounts the device depending on the resultfrom the first task.

Please note "register" and "when" keywords.

- name: Check if the data volume partition exists stat: path=/dev/sdc1 register: partition

- name: Ensure the PostgreSQL data volume is mounted mount: src=/dev/sdc1 name="/var/lib/postgresql/9.4" fstype=ext4 state=mounted when: partition.stat.exists is defined and partition.stat.exists

Page 27: Managing Postgres with Ansible

27

Building Blocks - VariableBuilding Blocks - VariableVariables in Ansible are very useful for reusinginformation. Sources for variables are:

Inventory: We can assign variables to hosts orgroups (group vars, host vars).YAML files: We can include files containing variables.Task results: Result of a task can be assigned to avariable using the register keyword as shown in theprevious slide.Playbooks: We can define variables in Ansibleplaybooks (more on that later).Command line: (-e means extra variable // -e "uservar=gulcin")

Page 28: Managing Postgres with Ansible

28

Building Blocks - VariableBuilding Blocks - VariableThere is also discovered variables (facts) and they can befound by using module:

All of the output in json: bios_version, architecture,default_ipv4_address, ansible_os_family, etc.

You can use variables in tasks, templates themselves andyou can iterate over using with_type functions.

setupansible -i hosts.ini -m setup hostname

- name: Ensure PostgreSQL users are present postgresql_user: state: present name: "{{ item.name }}" password: "{{ item.password }}" role_attr_flags: "{{ item.roles }}" with_items: postgresql_users

Page 29: Managing Postgres with Ansible

29

Building Blocks - TemplateBuilding Blocks - TemplateWe can think templates as our configuration files. Ansiblelets us use the for reforming andparameterising our files. The Jinja2 templating engine offers a wide range ofcontrol structures, functions and ..Some of useful capabilities of Jinja2:

for-loopsjoin(), default(), range(), format()union(), intersect(), difference()| to_json, | to_nice_yaml, | from_json, | from_yml| min, | max, | unique, | version_compare, | random

Jinja2 template engine

filters

Page 30: Managing Postgres with Ansible

30

Building Blocks - TemplateBuilding Blocks - Template

*:*:*:{{ postgresql_replication_user.name }}:{{ postgresql_replication_user.password }}

Let's check our template:pgpass.j2

Let's have a look at file, too:pg_hba.conf.j2

# Access to user from local without password, from subnet with password{% for user in postgresql_users %}local all {{ user.name }} trusthost all {{ user.name }} {{ ec2_subnet }} md5{% endfor %}

# Replication user for standby servers access{% for standby_ip in postgresql_standby_ips %}host replication {{ postgresql_replication_user.name }} {{ standby_ip }}/32 md5{% endfor %}

Page 31: Managing Postgres with Ansible

31

Building Blocks - PlaybookBuilding Blocks - Playbook

Playbooks contains PlaysPlays contain Tasks

Tasks call Modules and may (optionally) triggerhandlers (run once, run at the end)

Page 32: Managing Postgres with Ansible

32

Building Blocks - PlaybookBuilding Blocks - PlaybookAnsible playbooks are written using the YAML syntax.Playbooks may contain more than one playsEach play contains:

name of host groups to connect totasks it needs to perform.

A play may also contain variables/roles/handlers, ifdefined.

Strict dependency ordering: everything in fileperforms in a sequential order.

Page 33: Managing Postgres with Ansible

33

Building Blocks - PlaybookBuilding Blocks - PlaybookLet's look at our playbook example ( ):main.yml---- name: Ensure all virtual machines are ready hosts: 127.0.0.1 connection: local vars_files: # load default variables from YAML files below - 'defaults/postgresql.yml' - 'defaults/aws.yml' tasks: - include: 'tasks/provision.yml' # load infrastructure setup tasks

- name: Ensure all required PostgreSQL dependencies ready hosts: postgresql-all # manage all PostgreSQL servers sudo: yes sudo_user: root vars_files: - 'defaults/postgresql.yml' - 'defaults/aws.yml' tasks: - include: 'tasks/postgresql.yml' # load PostgreSQL setup tasks

...

Page 34: Managing Postgres with Ansible

34

Building Blocks - RoleBuilding Blocks - Role

“ You absolutely should be using roles. Roles are great.Use roles. Roles! Did we say that enough? Roles are great. In Ansible,

playbooks organize tasksroles organize playbooks

Imagine that we have lots of independent resources tomanage (e.g., web servers, PostgreSQL servers, logging,monitoring, AWS, ...). Putting everything in a single playbook may result in anunmaintainable solution.

Page 35: Managing Postgres with Ansible

35

Building Blocks - RoleBuilding Blocks - RoleTo reduce such complexity, roles help us with:

Splitting tasks into much smaller playbooksThis allows us to focus on resources, independently.That makes it simpler to maintain and debug. Also itwill be much easier to understand the structure. Reusing configs, files, templates, tasksThis way we can easily share those componentsbetween playbooks, without rewriting over and over. Handling playbook dependenciesWhen we execute a role, we can be sure that all ofpreconditions are satisfied for that role.

Page 36: Managing Postgres with Ansible

36

Building Blocks - RoleBuilding Blocks - RoleHere you can see a dependency graph and thecorresponding role directory structure:

Page 37: Managing Postgres with Ansible

37

How to Invoke Ansible?How to Invoke Ansible?

To work with Ansible, we have 2 main alternatives;

1. Running ad-hoc commands2. Running playbooks

Let's check them out one by one.

Page 38: Managing Postgres with Ansible

38

Ad-hoc CommandsAd-hoc CommandsWe can call any Ansible module from the command line, anytime.The ansible CLI tool works like a single task. It requires an inventory,a module name, and module parameters. For example, given an inventory file like:

Now we can call any module.

[dbservers] db.example.com

Page 39: Managing Postgres with Ansible

39

Ad-hoc CommandsAd-hoc CommandsWe can check uptimes of all hosts in dbservers using:

Here we can see the Ansible output:

ansible dbservers -i hosts.ini -m command -a "uptime"

gulcin@apathetic ~ # ansible dbservers -i hosts.ini -m command -a "uptime"db.example.com | success | rc=0 >>21:16:24 up 93 days, 9:17, 4 users, load average: 0.08, 0.03, 0.05

Page 40: Managing Postgres with Ansible

40

How to Run Playbooks?How to Run Playbooks?For more complex scenarios, we can create playbooks or roles andask Ansible to run them. When we run a playbook or a role, Ansible first gathers a lot ofuseful facts about remote systems it manages. These facts can laterbe used in playbooks, templates, config files, etc. We can use the ansible-playbook CLI tool to run playbooks.

Page 41: Managing Postgres with Ansible

41

How to Run Playbooks?How to Run Playbooks?Given an inventory file like this:

We may have a playbook that connects to hosts in dbservers group,executes the uptime command, and then displays that command'soutput. Now let's create a simple playbook to see how it can be ran.

[dbservers] db.example.com

Page 42: Managing Postgres with Ansible

42

How to Run Playbooks?How to Run Playbooks?Here is the main.yml file for the playbook we just described:

---

- hosts: dbservers

tasks: - name: retrieve the uptime command: uptime register: command_result # Store this command's result in this variable

- name: Display the uptime debug: msg="{{ command_result.stdout }}" # Display command output here

Page 43: Managing Postgres with Ansible

43

How to Run Playbooks?How to Run Playbooks?Now we can run the playbook and see it's output here:

gulcin@apathetic ~ $ ansible-playbook -i hosts.ini main.yml

PLAY [dbservers] **************************************************************

GATHERING FACTS ***************************************************************ok: [db.example.com]

TASK: [retrieve the uptime] ***************************************************changed: [db.example.com]

TASK: [Display the uptime] ****************************************************ok: [db.example.com] => { "msg": " 15:54:47 up 3 days, 14:32, 2 users, load average: 0.00, 0.01, 0.05"}

PLAY RECAP ********************************************************************db.example.com : ok=3 changed=1 unreachable=0 failed=0

Page 44: Managing Postgres with Ansible

44

Playbook controlPlaybook control--tags / --skip-tagsRuns or skips tasks with specified tags --limitManages only specified hosts or groups --start-at-taskStart execution from a specific task --stepExecutes step-by-step and asks for confirmation to continue --check / --diff / --syntax-checkRuns a playbook without actually executing anything

Page 45: Managing Postgres with Ansible

45

Playbook Playbook loopsloopsN.B! register assigns result of a task to a variable. Ansible supports iterating over facts with loop statements:

with_items: runs the task using the provided array variablewith_indexed_items: runs the task using the provided arrayvariable and adds item indexwith_flattened: runs the task using merged variableswith_file: runs the task using given file's contents

Some other with_type functions: with_together, with_nested, with_subelements, with_sequence,with_random_choice, with_first_found, with_lines..

Page 46: Managing Postgres with Ansible

46

with_itemswith_items

- vars: postgresql_users: - { name: gulcin, password: apathetic, roles: "SUPERUSER,LOGIN" } - { name: foo, password: foobarbaz, roles: "CREATEDB,LOGIN" }

- name: Ensure PostgreSQL users are present postgresql_user: state: present name: "{{ item.name }}" password: "{{ item.password }}" role_attr_flags: "{{ item.roles }}" with_items: postgresql_users

Executes the task for each element in given arrayCurrent element value can be accessed using {{ item }}

Page 47: Managing Postgres with Ansible

47

with_indexed_itemswith_indexed_items

- name: Ensure standby hosts have updated IP addresses add_host: name: "postgresql-standby-{{ item.0 + 1 }}" # e.g., postgresq-standby-1 groups: "postgresql-standbys" ansible_ssh_host: "{{ item.1.tagged_instances[0].public_ip }}" ansible_ssh_user: "{{ ec2_ssh_user }}" with_indexed_items: ec2_standbys.results

Executes the task for each element in given arrayCurrent element index can be accessed using {{ item.0 }}Current element value can be accessed using {{ item.1 }}

Page 48: Managing Postgres with Ansible

48

with_flattenedwith_flattened

- name: Wait for SSH to become ready wait_for: host="{{ item.tagged_instances[0].public_ip }}" port=22 timeout=320 state=started with_flattened: - [ ec2_master ] # ec2_master is a single item. # By putting [ and ], we convert it into an array - ec2_standbys.results

Merges given arrays into a single oneExecutes the task for each element in merged arrayCurrent element value can be accessed using {{ item }}

Page 49: Managing Postgres with Ansible

49

with_filewith_file

Executes the task for the given fileContents of the file can be accessed using {{ item }}

- name: Ensure the SSH key is present ec2_key: state: present region: "eu-central-1" # Amazon's Frankfurt Datacenter name: "postgresql-key" # we will use this name later key_material: "{{ item }}" with_file: "id_rsa.pub" # reads contents of this file to use in "key_material"

Page 50: Managing Postgres with Ansible

50

Advanced playbook featuresAdvanced playbook featuresFor some advanced operations, we can use following in our tasks:

failed_when (condition) / changed_when (condition):Lets tasks to know when they're going to fail or change based onthe given condition. It might be stdout or stderr, or text to log. until (condition):Repeats the task until the given condition is satisfied. ignore_errors:Doesn't fail on errors run_always:Runs even in dry-run / check / diff modes

Page 51: Managing Postgres with Ansible

51

Advanced playbook featuresAdvanced playbook featuresserial:Runs the task on a single host at a single time pre_tasks / post_tasks:Defines names of tasks that are executed before or after thecurrent task. These referenced tasks are not either main tasksor roles, but they setup the environment to be ready for that. delegate_to (host):This tells Ansible to execute the current task on the given hostinstead of the current host that is defined by the inventory.

For example, if we want to start a virtual machine, we can't dothat on that machine, because it is not running. We can simplydelegate this task to localhost (127.0.0.1).

Page 52: Managing Postgres with Ansible

52

Postgres modulesPostgres modulesMy blog post covers Ansible PostgreSQL modules: You can find the related Postgres modules' examples on my github .

Ansible Loves PostgreSQL

repo

: Creates/removes a given db.

: Adds/removes extensions from a db.

: Adds/removes users and roles from a db.

: Grants/revokes privileges on db objects (table,sequence, function, db, schema, language, tablespace, group).

: Adds, removes or changes procedural languageswith a db.

postgresql_db

postgresql_ext

postgresql_user

postgresql_privs

postgresql_lang

Page 53: Managing Postgres with Ansible

53

postgresql_dbpostgresql_dbCreates/removes a given db. In Ansible terminology, it ensures thata given db is present or absent.

Required param: nameConnection params: login_host, port,login_user, login_passwordImportant param: state (present or absent)

Create db module_test Remove db module_test

Page 54: Managing Postgres with Ansible

54

postgresql_extpostgresql_extAdds/removes extensions from a db. PostgreSQL offers a wide rangeof extensions which are extremely helpful.

Mandatory params: db and name (for extension)Other params: state and connection params (login_host, port) asin postgres_db

Page 55: Managing Postgres with Ansible

55

postgresql_userpostgresql_userAdds/removes users and roles from a db. Parameters: name (mandatory), state, connection params, db,password, priv, role_attr_flags ([NO]SUPERUSER, [NO]CREATEDB..)

Creates userAlters role and giveslogin and createdb

Page 56: Managing Postgres with Ansible

56

postgresql_privspostgresql_privsGrants/revokes privileges on db objects (table, sequence, function, db,schema, language, tablespace, group).

Required params: database, rolesImportant opt. params: type (i.e. table,sequence,function), objs,privs (i.e. ALL; SELECT, UPDATE, INSERT)

Page 57: Managing Postgres with Ansible

57

postgresql_langpostgresql_lang Adds, removes or changes procedural languages with a db.

“ One of the very powerful features of PostgreSQL is its support forvirtually any language to be used as a procedural language.

Params: lang (mandatory), db, connection params, state,cascade, trust

Page 58: Managing Postgres with Ansible

58

AWS modulesAWS modulesAnsible provides more than 50 modules for provisioningand managing Amazon Web Services resources.

We have modules for:

EC2 (Virtual machine instances)AMI (Virtual machine images)ELB (Load balancers)VPC (Networking infrastructure)EBS (Storage infrastructure)...

http://docs.ansible.com/ansible/list_of_cloud_modules.html

Page 59: Managing Postgres with Ansible

59

AWS modulesAWS modulesAnsible AWS modules are developed using .Boto is a Python package that provides interfaces to Amazon WebServices. For more complex workflows, we can use the script todiscover dynamic inventories for our playbooks:

Download ec2.py and ec2.inifrom cp ec2.py /etc/ansible/hostscp ec2.ini /etc/ansiblechmod +x /etc/ansible/hosts

Boto

ec2.py

github.com/ansible/ansible/tree/devel/contrib/inventory

Page 60: Managing Postgres with Ansible

60

ec2_keyec2_keyManages SSH key pairs that we use to connect to our instances.Parameters:

state: present or absentname: name of the SSH keykey_material: contents of the public SSH keyregion: AWS region to deploy key to (Ireland, Frankfurt, etc)wait: wait for the state 'available' before returning

- name: Ensure the SSH key is present ec2_key: state: present region: "eu-central-1" # Amazon's Frankfurt Datacenter name: "postgresql-key" # we will use this name later key_material: "{{ item }}" with_file: "id_rsa.pub" # reads contents of this file to use in "key_material"

Page 61: Managing Postgres with Ansible

61

ec2_vpcec2_vpcManages virtual private clouds & networking.

state, region, wait: ... same as previous module ...cidr_block: CIDR representing this cloud (e.g., 10.0.0.0/20)internet_gateway: is internet access enabled? resource_tags: A list of tags in form:{ tag1: value1, tag2: value2, ... } subnets: An array of subnets in form:[{ cidr: BLOCK, az: ZONE, resource_tags: [ TAGS ] }] route_tables: An array of routes in form:[{ subnets: [ BLOCKS ], routes: [ ROUTES ], resource_tags: [ TAGS ] }]

Page 62: Managing Postgres with Ansible

62

ec2_vpcec2_vpc

- name: Ensure VPC is present ec2_vpc: state: present # Create a VPC region: "eu-central-1" # Amazon's Frankfurt Datacenter cidr_block: "10.0.0.0/20" subnets: - cidr: "10.0.0.0/20" route_tables: - subnets: [ "10.0.0.0/20" ] routes: - dest: 0.0.0.0/0 # Everywhere gw: igw # Name of Amazon's default gateway internet_gateway: yes # We want our instances to connect to internet wait: yes # Wait until the VPC is ready resource_tags: { environment: "production", tier: "DB" } register: vpc # Store task results in this variable to use in later tasks

Page 63: Managing Postgres with Ansible

63

ec2_groupec2_groupManages firewalls.

state, region, name: ... same as previous module ...vpc_id: identifier of the VPCdescription: Text for purpose of this security group rules: An array of security rules in form:[{ proto: protocol, from_port: port, to_port: port, cidr_ip: block }]

protocol can be tcp, udp, icmp, any, ...port can be a port number or -1 (any port)block should be a CIDR block (e.g., 10.0.0.4/32)

Page 64: Managing Postgres with Ansible

64

ec2_groupec2_group

- name: Ensure the PostgreSQL security group is present ec2_group: state: present vpc_id: "{{ vpc.vpc_id }}" # Obtain recently provisioned VPC's ID region: "eu-central-1" # Amazon's Frankfurt Datacenter name: "PostgreSQL" description: "Security group for PostgreSQL database servers" rules: - proto: tcp # Allow SSH access from anywhere from_port: 22 to_port: 22 cidr_ip: 0.0.0.0/0 - proto: all # Allow everything from machines in the subnet from_port: -1 # Any port to_port: -1 # Any port cidr_ip: "10.0.0.0/20" register: security_group # Use results of this task later

Page 65: Managing Postgres with Ansible

65

ec2ec2Creates, deletes, starts, stops, restarts virtual machine instances.

state, region, name, wait: ... same as previous modules ...instance_tags: same as resource_tagsgroup_id: identifier of the security groupvpc_subnet_id: identifier of the VPC subnetkey_name: SSH key nameinstance_type: type of the instance (e.g., c3.large, t2.micro, etc.)private_ip: IP address in the subnet we providedassign_public_ip: should the instance have a public IP address?volumes: an array of disk volumes in form:{[ device_name: NAME, volume_size: SIZE, delete_on_termination:FLAG ]}

Page 66: Managing Postgres with Ansible

66

ec2ec2

- name: Ensure master EC2 instances & volumes are present ec2: assign_public_ip: yes # our machines should access internet instance_tags: { Name: "pg-master {{ postgresql_master_ip }}", environment: "production" } exact_count: 1 count_tag: Name: "pg-master {{ postgresql_master_ip }}" image: "ami-accff2b1" # Ubuntu Server 14.04 instance_type: "t2.micro" group_id: "{{ security_group.group_id }}" key_name: "postgresql-key" private_ip: "{{ postgresql_master_ip }}" # e.g., 10.0.0.5 region: "eu-central-1" # Amazon's Frankfurt Datacenter volumes: - device_name: /dev/sdc volume_size: "50" # 50GB disk volume delete_on_termination: false # Volume should remain after instance is deleted vpc_subnet_id: "{{ vpc.subnets[0].id }}" wait: yes # Wait until the instance becomes running register: ec2_master # Register results here so we can use them later

Page 67: Managing Postgres with Ansible

67

Action! Action!

Page 68: Managing Postgres with Ansible

68

Demo playbook structureDemo playbook structure

Page 69: Managing Postgres with Ansible

69

Playbook AWS architecturePlaybook AWS architecture

Page 70: Managing Postgres with Ansible

70

First runFirst run

Page 71: Managing Postgres with Ansible

71

Verify setupVerify setup

Page 72: Managing Postgres with Ansible

72

New standby serverNew standby server

Page 73: Managing Postgres with Ansible

73

ConclusionConclusion

Ansible loves PostgreSQL and Ansible has a very active community.<wishlist>

That's why more PostgreSQL modules would be helpful for everyone.

Making contributions to Ansible will be appreciated :)</wishlist>

Page 74: Managing Postgres with Ansible

74

Questions? Questions?

Huge Thanks!Huge Thanks!

Page 75: Managing Postgres with Ansible

75

ReferencesReferencesAnsible quick start video

Review: Puppet vs Chef vs Ansible vs Salt

Managing PostgreSQL with Ansible in EC2

Jinja2 for better Ansible playbooks and templates

Edmund The Elephant

http://www.ansible.com/videos

http://www.infoworld.com/article/2609482/data-

center/data-center-review-puppet-vs-chef-vs-ansible-vs-salt.html

https://www.safaribooksonline.com/library/view/velocity-conference-

2013/9781449371630/part22.html

https://blog.codecentric.de/en/2014/08/jinja2-better-ansible-playbooks-templates/

https://dribbble.com/shots/2223604-Edmund-The-Elephant