Upload
amazon-web-services
View
2.760
Download
1
Embed Size (px)
Citation preview
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Chris Munns
Business Development Manager – DevOps
Amazon Web Services
September 2016
Infrastructure as Code: Best
Practices with AWS CloudFormation
CloudFormation concepts and technology
JSON/YAML formatted file
Parameter definition
Resource creation
Configuration actions
Framework
Stack creation
Stack updates
Error detection and rollback
Configured AWS resources
Comprehensive service support
Service event aware
Customizable
Template CloudFormation Stack
Infrastructure as Code workflow
codeversion control
code review
integrate deploy
Infrastructure as Code workflow
codeversion control
code review
integrate deploy
Text EditorGit/SVN/
Perforce
Review Tools
Syntax Validation
Tools
AWS Services
Infrastructure as Code workflow
codeversion control
code review
integrate deploy
“It’s all software”
Text EditorGit/SVN/
Perforce
Review Tools
Syntax Validation
Tools
AWS Services
In-place Blue-Green
Traffic
• Quicker and more cost efficient
• Updates or replaces resources
• Simpler state and data
migration
• Working stack stays intact for
easy roll backs
• Potentially need to sync data
• Shift traffic to new stack
Templates
Stacks
Update like software
Traffic
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "AWS CloudFormation Sample Template EC2InstanceSample: **WARNING** This template an Amazon EC2 instances. You will be billed for the AWS resources used if
you create a stack from this template.",
"Parameters" : {
"KeyName" : {
"Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance",
"Type" : "String"
},
"Environment": {
"Type" : "String",
"Default" : ”Dev",
"AllowedValues" : [”Mgmt", "Dev", ”Staging", "Prod"],
"Description" : "Environment that the instances will run in.”
}
},
"Mappings" : {
"RegionMap" : {
"us-east-1" : { "AMI" : "ami-7f418316" },
"us-west-2" : { "AMI" : "ami-16fd7026" }
}
},
"Conditions" : {
”EnableEBSOptimized" : {"Fn::Equals" : [{"Ref" : " Environment "}, ”Prod"]},
},
"Resources" : {
"Ec2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"KeyName" : { "Ref" : "KeyName" },
"EbsOptimized " : {"Fn::If": [ " EnableEBSOptimized ", {“true”}, {“false”}]},
"ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]},
"UserData" : { "Fn::Base64" : "80" }
}
}
},
"Outputs" : {
"InstanceId" : {
"Description" : "InstanceId of the newly created EC2 instance",
"Value" : { "Ref" : "Ec2Instance" }
},
"PublicDNS" : {
"Description" : "Public DNSName of the newly created EC2 instance",
"Value" : { "Fn::GetAtt" : [ "Ec2Instance", "PublicDnsName" ] }
}
}
}
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "AWS CloudFormation Sample Template EC2InstanceSample: **WARNING** This template an Amazon EC2 instances. You will be billed for the AWS resources used if
you create a stack from this template.",
"Parameters" : {
"KeyName" : {
"Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance",
"Type" : "String"
},
"Environment": {
"Type" : "String",
"Default" : ”Dev",
"AllowedValues" : [”Mgmt", "Dev", ”Staging", "Prod"],
"Description" : "Environment that the instances will run in.”
}
},
"Mappings" : {
"RegionMap" : {
"us-east-1" : { "AMI" : "ami-7f418316" },
"us-west-2" : { "AMI" : "ami-16fd7026" }
}
},
"Conditions" : {
”EnableEBSOptimized" : {"Fn::Equals" : [{"Ref" : " Environment "}, ”Prod"]},
},
"Resources" : {
"Ec2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"KeyName" : { "Ref" : "KeyName" },
"EbsOptimized " : {"Fn::If": [ " EnableEBSOptimized ", {“true”}, {“false”}]},
"ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]},
"UserData" : { "Fn::Base64" : "80" }
}
}
},
"Outputs" : {
"InstanceId" : {
"Description" : "InstanceId of the newly created EC2 instance",
"Value" : { "Ref" : "Ec2Instance" }
},
"PublicDNS" : {
"Description" : "Public DNSName of the newly created EC2 instance",
"Value" : { "Fn::GetAtt" : [ "Ec2Instance", "PublicDnsName" ] }
}
}
}
HEADERS
PARAMETERS
MAPPINGS
RESOURCES
OUTPUTS
CONDITIONALS
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Description" : "AWS CloudFormation Sample Template EC2InstanceSample: **WARNING** This template an Amazon EC2 instances. You will be billed for the AWS resources used if
you create a stack from this template.",
"Parameters" : {
"KeyName" : {
"Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance",
"Type" : "String"
},
"Environment": {
"Type" : "String",
"Default" : ”Dev",
"AllowedValues" : [”Mgmt", "Dev", ”Staging", "Prod"],
"Description" : "Environment that the instances will run in.”
}
},
"Mappings" : {
"RegionMap" : {
"us-east-1" : { "AMI" : "ami-7f418316" },
"us-west-2" : { "AMI" : "ami-16fd7026" }
}
},
"Conditions" : {
”EnableEBSOptimized" : {"Fn::Equals" : [{"Ref" : " Environment "}, ”Prod"]},
},
"Resources" : {
"Ec2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"KeyName" : { "Ref" : "KeyName" },
"EbsOptimized " : {"Fn::If": [ " EnableEBSOptimized ", {“true”}, {“false”}]},
"ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]},
"UserData" : { "Fn::Base64" : "80" }
}
}
},
"Outputs" : {
"InstanceId" : {
"Description" : "InstanceId of the newly created EC2 instance",
"Value" : { "Ref" : "Ec2Instance" }
},
"PublicDNS" : {
"Description" : "Public DNSName of the newly created EC2 instance",
"Value" : { "Fn::GetAtt" : [ "Ec2Instance", "PublicDnsName" ] }
}
}
}
HEADERS
PARAMETERS
MAPPINGS
RESOURCES
OUTPUTS
CONDITIONALS
Description of what your stack does, contains, etc
Provision time values that add structured flexibility and
customization
Pre-defined conditional case statements
Conditional values set via evaluations of passed references
AWS resource definitions
Resulting attributes of stack resource creation
{ "Description" : "Create an EC2 instance.”,"Parameters": {
"KeyName": { "Description" : "Name of an existing EC2 KeyPair to enable SSH
access into the WordPress web server", "Type": "AWS::EC2::KeyPair::KeyName"
},"EC2InstanceType" : {
"Description" : "EC2 instance type", "Type" : "String","Default" : "t2.micro", "AllowedValues" : [ "t2.micro", "t2.small", "t2.medium" ], "ConstraintDescription" : "Must be t2.micro, t2.small, t2.medium"
},},
Template Anatomy - Parameters
{ "Description" : "Create an EC2 instance.”,"Resources" : {
"Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : {
"KeyName" : “my-key-pair”, "ImageId" : "ami-6869aa05”,“InstanceType” : “m3.medium”
} }
}}
Template Anatomy - Resources
"Outputs" : { "WebsiteURL" : {
"Description" : ”DNS name of the website", "Value" : {
"Fn::GetAtt" : [ “LoadBalancer”, “DNSName” ] }
} }
Template Anatomy - Outputs
CloudFormation Best Practices
Visualize template resources
Modify template with drag-drop gestures
Customize sample templates
CloudFormation Designer
Avoid manual resource modifications
Avoid making quick-fixes out of band
Update your stacks with CloudFormation
Do not manually change resources
Consider using resource based permissions to
limit ability to make changes directly
Preview updates with Change Sets
Learn the intrinsic functions
"Mappings" : {"RegionMap" : {
"us-east-1" : { "32" : "ami-6411e20d", "64" : "ami-7a11e213" },"us-west-1" : { "32" : "ami-c9c7978c", "64" : "ami-cfc7978a" },"eu-west-1" : { "32" : "ami-37c2f643", "64" : "ami-31c2f645" },"ap-southeast-1" : { "32" : "ami-66f28c34", "64" : "ami-60f28c32" },"ap-northeast-1" : { "32" : "ami-9c03a89d", "64" : "ami-a003a8a1" }
}},
Fn::FindInMap
Fn::FindInMap
"Resources" : {"myEC2Instance" : {
"Type" : "AWS::EC2::Instance","Properties" : {
"ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "32"]},
"InstanceType" : "m1.small"}
}}
"Resources" : {
"Ec2Instance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"KeyName" : { "Ref" : "KeyName" },
"SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ],
"ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]},
"UserData" : { "Fn::Base64" : { "Fn::Join" : ["",[
"#!/bin/bash -ex","\n",
"yum -y install gcc-c++ make","\n",
"yum -y install mysql-devel sqlite-devel","\n",
"yum -y install ruby-rdoc rubygems ruby-mysql ruby-devel","\n",
"gem install --no-ri --no-rdoc rails","\n",
"gem install --no-ri --no-rdoc mysql","\n",
"gem install --no-ri --no-rdoc sqlite3","\n",
"rails new myapp","\n",
"cd myapp","\n",
"rails server -d","\n"]]}}
}
}
Use EC2 UserData, which is available as a property of AWS::EC2::Instance resources
Bootstrap your applications using EC2 UserData
AWS CloudFormation provides helper scripts
for deployment within your EC2 instances
Metadata Key —AWS::CloudFormation::Init
Cfn-init reads this metadata key and installs
the packages listed in this key (e.g., httpd,
mysql, and php). Cfn-init also retrieves and
expands files listed as sources.
Amazon EC2
AWS CloudFormation
cfn-init
cfn-hup
cfn-signal
cfn-get-
metadata
Bootstrap your applications using helper scripts
"Metadata": {
"AWS::CloudFormation::Init" : {
"config" : {
"packages" : {
},
"sources" : {
},
"commands" : {
},
"files" : {
},
"services" : {
},
"users" : {
},
"groups" : {
}
}
}
“WebAppHost" : {
"Type" : "AWS::EC2::Instance",
"Metadata" : {
"AWS:CloudFormation::Init" : {
"config" : {
"packages" : {
"yum" : {
"gcc" : [],
"gcc-c++" : [],
"make" : [],
"automake" : [],
Use AWS::CloudFormation::Init with cfn-init to help bootstrap instances:
Bootstrapping example
{
"Statement" : [
{
"Effect" : "Allow",
"Action" : "Update:*",
"Principal": "*",
"Resource" : "*"
},
{
"Effect" : "Deny",
"Action" : "Update:*",
"Principal": "*",
"Resource" : "LogicalResourceId/ProductionDatabase"
}
]
}
Prevent stack updates to protected resources using Stack policies
Protect your resources using Stack policies
Ownership based template design
Limit one template to a single service
Aim for reusability across environments (development, testing, production)
Use nested stacks and cross-stack reference to break up large templates
Organize templates according to team structure/job function/line of business
Ownership based template design
Template File
Defining Stack
The entire infrastructure can be represented in an AWS
CloudFormation template.
Many Stacks & Environments from One Template
Template File
Defining Stack
The entire infrastructure can be represented in an AWS
CloudFormation template.
Use the version control system of your choice to store and track changes to this template
Many Stacks & Environments from One Template
Git
Perforce
SVN
…
Template File
Defining Stack
Git
Perforce
SVN
…
Dev
Test
Prod
The entire infrastructure can be represented in an AWS
CloudFormation template.
Use the version control system of your choice to store and track changes to this template
Build out multiple environments, such as for Development, Test, Production and even DR using the same template
Many Stacks & Environments from One Template
Ownership based template design
Ownership – nested stacks
Web-SG
Ownership – cross-stack references
App-SG
App-SG
DB-SG
Re-usable Templates – across AWS Regions
Consider environmental or regional differences
Amazon EC2 image Ids
VPC environment or “classic” environment
Available instance types
IAM policy principals
Endpoint names
Amazon Resource Names (arns)
"LogsBucketPolicy": {"Type": "AWS::S3::BucketPolicy","Properties": {
"Bucket": {"Ref": "LogsBucket”},"PolicyDocument": {
"Version": "2008-10-17","Statement": [{"Sid": "ELBAccessLogs","Effect": "Allow","Resource": {
"Fn::Join": [ "", [ “arn:aws:s3:::", { "Ref": "LogsBucket" }, "/", "Logs",
"/AWSLogs/", { "Ref": "AWS::AccountId" }, "/*” ]]},"Principal": …,"Action": [ "s3:PutObject" ]
}]}
}},
Use “pseudo-parameters” to retrieve environmental data
Account Id
Region
Stack Name and Id
Re-usable Templates – “Pseudo-Parameters”
Re-usable Templates - Using mappings
"LogsBucketPolicy": {"Type": "AWS::S3::BucketPolicy","Properties": {
"Bucket": {"Ref": "LogsBucket”},"PolicyDocument": {"Version": "2008-10-17","Statement": [{
"Sid": "ELBAccessLogs","Effect": "Allow","Resource": {"Fn::Join": [ "", [
{ "Fn::FindInMap" : ["RegionalConfig", {"Ref" : "AWS::Region"},"ArnPrefix”]},
"s3:::”, { "Ref": "LogsBucket" }, "/", "Logs", "/AWSLogs/”,{ "Ref": "AWS::AccountId" }, "/*" ] ]
},}
“Mappings” : {
“RegionalConfig” : {“us-east-1” : {
“AMI” : “ami-12345678”,
”ELBAccountId": "127311923021”,
“ArnPrefix” : “arn:aws:”
},“us-west-1” : {
“AMI” : “ami-98765432”
”ELBAccountId": “027434742980"
“ArnPrefix” :“arn:aws:”
},:
}}
Use mappings to define variables
Single place for configuration
Re-usable within the template
Use conditionals to customize
resources and parameters
"DBEC2SG": {"Type": "AWS::EC2::SecurityGroup","Condition" : "Is-EC2-VPC","Properties" : {…}
},
"DBSG": {"Type": "AWS::RDS::DBSecurityGroup","Condition" : "Is-EC2-Classic","Properties": {…}
},
"MySQLDatabase": {"Type": "AWS::RDS::DBInstance","Properties": {
:
"VPCSecurityGroups": { "Fn::If" : [ "Is-EC2-VPC", [ { "Fn::GetAtt": [ "DBEC2SG", "GroupId" ] } ], { "Ref" : "AWS::NoValue"}]},
"DBSecurityGroups": { "Fn::If" : [ "Is-EC2-Classic", [ { "Ref": "DBSG" } ],{ "Ref" : "AWS::NoValue"}]}
}}
}
"Conditions" : {
"Is-EC2-VPC” : { "Fn::Or" : [ {"Fn::Equals" : [
{"Ref” : "AWS::Region"},"eu-central-1" ]},
{"Fn::Equals" : [{"Ref" : "AWS::Region"}, "cn-north-1" ]}]},
"Is-EC2-Classic" : { "Fn::Not" : [{ "Condition" : "Is-EC2-VPC"}]}
},
Re-usable Templates – Using conditionals
Best Practices Summary
CloudFormation Designer
Avoid manual resource modifications
Preview updates with Change Sets
Learn the intrinsic functions
Bootstrap your applications using UserData and helper scripts
Protect critical resources using stack policies
Ownership based template design
Plan for environment and multi-region
Use Pseudo-Parameters
Use Mappings
Use Conditionals
YAML formatted templates
Overview of template structure / basics
New function formatting (!Ref / !GetAZs / !FindInMap)
New Intrinsic Function ( Fn::Sub )
Cross Stack References
New function Fn::ImportValue
Allows use of outputs from unrelated stacks with custom resource
New
Key new features
New
CloudFormation - YAML
Why YAML?
Better authoring and readability of templates
Native comment support
Simplification as templates get more and more complex
New
Cloudformation - YAML
Resources:
VPC1:
Type: "AWS::EC2::VPC"
Properties:
CidrBlock: !Ref VPC1Cidr
Tags:
-
Key: "Name"
Value: "TroubleShooting"
Structure is shown through indentation (one or more spaces).
Sequence items are denoted by a dash
Key value pairs within a map are separated by a colon.
Tips: Use a monospace font, don’t use Tab, save using UTF-8
All sections are the same as in a JSON template
---
AWSTemplateFormatVersion: "version date"
Description:
String
Metadata:
template metadata
Parameters:
set of parameters
Mappings:
set of mappings
Conditions:
set of conditions
Resources:
set of resources
Outputs:
set of outputs
CloudFormation – YAML Template Structure
Two ways to declare Intrinsic functions: Long and Short
Short Form:
!FindInMap [ MapName, TopLevelKey, SecondLevelKey ]
Long Form:
"Fn::FindInMap" : [ "MapName", "TopLevelKey", "SecondLevelKey"]
Tag = ! (Its not Negation operator)
Few things to note with Tags
You cannot use one tag immediately after another
!Base64 !Sub…
Instead, you can do this
"Fn::Base64": !Sub...
!Select [ !Ref Value, [1,2,3]]
CloudFormation – YAML Function Declaration
Fn::Base64 Fn::AndShort !Base64 valueToEncode Short !And [condition]
Long "Fn::Base64": valueToEncode Long "Fn::And": [condition]
Fn::Equals Fn::IfShort !Equals [value_1, value_2] Short !If [condition_name, value_if_true, value_if_false]
Long "Fn::Equals": [value_1, value_2] Long "Fn::If": [condition_name, value_if_true, value_if_false]
Fn::Not Fn::OrShort !Not [condition] Short !Or [condition, ...]
Long "Fn::Not": [condition] Long "Fn::Or": [condition, ...]
CloudFormation – Intrinsic Functions
NewCloudFormation – Fn::Sub
Substitute variables in an input string with values
Function accepts a string or a map as a parameter.
Usage
VarName: ${MyVariableValue}
Literal: ${!LiteralValue}
Use | if you are spanning multiple lines
Available in JSON as well
/tmp/create-wp-config:
content: !Sub |
#!/bin/bash -xe
cp /var/www/html/wordpress/wp-config-sample.php /var/www/html/wordpress/wp-config.php
sed -i "s/'database_name_here'/'${DBName}'/g" wp-config.php
sed -i "s/'username_here'/'${DBUser}'/g" wp-config.php
sed -i "s/'password_here'/'${DBPassword}'/g" wp-config.php
mode: '000500'
owner: root
group: root
configure_wordpress:
commands:
01_set_mysql_root_password:
command: !Sub |
mysqladmin -u root password '${DBRootPassword}'
test: !Sub |
$(mysql ${DBName} -u root --password='${DBRootPassword}' >/dev/null 2>&1 </dev/null); (( $? != 0 ))
02_create_database:
command: !Sub |
mysql -u root --password='${DBRootPassword}' < /tmp/setup.mysql
test: !Sub |
$(mysql ${DBName} -u root --password='${DBRootPassword}' >/dev/null 2>&1 </dev/null); (( $? !=0))
03_configure_wordpress:
command: /tmp/create-wp-config
cwd: /var/www/html/wordpress
CloudFormation – Fn::Sub Examples
CloudFormation – Cross Stack References
Sharing resources made easy
IAM roles, VPC, Security groups
Add an explicit “Export” declaration to stack output
Use the resource in another stack using a new intrinsic function,
Fn::ImportValue`
Few guidelines:
Export names must be unique within an account and region
Cannot create references across regions
Cannot delete a stack that is referenced by another stack (Dependencies
are communicated in errors)
Outputs cannot be modified or removed as long as it is referenced by a
current stack
New
The new intrinsic function for accessing exported outputs.
JSON{ "Fn::ImportValue" : sharedValueToImport }
YAML"Fn::ImportValue": sharedValueToImport
!ImportValue sharedValueToImport
CloudFormation – Fn::ImportValue
Stack A
Stack B
"Outputs": {
"WebServerSecurityGroup": {
"Description": "TheIDofthesecuritygroup",
"Value": {"Fn: : GetAtt": ["WebServerSecurityGroup", "GroupId"]},
"Export": { "Name": "AccountSecGroup"}}
}
"Resources" : {
"WebServerInstance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"InstanceType" : "ts.micro",
"ImageId" : "ami-a1b23456",
"NetworkInterfaces" : [{
"GroupSet" : [{ "Fn::ImportValue" : "AccountSecGroup" ]} ]}
}
}
}
CloudFormation – Cross Stack Examples
Questions?
Thank you!