Upload
amazon-web-services
View
3.600
Download
5
Embed Size (px)
DESCRIPTION
AWS OpsWorks lets you model your application with layers that define the building blocks of your application: load balancers, application servers, databases, etc. But did you know that you can also extend OpsWorks layers or build your own custom layers? Whether you need to perform a specific task or install a new software package, OpsWorks gives you the tools to install and configure your instances consistently, and evolve them in an automated and predictable fashion through your application’s lifecycle. We'll dive into the development process including how to use attributes, recipes, and lifecycle events; show how to develop your environment locally; and provide troubleshooting steps that reduce your development time.
Citation preview
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
DMG304 - AWS OpsWorks Under the Hood
Jonathan Weiss & Reza Spagnolo, Amazon Web Services
November 14th, 2013
Agenda
• AWS OpsWorks event life cycle
• How to use custom cookbooks: MongoDB
• Chef in AWS OpsWorks vs. Chef server
Event Life Cycle
Agent on each
EC2 instance
The Heart of the Service
4
Agent on each
Amazon EC2 instance
Understands a set of commands that are
triggered by AWS OpsWorks.
The agent then runs a Chef solo run.
Life Cycle Events
5
setup configure deploy undeploy shutdown
Continuous Configuration new
Continuous Configuration new
Continuous Configuration new
onlin
e
setup
configure
Continuous Configuration new
onlin
e
setup
configure
deploy
Continuous Configuration n
ew
/ s
topped
onlin
e
setup
configure
terminating
shutting
down
deploy
configure
Setup Event
• Sent when instance boots
• Includes deploy event
• Use for initial installation
of software & services
Setup Event – Recipe Execution Order
AWS OpsWorks
setup recipes
Your setup recipes
AWS OpsWorks
deploy recipes
Your deploy recipes
Configure Event
• Sent to all instances when
any instance enters or
leaves online state
• Use for making sure the
configuration is up-to-date
Deploy Event
• Sent by deploy via UI/API
and is also part of each
setup
• Use for custom deployment
Undeploy Event
• Sent via UI/API when
apps are deleted
• Use to remove apps from
running instances
Shutdown Event
• Sent when an instance
is shut down
• ~45s to execute
• Use for clean shutdown
Customizing AWS OpsWorks
Customization Options
Built-in layers
Override Chef attributes via custom JSON
Override Chef attributes via custom cookbook
Overwrite Chef template file
Deploy hooks
Provide custom recipe to extend built-in layer
Provide custom recipe to create custom layer contr
ol
sim
ple
Built-in Layers
Open Source at http://github.com/aws/opsworks-cookbooks
Rails MySQL
PHP HAProxy
Node.js Memcached
Java Ganglia
Chef Templates
Template for /etc/apache2/apache2.conf
Custom JSON
Override Chef attributes of built-in cookbooks
# in apache2/attributes/default.rb default[:apache][:timeout] = 120
node.apache.timeout # => 60
{ ‘apache2‘: {‘timeout‘: 60 } }
Custom Cookbook Attributes
Override Chef attributes of built-in cookbooks
# in apache2/attributes/default.rb default[:apache][:timeout] = 120
node.apache.timeout # => 60
# in mycookbook/attributes/apache.rb set[:apache][:timeout] = 60
Overwriting Chef Templates
Provide custom file at same location
$ cat mycookbook-repo/apache2/templates/apache2.conf.erb ... # # Example custom content # LogLevel error # change default type DefaultType text/json
Deploy Hooks
$ ls myapp/deploy/
before_migrate.rb
before_symlink.rb
before_restart.rb
after_restart.rb
Hooks use Capistrano syntax
Example Deploy Hook
Rails asset pipeline support
$ cat deploy/before_symlink.rb run "cd #{release_path} && \
RAILS_ENV=production bundle exec rake assets:precompile
sudo "/etc/init.d/myservice restart"
Custom Cookbooks
Custom Layers
Run anything that is scriptable with Chef – Erlang app server
– Cassandra DB cluster
– C daemon
– Custom PHP install
– …
Custom Layers
Custom Layers
Custom Layers
$ ls mycookbook-repo/erlang/
recipes/install.rb
recipes/uninstall.rb
$ ls mycookbook-repo/e-app/
recipes/deploy.rb
recipes/undeploy.rb
Custom Layers
Different Configuration Managers
Call bash script from Chef bash "legacy bootstrap" do user "root" cwd "/tmp" code <<-EOH wget http://www.example.com/installer.tar.gz tar -zxf installer.tar.gz cd installer ./configure make make install EOH end
AWS OpsWorks Attribute Tree
Custom JSON
Stack configuration JSON
Deployment JSON
Cookbook attributes
Chef - Ohai
Sources:
AWS OpsWorks Attribute Tree - Categories
Multiple categories and namespaces
node[:opsworks]
node[:opsworks][:layers]
node[:opsworks][:instance]
node[:opsworks][:stack]
Built in layer specific -> i.e. node[:opsworks_java]
Built in recipe specific -> i.e. node[:apache2]
Walkthrough:
MongoDB on AWS OpsWorks
Architecture – MongoDB Replica Set
Objectives
• MongoDB as custom layer
• Use community cookbook
• Extend Java application layer
• Manage system life cycle
MongoDB Custom Layer
• Chef community cookbook for MongoDB – https://github.com/edelight/chef-mongodb
• Cookbook functionality – Install
– Register with peers
– Service startup
– Clean shutdown
MongoDB Custom Layer - Recipes
MongoDB Custom Layer - Changes
Enable node discovery through AWS OpsWorks JSON
def replicaset_members(node)
replicaset_layer = node['opsworks']['instance']['layers'].first
instances = node['opsworks']['layers'][replicaset_layer]['instances']
instances.map do |name, instance|
member = Chef::Node.new
member.name(name)
member.default['hostname'] = name
member.default['fqdn'] = instance['private_dns_name']
member.default['ipaddress'] = instance['private_ip']
member
end
end
MongoDB Custom Layer - Configuration
Custom JSON in MongoDB stack
Made available to every
Chef recipe
Integrates out of the box
with community cookbook
MongoDB Custom Layer – EBS Setup
AWS OpsWorks support for EBS RAID
Setup the mount point passed through custom JSON
Extend Application Layer - Configure
Leverage configure
event definition of
built-in Java layer
Extend Application Layer - Template
Override application context template of built-in Java layer
$ cat opsworks_java/templates/default/webapp_context.xml.erb
<%
replicaset_name = node['mongodb']['replicaset_name']
mongo_nodes = node['opsworks']['layers'][replicaset_name]
['instances'].keys.map{|name| "#{name}:27017"}.join(",")
%>
<Context>
<Environment name="mongoNodes"
type="java.lang.String"
value="<%= mongo_nodes %>" />
</Context>
Package Your Custom Cookbook
Package your cookbook and its dependencies in a
single archive
java-mongodb-compound-cookbook/
├── apt/
├── build-essential/
├── python/
├── yum/
├── opsworks_java/
└── chef-mongodb/
Done !
Chef in AWS OpsWorks
vs.
Chef Server
Main Differences
• Chef setup
• One run vs. discrete events
• Push vs. pull
• Discovery: search & AWS OpsWorks attribute tree
• Data bags
Chef Setup
Chef Solo
&
AWS OpsWorks Backend
Chef Client
&
Chef Server
Life Cycle Events
• Give you fine-grained control
• Faster to execute
• Context
Push vs. Pull
• On-demand and automatic
• Respond immediately to changes in the stack
Discovery: Chef Search
AWS OpsWorks does not offer attribute search
Alternative:
node[:opsworks] with similar capabilities
to partial_search
AWS OpsWorks Attribute Tree
Find all Rails application servers
rails_servers = node['opsworks']['layers']['rails-app']['instances']
rails_ips = rails_servers.map{|i| i['private_ip'] }
template "/etc/rails-server.conf" do
...
variables({
:ips => rails_server_ips
})
end
Encrypted Data Bags
• Upload encrypted JSON to S3
• Have instances access via IAM roles in a recipe
Custom JSON
Arbitrary JSON on stack that is available in Chef
if node.foo.bar ... elsif node.foo.baz ... end
{ ‘foo‘: { ‘bar‘: true, ‘baz‘: false } }
Store Secrets on Amazon S3
Access from instance via instance profiles
bucket = node['acme']['bucket'] key = node['acme']['key'] s3 = AWS::S3.new obj = s3.buckets[bucket].objects[key] obj.read
Recap
AWS OpsWorks
• Life cycle framework
• Highly customizable
• Run anything
More Information about AWS OpsWorks
• Do the AWS OpsWorks lab!
• Find us on the AWS Booth
• Follow us on twitter @AWSOpsWorks
• Find us on YouTube
• Blog: http://blogs.aws.amazon.com/application-management
• Survey: http://tinyurl.com/OpsWorksSurvey2013
Recommend Sessions
DMG305 - How Intuit Leveraged AWS OpsWorks
as the Engine of Our PaaS
Capen Brinkley & Rick Mendes of Intuit, inc.
Thursday, Nov 14, 4:15 PM - 5:15 PM – Murano 3206
Please give us your feedback on this
presentation
As a thank you, we will select prize
winners daily for completed surveys!
DMG304