77
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Shahbaz Alam Manager, AWS Professional Services Peter Marney SVP, Chief Product Technology Officer, John Wiley & Sons Mahdi Sajjadpour Senior Consultant, AWS Professional Services December 1, 2016 DEV321 Enabling DevOps for an Enterprise with AWS Service Catalog The John Wiley & Sons Journey with AWS ProServe

AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Embed Size (px)

Citation preview

Page 1: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Shahbaz Alam – Manager, AWS Professional Services

Peter Marney – SVP, Chief Product Technology Officer, John Wiley & Sons

Mahdi Sajjadpour – Senior Consultant, AWS Professional Services

December 1, 2016

DEV321

Enabling DevOps for an Enterprise

with AWS Service CatalogThe John Wiley & Sons Journey with AWS ProServe

Page 2: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

What to Expect from the Session

• Understand how AWS CloudFormation and AWS Service Catalog

can be leveraged to balance control and agility.

• AWS Service Catalog Best Practices.

• Understand how to replicate the pattern used by John Wiley & Sons

to help transform your company.

Page 3: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

AWS CloudFormation

Page 4: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

AWS CloudFormation Concepts and Technology

JSON/YAML formatted file

Parameter definition

Resource creation

Configuration actions

Framework

Stack creation

Stack updates

Error detection and rollback

Configured AWS resources

Comprehensive service support

Service event aware

Customizable

Template CloudFormation Stack

Page 5: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

AWS CloudFormation Benefits

• Version control/replicate/update the templates like

application code

• Integrates with development, CI/CD, management tools

• No additional charge to use

Page 6: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Infrastructure as Code Workflow

CodeVersion Control

Code Review

Integrate Deploy

Page 7: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Infrastructure as Code Workflow

CodeVersion Control

Code Review

Integrate Deploy

Text EditorGit/SVN/

Perforce

Review Tools

Syntax Validation

Tools

AWS Services

Page 8: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Infrastructure as Code Workflow

CodeVersion Control

Code Review

Integrate Deploy

“It’s all software”

Text EditorGit/SVN/

Perforce

Review Tools

Syntax Validation

Tools

AWS Services

Page 9: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

What do customers tell us about Asset

Management Deployment? 1. Define the resources and

landscapes where software

and application are

deployed

2. ‘Approve once and deploy

many’

3. Enable self service deploy

with confidence

4. Automate deployments

Page 10: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

AWS Service CatalogBuilt to manage approved templates and control access to them

Page 11: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

AWS Service Catalog

AWS Service Catalog allows organizations to create and manage catalogs of

IT services. It enables users to quickly deploy approved IT services they need

in a self-service manner.

Administrator Users

Control

Standardization

Governance

Agility

Self-service

Time to market

Page 12: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

AWS Service Catalog – A Few Terms to Note

Product

Portfolio Stack

Constraint

an IT service that you

want to make available

for deployment on AWS.

a collection of products,

together with configuration

information.

restrict the ways that specific

AWS resources can be

deployed for a product

every AWS Service Catalog

product is launched as an AWS

CloudFormation stack

Page 13: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

AWS Service Catalog Overview

Enable

• 11 User API methods

• 37 Admin API methods

• Share products across Portfolios and AWS Accounts

Orchestrate

• Version Products

• Limit console access

• Provide various levels of user access

Automate

• Launch constraints

• Template constraints

Page 14: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Creates portfolio and

assigns product portfolio

1

Administrator

Adds constraints, grant access

and add tags

4

2 Creates

product

Authors

template

Administrator Interaction

ProductX

Versions

Portfolio BPortfolio A

• Users and Roles

• Constraints

• Tags

Service Catalog

3

DevOps

Automation

Page 15: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Opportunities to Strengthen the Handshake

User generated

products to foster

innovation

Back-end micro-services

acting on the stacks

Administrator

Products

Page 16: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Browse

Products

5

43

2

1

Portfolio

Cloud

Consumers

Select version,

Provision

Product,

configure

parametersDeploy

Notifications

and outputs

Notifications and outputs

4Scheduled

functions

Administrator

Cloud Consumer Interaction

Page 17: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

AWS Service Catalog Benefits for Enterprises

• One-stop shop for end users

• Simple user access controls to the entire AWS platform

• Built-in governance

• Granular controls on CloudFormation templates

• Version control on products

Access and Governance:

• Reusability of Products across AWS Accounts

• API/CLI and console access

• Tagging enforcement

Reusability and Automation

Page 18: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 19: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 20: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 21: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 22: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 23: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 24: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 25: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 26: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 27: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 28: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 29: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 30: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 31: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 32: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 33: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 34: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 35: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 36: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 37: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 38: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 39: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 40: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 41: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 42: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 43: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 44: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 45: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 46: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 47: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 48: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 49: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Why AWS Service Catalog for Wiley?

Standardize

Enforce Consistency

Limit Access

Enforce Tagging, Security Groups

One-Stop Shop

Automate Deployments

Agile Governance

Page 50: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Wiley AWS Service Catalog

Implementation

Page 51: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Infrastructure Meets Application Needs

web app cache database

Application A

Web Tier App Tier Cache Tier DB Tier

web server app server cache cluster database

Portfolio

Tier AlignmentAccess Alignment

Page 52: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

How Did We Approach the Environment?

- Design the Infrastructure to meet the Application

- Security and Separation at multiple levels:

- Application Level

- Application Tier Level

- Functional/Access Level

- Security/Network alignment with Application Design

Page 53: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

App Stack Deployment Model

Page 54: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Concrete

Application

Infrastructure

Environment Configuration

Application Deployment

AWS

Service Catalog

AWS

CloudFormation

AWS

CloudFormation

De

ve

lop

ment Te

am

Op

era

tio

ns T

ea

m

Au

tom

ation/R

ele

ase M

gm

t. T

ea

m

Page 55: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Developer Experience

Page 56: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Developer Experience

- Single product launch

- Application stack launch

Page 57: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Developer

Find

Product

AWS

Service Catalog

Non-Prod Workflow

web

app

dbwebAWS

CloudFormation

AWS Lambda

Launch Web

Server

Launch a Server

Amazon Route

53 hosted

zone

Amazon

CloudWatch

Events

Amazon

SNS

ITSM

Processes

Amazon

CloudWatch

Review

Metrics

Page 58: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

AWS

Service Catalog

AWS

CloudFormation

APPLICATION LOGIN PAGE

Application Deployment

Environmental Configuration

Developer

Launch an App Stack

Infrastructure Deployment

AWS Service Catalog CLI

Page 59: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Leverage the CLI to Provision a Product

]$ aws servicecatalog search-products(list all products)

]$ aws servicecatalog describe-product --id prod-XXXXXX(this gets the provisioning artifact ID)

]$ aws servicecatalog list-launch-paths --product-id prod-XXXXXX (this gets the path ID)

]$ aws servicecatalog describe-provisioning-parameters --product-id prod-XXXXX --provisioning-artifact-id checkUpdateVersion-12345678900 --path-id lp-YYYYYY (this uses the provisioning artifact ID and path ID, and gets the parameters)

Page 60: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Launch a Product with the CLI

]$ aws servicecatalog provision-product --product-id prod-XXXXX --provisioning-artifact-id checkUpdateVersion-123456789000 --path-id lp-YYYYYY --provisioning-parameters Key=KeyName,Value=MyKeyPair3 Key=InstanceType,Value=m4.medium --provisioned-product-name reInvent-CLI-example --provision-token exampletoken

(launch product with parameters listed, you can also supply a

JSON file)

Page 61: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Production Rollout Experience

Page 62: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

AWS

Service CatalogAWS

CloudFormation

APPLICATION LOGIN PAGENon-Prod

Release

Management

Finalize

template

AWS

Service Catalog

Non-Prod

Prod

Share or

Import

template

Automate

Deployments

Operations

Create

Product

Production Workflow

Trigger Infrastructure and Application

builds via Jenkins

AWS Service Catalog CLI

Page 63: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

10+ AWS Service Catalog Portfolios

50+ AWS Service Catalog Products

800+ product launches

in the past 3 months!

The Numbers…

Page 64: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Enabling DevOps

Page 65: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Consumers Creators Managers

Wiki

DevOps

Infrastructure

FAQs

Page 66: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Consumers Creators Managers

Function Consume Resources Create Artifacts

Automate Processes

Create Environment

& Manage Resources

Typical Job Role Developers Automation/Release Mgmt Operations & InfoSec

AWS Access Launch Resources Create Artifacts Manage Environment

Governance

Responsibility

Meet Cost Requirements Artifacts that meet Standards Environment &

Compliance

Logging and

Monitoring

Read-Only Create Alarms & Dashboards Monitor & Audit

Service Catalog

Alignment

EndUserFullAccess AdminFullAccess AdminFullAccess + Full

IAM access

Page 67: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Consumers Creators Managers

Function Consume Resources Create Artifacts

Automate Processes

Create Environment

& Manage Resources

AD Group Publishing-Platform-Developers Publishing-Platform-DevOps AWS-admins

IAM role Publishing-Platform-Developers Publishing-Platform-DevOps AWS-admins

Policies attached

to Roles

ServiceCatalogEndUserFullAccess

ReadOnlyAccess

AWSSupportAccess

CloudWatchCreateDashboard

ServiceCatalogAdminFullAccess

ReadOnlyAccess

AWSSupportAccess

CloudFrontFullAccess

PublishingSQSAccess

AdministratorAccess

Service Catalog

Portfolio Access

Publishing-Platform Publishing-Platform

All of Service Catalog

All of Service Catalog

Example

Page 68: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Creates AD groups and AWS

IAM roles for application,

create IAM policiesOperations

Defines and creates Launch

constraints

2

Operations/Infrastructure InteractionManaging Environment

Web

Server

Versions

Application BApplication A

• Users

• Constraints

• Tags

Service Catalog1

Defines template constraints

AMI, security group, subnet,

instance types, tags

3

Page 69: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Creates portfolio and

assigns products to portfolio

1

Adds template constraints,

grant access and add tags

4

2 Creates

product

Authors

template

Automation/Release Mgmt InteractionManaging & Creating Products

Web

Server

Versions

Application BApplication A

• Users

• Constraints

• Tags

Service Catalog

3

Release

Mgmt

Page 70: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Set Constraints with CLI

]$ aws servicecatalog create-constraint --portfolio-id port-ZZZZZZ --product-id prod-XXXXXX --parameters "{\"Rules\": {\"Rule1\": {\"Assertions\": [{\"Assert\": {\"Fn::Contains\": [[\"EXAMPLE-AMI-ID-1\",\"EXAMPLE-AMI-ID-2\"],{\"Ref\": \"ami-id\"}]},\"AssertDescription\": \"AMI ID should be either EXAMPLE-AMI-ID-1 or EXAMPLE-AMI-ID-2\"}]}}}" --type TEMPLATE –idempotency-token exampletoken

New marketplace AMI

Custom AMI

AMI

Template

Constraint

Page 71: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Alignment Consistency ReusabilityAgility & Flexibility

Time to Market

Built-In Governance

Automation

Page 72: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 73: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 74: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 75: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)
Page 76: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Thank you!

Page 77: AWS re:Invent 2016: Enabling DevOps for an Enterprise with AWS Service Catalog: The John Wiley & Sons Journey with AWS Professional Services (DEV321)

Remember to complete

your evaluations!