Upload
amazon-web-services
View
1.022
Download
3
Embed Size (px)
Citation preview
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Kyle Knapp, Amazon Web Services
December 1, 2016
DEV402
The Effective AWS CLI User
Important:
This talk will be advanced
Background Material
• AWS CLI User Guide
• AWS CLI Command Reference
• 2015 AWS CLI re:Invent talk
• 2014 AWS CLI re:Invent talk
• 2013 AWS CLI re:Invent talk
Background Material
• AWS CLI User Guide
• AWS CLI Command Reference
• 2015 AWS CLI re:Invent talk
• 2014 AWS CLI re:Invent talk
• 2013 AWS CLI re:Invent talk
Slides will be online!
Unified tool to manage AWS services
AWS Command Line Interface
The Effective AWS CLI User Tenets
The effective AWS CLI user:
The Effective AWS CLI User Tenets
The effective AWS CLI user:
Uses an iterative workflow
The Effective AWS CLI User Tenets
The effective AWS CLI user:
Uses an iterative workflow
Troubleshoots well
The Effective AWS CLI User Tenets
The effective AWS CLI user:
Uses an iterative workflow
Troubleshoots well
Is resourceful with tooling
The Effective AWS CLI User Tenets
The effective AWS CLI user:
Uses an iterative workflow
Troubleshoots well
Is resourceful with tooling
Understands performance implications
A Sample Application
Amazon VPC
Amazon VPC Amazon EC2
Amazon VPC Amazon EC2 IAM
Amazon VPC Amazon EC2 IAM Amazon S3
AWS Cloud
VPC
10.0.0.0/16
AWS Cloud
router
Destination Target
10.0.0.0/16 local
VPC
10.0.0.0/16
Subnet
10.0.0.0/24
AWS Cloud
router
Destination Target
10.0.0.0/16 local
VPC
10.0.0.0/16
Subnet
10.0.0.0/24
AWS Cloud
Internet
gateway
router
Destination Target
10.0.0.0/16 local
0.0.0.0/0 igw-id
VPC
10.0.0.0/16
Subnet
10.0.0.0/24
security group
AWS Cloud
Internet
gateway
router
Destination Target
10.0.0.0/16 local
0.0.0.0/0 igw-id
VPC
10.0.0.0/16
Subnet
10.0.0.0/24
security group
AWS Cloud
instance
Internet
gateway
router
Destination Target
10.0.0.0/16 local
0.0.0.0/0 igw-id
role
VPC
10.0.0.0/16
Subnet
10.0.0.0/24
security group
AWS Cloud
instance
Internet
gateway
router
Amazon
S3
Destination Target
10.0.0.0/16 local
0.0.0.0/0 igw-id
role
The Effective AWS CLI User Tenets
The effective AWS CLI user:
Uses an iterative workflow
$ aws ec2 create-vpc --cidr-block 10.0.0.0/16
$ aws ec2 create-vpc --cidr-block 10.0.0.0/16
{
"Vpc": {
"VpcId": "vpc-24de2d4d",
"InstanceTenancy": "default",
"State": "pending",
"DhcpOptionsId": "dopt-f8ca2291",
"CidrBlock": "10.0.0.0/16",
"IsDefault": false
}
}
$ aws ec2 create-vpc --cidr-block 10.0.0.0/16
{
"Vpc": {
"VpcId": "vpc-24de2d4d",
"InstanceTenancy": "default",
"State": "pending",
"DhcpOptionsId": "dopt-f8ca2291",
"CidrBlock": "10.0.0.0/16",
"IsDefault": false
}
}
$ aws ec2 create-vpc --cidr-block 10.0.0.0/16
{
"Vpc": {
"VpcId": "vpc-24de2d4d",
"InstanceTenancy": "default",
"State": "pending",
"DhcpOptionsId": "dopt-f8ca2291",
"CidrBlock": "10.0.0.0/16",
"IsDefault": false
}
}
$ vpc_id=vpc-24de2d4d
$ aws ec2 create-vpc --cidr-block 10.0.0.0/16
{
"Vpc": {
"VpcId": "vpc-24de2d4d",
"InstanceTenancy": "default",
"State": "pending",
"DhcpOptionsId": "dopt-f8ca2291",
"CidrBlock": "10.0.0.0/16",
"IsDefault": false
}
}
$ vpc_id=vpc-24de2d4dWe can do better…
$ vpc_id=$(aws ec2 create-vpc --cidr-block 10.0.0.0/16 \
--query Vpc.VpcId --output text)
$ vpc_id=$(aws ec2 create-vpc --cidr-block 10.0.0.0/16 \
--query Vpc.VpcId --output text)
{"Vpc": {
"VpcId": "vpc-24de2d4d", "InstanceTenancy": "default", "State": "pending", "DhcpOptionsId": "dopt-f8ca2291", "CidrBlock": "10.0.0.0/16", "IsDefault": false
}}
$ vpc_id=$(aws ec2 create-vpc --cidr-block 10.0.0.0/16 \
--query Vpc.VpcId --output text)
{"Vpc": {
"VpcId": "vpc-24de2d4d", "InstanceTenancy": "default", "State": "pending", "DhcpOptionsId": "dopt-f8ca2291", "CidrBlock": "10.0.0.0/16", "IsDefault": false
}}
$ vpc_id=$(aws ec2 create-vpc --cidr-block 10.0.0.0/16 \
--query Vpc.VpcId --output text)
{"Vpc": {
"VpcId": "vpc-24de2d4d", "InstanceTenancy": "default", "State": "pending", "DhcpOptionsId": "dopt-f8ca2291", "CidrBlock": "10.0.0.0/16", "IsDefault": false
}}
$ vpc_id=$(aws ec2 create-vpc --cidr-block 10.0.0.0/16 \
--query Vpc.VpcId --output text)
{"Vpc": {
"VpcId": "vpc-24de2d4d", "InstanceTenancy": "default", "State": "pending", "DhcpOptionsId": "dopt-f8ca2291", "CidrBlock": "10.0.0.0/16", "IsDefault": false
}}
$ vpc_id=$(aws ec2 create-vpc --cidr-block 10.0.0.0/16 \
--query Vpc.VpcId --output text)
{"Vpc": {
"VpcId": "vpc-24de2d4d", "InstanceTenancy": "default", "State": "pending", "DhcpOptionsId": "dopt-f8ca2291", "CidrBlock": "10.0.0.0/16", "IsDefault": false
}}
$ vpc_id=$(aws ec2 create-vpc --cidr-block 10.0.0.0/16 \
--query Vpc.VpcId --output text)
$ echo "$vpc_id”
vpc-24de2d4d
$ vpc_id=$(aws ec2 create-vpc --cidr-block 10.0.0.0/16 \
--query Vpc.VpcId --output text)
$ echo "$vpc_id"
vpc-24de2d4d
$ create_vpc_output=$(aws ec2 create-vpc --cidr-block 10.0.0.0/16)
$ vpc_id=$(aws ec2 create-vpc --cidr-block 10.0.0.0/16 \
--query Vpc.VpcId --output text)
$ echo "$vpc_id"
vpc-24de2d4d
$ create_vpc_output=$(aws ec2 create-vpc --cidr-block 10.0.0.0/16)
$ vpc_id=$(jp -u Vpc.VpcId <<< "$create_vpc_output“)
$ vpc_id=$(aws ec2 create-vpc --cidr-block 10.0.0.0/16 \
--query Vpc.VpcId --output text)
$ echo "$vpc_id"
vpc-24de2d4d
$ create_vpc_output=$(aws ec2 create-vpc --cidr-block 10.0.0.0/16)
$ vpc_id=$(jp -u Vpc.VpcId <<< "$create_vpc_output")
$ echo "$vpc_id"
vpc-24de2d4d
$ vpc_id=$(aws ec2 create-vpc --cidr-block 10.0.0.0/16 \
--query Vpc.VpcId --output text)
$ echo "$vpc_id"
vpc-24de2d4d
$ create_vpc_output=$(aws ec2 create-vpc --cidr-block 10.0.0.0/16)
$ vpc_id=$(jp -u Vpc.VpcId <<< "$create_vpc_output")
$ echo "$vpc_id"
vpc-24de2d4d
And we can make this even better!
Problem Statement
Demo
$
Let’s talk about how
$ aws ec2 create-vpc --cidr-block 10.0.0.0/16 --generate-cli-skeleton output
{
"Vpc": {
"VpcId": "VpcId",
"State": "State",
"CidrBlock": "CidrBlock",
"DhcpOptionsId": "DhcpOptionsId",
"Tags": [
{
"Key": "Key",
"Value": "Value"
}
],
"InstanceTenancy": "InstanceTenancy",
"IsDefault": true
}
}
$ aws ec2 create-vpc --cidr-block 10.0.0.0/16 --generate-cli-skeleton output
{
"Vpc": {
"VpcId": "VpcId",
"State": "State",
"CidrBlock": "CidrBlock",
"DhcpOptionsId": "DhcpOptionsId",
"Tags": [
{
"Key": "Key",
"Value": "Value"
}
],
"InstanceTenancy": "InstanceTenancy",
"IsDefault": true
}
}
"Vpc":{"type":"structure","members":{
"VpcId":{"shape":"String"
},"State":{"shape":"VpcState"
},"CidrBlock":{"shape":"String"
},"DhcpOptionsId":{"shape":"String"
},"Tags":{"shape":"TagList"
},"InstanceTenancy":{"shape":"Tenancy"
},"IsDefault":{"shape":"Boolean"
}}
} botocore/botocore/data/ec2/2016-09-15/service-2.json
$ aws ec2 create-vpc --cidr-block 10.0.0.0/16 --generate-cli-skeleton output
{
"Vpc": {
"VpcId": "VpcId",
"State": "State",
"CidrBlock": "CidrBlock",
"DhcpOptionsId": "DhcpOptionsId",
"Tags": [
{
"Key": "Key",
"Value": "Value"
}
],
"InstanceTenancy": "InstanceTenancy",
"IsDefault": true
}
}
"Vpc":{"type":"structure","members":{
"VpcId":{"shape":"String"
},"State":{"shape":"VpcState"
},"CidrBlock":{"shape":"String"
},"DhcpOptionsId":{"shape":"String"
},"Tags":{"shape":"TagList"
},"InstanceTenancy":{"shape":"Tenancy"
},"IsDefault":{"shape":"Boolean"
}}
} botocore/botocore/data/ec2/2016-09-15/service-2.json
$ aws ec2 create-vpc --cidr-block 10.0.0.0/16 --generate-cli-skeleton output
{
"Vpc": {
"VpcId": "VpcId",
"State": "State",
"CidrBlock": "CidrBlock",
"DhcpOptionsId": "DhcpOptionsId",
"Tags": [
{
"Key": "Key",
"Value": "Value"
}
],
"InstanceTenancy": "InstanceTenancy",
"IsDefault": true
}
}
"Vpc":{"type":"structure","members":{
"VpcId":{"shape":"String"
},"State":{"shape":"VpcState"
},"CidrBlock":{"shape":"String"
},"DhcpOptionsId":{"shape":"String"
},"Tags":{"shape":"TagList"
},"InstanceTenancy":{"shape":"Tenancy"
},"IsDefault":{"shape":"Boolean"
}}
} botocore/botocore/data/ec2/2016-09-15/service-2.json
$ aws ec2 create-vpc --cidr-block 10.0.0.0/16 --generate-cli-skeleton output
{
"Vpc": {
"VpcId": "VpcId",
"State": "State",
"CidrBlock": "CidrBlock",
"DhcpOptionsId": "DhcpOptionsId",
"Tags": [
{
"Key": "Key",
"Value": "Value"
}
],
"InstanceTenancy": "InstanceTenancy",
"IsDefault": true
}
}
"Vpc":{"type":"structure","members":{
"VpcId":{"shape":"String"
},"State":{"shape":"VpcState"
},"CidrBlock":{"shape":"String"
},"DhcpOptionsId":{"shape":"String"
},"Tags":{"shape":"TagList"
},"InstanceTenancy":{"shape":"Tenancy"
},"IsDefault":{"shape":"Boolean"
}}
} botocore/botocore/data/ec2/2016-09-15/service-2.json
AWS Cloud
VPC
10.0.0.0/16
Destination Target
10.0.0.0/16 local
router
VPC
10.0.0.0/16
Subnet
10.0.0.0/24
AWS Cloud
Internet
gateway
router
Destination Target
10.0.0.0/16 local
0.0.0.0/0 igw-id
Demo
$
Important Takeaways
• --generate-cli-skeleton output
• Leverage UNIX commands (e.g.
echo, history)
The Effective AWS CLI User Tenets
The effective AWS CLI user:
Troubleshoots well
$ aws ec2 create-security-group --group-name re:Invent \
--vpc-id VpcId
$ aws ec2 create-security-group --group-name re:Invent \
--vpc-id VpcId
aws: error: argument --description is required
$ aws ec2 create-security-group --group-name re:Invent \
--vpc-id VpcId
aws: error: argument --description is required
$ aws ec2 create-security-group --group-name re:Invent \
--description 're:Invent demo' --vpc-id VpcId
$ aws ec2 create-security-group --group-name re:Invent \
--vpc-id VpcId
aws: error: argument --description is required
$ aws ec2 create-security-group --group-name re:Invent \
--description 're:Invent demo' --vpc-id VpcId
An error occurred (InvalidVpcID.NotFound) when calling the CreateSecurityGroup operation: The vpc ID 'VpcId' does not exist
--debug
$ aws ec2 create-security-group --group-name re:Invent \
--description 're:Invent demo' --vpc-id "$vpc_id” --debug
$ aws ec2 create-security-group --group-name re:Invent \
--description 're:Invent demo' --vpc-id "$vpc_id” --debug
$ aws ec2 create-security-group --group-name re:Invent \
--description 're:Invent demo' --vpc-id "$vpc_id" -–debug
2016-11-04 10:30:33,532 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/1.11.10 Python/2.7.10 Darwin/15.6.0 botocore/1.4.65
2016-11-04 10:30:33,533 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['ec2', 'create-security-group', '--group-name', 're:Invent', '--description', 're:Invent demo', '--vpc-id', 'vpc-43cb382a', '--debug']
2016-11-04 10:30:33,533 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_scalar_parsers at 0x10eefc488>
2016-11-04 10:30:33,533 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_assume_role_provider_cache at 0x10ecde578>
2016-11-04 10:30:33,533 - MainThread - botocore.credentials - DEBUG - Skipping environment variable credential check because profile name was explicitly set.
2016-11-04 10:30:33,537 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /Users/kyleknap/GitHub/botocore/botocore/data/ec2/2016-09-15/service-2.json
2016-11-04 10:30:33,622 - MainThread - botocore.hooks - DEBUG - Event service-data-loaded.ec2: calling handler <function register_retries_for_service at 0x10e904500>
2016-11-04 10:30:33,623 - MainThread - botocore.handlers - DEBUG - Registering retry handlers for service: ec2
2016-11-04 10:30:33,628 - MainThread - botocore.hooks - DEBUG - Event building-command-table.ec2: calling handler <functools.partial object at 0x10ef02940>
2016-11-04 10:30:33,628 - MainThread - awscli.customizations.removals - DEBUG -Removing operation: import-instance
2016-11-04 10:30:33,628 - MainThread - awscli.customizations.removals - DEBUG -Removing operation: import-volume
2016-11-04 10:30:33,628 - MainThread - botocore.hooks - DEBUG - Event building-command-table.ec2: calling handler <function add_waiters at 0x10ef03578>
2016-11-04 10:30:33,631 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /Users/kyleknap/GitHub/botocore/botocore/data/ec2/2016-09-15/waiters-2.json
2016-11-04 10:30:33,635 - MainThread - awscli.clidriver - DEBUG - OrderedDict([(u'dry-run', <awscli.arguments.BooleanArgument object at 0x10febe950>), (u'no-dry-run', <awscli.arguments.BooleanArgument object at 0x10febe990>), (u'group-name', <awscli.arguments.CLIArgument object at 0x10febe9d0>), (u'description', <awscli.arguments.CLIArgument object at 0x10febea10>), (u'vpc-id', <awscli.arguments.CLIArgument object at 0x10febea50>)])
2016-11-04 10:30:33,623 - MainThread - botocore.handlers - DEBUG - Registering retry handlers for service: ec2
2016-11-04 10:30:33,628 - MainThread - botocore.hooks - DEBUG - Event building-command-table.ec2: calling handler <functools.partial object at 0x10ef02940>
2016-11-04 10:30:33,628 - MainThread - awscli.customizations.removals - DEBUG -Removing operation: import-instance
2016-11-04 10:30:33,628 - MainThread - awscli.customizations.removals - DEBUG -Removing operation: import-volume
2016-11-04 10:30:33,628 - MainThread - botocore.hooks - DEBUG - Event building-command-table.ec2: calling handler <function add_waiters at 0x10ef03578>
2016-11-04 10:30:33,631 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /Users/kyleknap/GitHub/botocore/botocore/data/ec2/2016-09-15/waiters-2.json
2016-11-04 10:30:33,635 - MainThread - awscli.clidriver - DEBUG - OrderedDict([(u'dry-run', <awscli.arguments.BooleanArgument object at 0x10febe950>), (u'no-dry-run', <awscli.arguments.BooleanArgument object at 0x10febe990>), (u'group-name', <awscli.arguments.CLIArgument object at 0x10febe9d0>), (u'description', <awscli.arguments.CLIArgument object at 0x10febea10>), (u'vpc-id', <awscli.arguments.CLIArgument object at 0x10febea50>)])
And this continues on for a while…
Parse command$ aws ec2 create-security-group \
--group-name re:Invent \--description 're:Invent demo' \--vpc-id vpc-24de2d4d \--query GroupId --output text
Parse command
botocore client call
# Python coderesponse = ec2_client.create_security_group(
GroupName=‘re:Invent’,Description=‘re:Invent demo’,VpcId=‘vpc-24de2d4d’
)
$ aws ec2 create-security-group \--group-name re:Invent \--description 're:Invent demo' \--vpc-id vpc-24de2d4d \--query GroupId --output text
Parse command
botocore client call
$ aws ec2 create-security-group \--group-name re:Invent \--description 're:Invent demo' \--vpc-id vpc-24de2d4d \--query GroupId --output text
# Python coderesponse = ec2_client.create_security_group(
GroupName=‘re:Invent’,Description=‘re:Invent demo’,VpcId=‘vpc-24de2d4d’
)
Parse command
botocore client call
$ aws ec2 create-security-group \--group-name re:Invent \--description 're:Invent demo' \--vpc-id vpc-24de2d4d \--query GroupId --output text
# Python coderesponse = ec2_client.create_security_group(
GroupName=‘re:Invent’,Description=‘re:Invent demo’,VpcId=‘vpc-24de2d4d’
)
Parse command
Format response
botocore client call
sg-1d032e64
$ aws ec2 create-security-group \--group-name re:Invent \--description 're:Invent demo' \--vpc-id vpc-24de2d4d \--query GroupId --output text
# Python coderesponse = ec2_client.create_security_group(
GroupName=‘re:Invent’,Description=‘re:Invent demo’,VpcId=‘vpc-24de2d4d’
)
Parse command
Format response
botocore client call
$ aws ec2 create-security-group \--group-name re:Invent \--description 're:Invent demo' \--vpc-id vpc-24de2d4d \--query GroupId --output text
# Python coderesponse = ec2_client.create_security_group(
GroupName=‘re:Invent’,Description=‘re:Invent demo’,VpcId=‘vpc-24de2d4d’
)
sg-1d032e64
Parse command
Format response
botocore client call
$ aws ec2 create-security-group \--group-name re:Invent \--description 're:Invent demo' \--vpc-id vpc-24de2d4d \--query GroupId --output text
# Python coderesponse = ec2_client.create_security_group(
GroupName=‘re:Invent’,Description=‘re:Invent demo’,VpcId=‘vpc-24de2d4d’
)
sg-1d032e64
Parse command
Format response
botocore client call
# Python coderesponse = ec2_client.create_security_group(
GroupName=‘re:Invent’,Description=‘re:Invent demo’,VpcId=‘vpc-24de2d4d’
)
Parse command
Format response
botocore client call
Validate/Serialize parameters
"CreateSecurityGroupRequest”{"type":"structure”,"required":["GroupName", "Description”],"members":{
"DryRun":{"shape":"Boolean”
},"GroupName":{
"shape":"String”},"Description":{
"shape":"String”},"VpcId":{
"shape":"String”}
}}
botocore/botocore/data/ec2/2016-09-15/service-2.json
Parse command
Format response
Make botocore client call
Validate/Serialize parameters
HTTP request/response
Parse command
Format response
Make botocore client call
Validate/Serialize parameters
HTTP request/response
Parse response
"CreateSecurityGroupResult”:{ "type":"structure”,"members":{
"GroupId":{"shape":"String”
}}
}botocore/botocore/data/ec2/2016-09-15/service-2.json
Parse command
Format response
Validate/Serialize parameters
HTTP request/response
Parse response
--debug
Parse command
Format response
Validate/Serialize parameters
HTTP request/response
Parse response
$ aws ec2 create-security-group \--group-name re:Invent \--description 're:Invent demo’ \--vpc-id "$vpc_id" -–debug
Parse command
Format response
Validate/Serialize parameters
HTTP request/response
Parse response
awscli.clidriver - DEBUG - Arguments entered to CLI: ['ec2', 'create-security-group', '--group-name', 're:Invent’,'--description', 're:Invent demo’,'--vpc-id', 'vpc-43cb382a', '--debug']
$ aws ec2 create-security-group \--group-name re:Invent \--description 're:Invent demo' \--vpc-id "$vpc_id" -–debug
Parse command
Format response
Validate/Serialize parameters
HTTP request/response
Parse response
awscli.clidriver - DEBUG - Arguments entered to CLI: ['ec2', 'create-security-group', '--group-name', 're:Invent’,'--description', 're:Invent demo’,'--vpc-id', 'vpc-43cb382a', '--debug']
$ aws ec2 create-security-group \--group-name re:Invent \--description 're:Invent demo' \--vpc-id "$vpc_id" -–debug
Parse command
Format response
Validate/Serialize parameters
HTTP request/response
Parse response
awscli.clidriver - DEBUG - Arguments entered to CLI: ['ec2', 'create-security-group', '--group-name', 're:Invent’,'--description', 're:Invent demo’,'--vpc-id', 'vpc-43cb382a', '--debug']
$ aws ec2 create-security-group \--group-name re:Invent \--description 're:Invent demo' \--vpc-id "$vpc_id" -–debug
Parse command
Format response
Validate/Serialize parameters
HTTP request/response
Parse response
$ aws ec2 create-security-group \--group-name re:Invent \--description file://description.txt \--vpc-id "$vpc_id" -–debug
Parse command
Format response
Validate/Serialize parameters
HTTP request/response
Parse response
$ aws ec2 create-security-group \--group-name re:Invent \--description file://description.txt \--vpc-id "$vpc_id" -–debug
Parse command
Format response
Validate/Serialize parameters
HTTP request/response
Parse response
$ aws ec2 create-security-group \--group-name re:Invent \--description file://description.txt \--vpc-id "$vpc_id" -–debug
awscli.clidriver - DEBUG - Arguments entered to CLI: ['ec2', 'create-security-group', '--group-name', 're:Invent’,'--description', 'file://description.txt’,'--vpc-id', 'vpc-43cb382a', '--debug']
Parse command
Format response
Validate/Serialize parameters
HTTP request/response
Parse response
$ aws ec2 create-security-group \--group-name re:Invent \--description file://description.txt \--vpc-id "$vpc_id" -–debug
awscli.clidriver - DEBUG - Arguments entered to CLI: ['ec2', 'create-security-group', '--group-name', 're:Invent’,'--description', 'file://description.txt’,'--vpc-id', 'vpc-43cb382a', '--debug']
Parse command
Format response
Validate/Serialize parameters
HTTP request/response
Parse responseawscli.arguments - DEBUG - Unpacked value of u're:Invent demo' for parameter "description": u're:Invent demo'
$ aws ec2 create-security-group \--group-name re:Invent \--description file://description.txt \--vpc-id "$vpc_id" -–debug
awscli.clidriver - DEBUG - Arguments entered to CLI: ['ec2', 'create-security-group', '--group-name', 're:Invent’,'--description', 'file://description.txt’,'--vpc-id', 'vpc-43cb382a', '--debug']
Parse command
Format response
Validate/Serialize parameters
HTTP request/response
Parse responseawscli.arguments - DEBUG - Unpacked value of u're:Invent demo' for parameter "description": u're:Invent demo'
$ aws ec2 create-security-group \--group-name re:Invent \--description file://description.txt \--vpc-id "$vpc_id" -–debug
awscli.clidriver - DEBUG - Arguments entered to CLI: ['ec2', 'create-security-group', '--group-name', 're:Invent’,'--description', 'file://description.txt’,'--vpc-id', 'vpc-43cb382a', '--debug']
Parse command
Format response
Validate/Serialize parameters
HTTP request/response
Parse response
$ aws ec2 create-security-group \--group-name re:Invent \--description 're:Invent demo' \--vpc-id "$vpc_id" -–debug
Parse command
Format response
Validate/Serialize parameters
HTTP request/response
Parse response
$ aws ec2 create-security-group \--group-name re:Invent \--description 're:Invent demo' \--vpc-id "$vpc_id" -–debug
botocore.endpoint - DEBUG - Making request for OperationModel(name=CreateSecurityGroup) (verify_ssl=True) with params: {
'body': {'Action': 'CreateSecurityGroup','GroupName': 're:Invent’,'Version': '2016-09-15’,'VpcId': 'vpc-43cb382a', ‘GroupDescription': 're:Invent demo’},
‘url':'https://ec2.us-east-2.amazonaws.com/’,...}
Parse command
Format response
Validate/Serialize parameters
HTTP request/response
Parse response
$ aws ec2 create-security-group \--group-name re:Invent \--description 're:Invent demo' \--vpc-id "$vpc_id" -–debug
botocore.endpoint - DEBUG - Making request for OperationModel(name=CreateSecurityGroup) (verify_ssl=True) with params: {
'body': {'Action': 'CreateSecurityGroup','GroupName': 're:Invent’,'Version': '2016-09-15’,'VpcId': 'vpc-43cb382a', ‘GroupDescription': 're:Invent demo’},
‘url':'https://ec2.us-east-2.amazonaws.com/’,...}
Parse command
Format response
Validate/Serialize parameters
HTTP request/response
Parse response
$ aws ec2 create-security-group \--group-name re:Invent \--description 're:Invent demo' \--vpc-id "$vpc_id" -–debug
botocore.endpoint - DEBUG - Making request for OperationModel(name=CreateSecurityGroup) (verify_ssl=True) with params: {
'body': {'Action': 'CreateSecurityGroup','GroupName': 're:Invent’,'Version': '2016-09-15’,'VpcId': 'vpc-43cb382a', ‘GroupDescription': 're:Invent demo’},
‘url':'https://ec2.us-east-2.amazonaws.com/’,...}
Parse command
Format response
Validate/Serialize parameters
HTTP request/response
Parse response
$ aws ec2 create-security-group \--group-name re:Invent \--description 're:Invent demo' \--vpc-id "$vpc_id" -–debug
botocore.endpoint - DEBUG - Making request for OperationModel(name=CreateSecurityGroup) (verify_ssl=True) with params: {
'body': {'Action': 'CreateSecurityGroup','GroupName': 're:Invent’,'Version': '2016-09-15’,'VpcId': 'vpc-43cb382a', ‘GroupDescription': 're:Invent demo’},
‘url':'https://ec2.us-east-2.amazonaws.com/’,...}
Parse command
Format response
Validate/Serialize parameters
HTTP request/response
Parse response
$ aws ec2 create-security-group \--group-name re:Invent \--description 're:Invent demo' \--vpc-id "$vpc_id" -–debug
botocore.vendored.requests.packages.urllib3.connectionpool - DEBUG - "POST / HTTP/1.1" 200 None
Parse command
Format response
Validate/Serialize parameters
HTTP request/response
Parse response
$ aws ec2 create-security-group \--group-name re:Invent \--description 're:Invent demo' \--vpc-id "$vpc_id" -–debug
botocore.vendored.requests.packages.urllib3.connectionpool - DEBUG - "POST / HTTP/1.1" 200 None
Parse command
Format response
Validate/Serialize parameters
HTTP request/response
Parse response
$ aws ec2 create-security-group \--group-name re:Invent \--description 're:Invent demo' \--vpc-id "$vpc_id" -–debug
botocore.parsers - DEBUG - Response body:<?xml version="1.0" encoding="UTF-8"?><CreateSecurityGroupResponse>
<requestId>request-id</requestId><return>true</return><groupId>sg-5ec80f37</groupId>
</CreateSecurityGroupResponse>
Parse command
Format response
Validate/Serialize parameters
HTTP request/response
Parse response
$ aws ec2 create-security-group \--group-name re:Invent \--description 're:Invent demo' \--vpc-id "$vpc_id" -–debug
botocore.parsers - DEBUG - Response body:<?xml version="1.0" encoding="UTF-8"?><CreateSecurityGroupResponse>
<requestId>request-id</requestId><return>true</return><groupId>sg-5ec80f37</groupId>
</CreateSecurityGroupResponse>
Parse command
Format response
Validate/Serialize parameters
HTTP request/response
Parse response
$ aws ec2 create-security-group \--group-name re:Invent \--description 're:Invent demo' \--vpc-id "$vpc_id" -–debug
botocore.parsers - DEBUG - Response body:<?xml version="1.0" encoding="UTF-8"?><CreateSecurityGroupResponse>
<requestId>request-id</requestId><return>true</return><groupId>sg-5ec80f37</groupId>
</CreateSecurityGroupResponse>
Parse command
Format response
Validate/Serialize parameters
HTTP request/response
Parse response
$ aws ec2 create-security-group \--group-name re:Invent \--description 're:Invent demo' \--vpc-id "$vpc_id" -–debug
botocore.retryhandler - DEBUG - No retry needed.
botocore.parsers - DEBUG - Response body:<?xml version="1.0" encoding="UTF-8"?><CreateSecurityGroupResponse>
<requestId>request-id</requestId><return>true</return><groupId>sg-5ec80f37</groupId>
</CreateSecurityGroupResponse>
Parse command
Format response
Validate/Serialize parameters
HTTP request/response
Parse response
$ aws ec2 create-security-group \--group-name re:Invent \--description 're:Invent demo' \--vpc-id "$vpc_id" -–debug
botocore.retryhandler - DEBUG - No retry needed.
botocore.parsers - DEBUG - Response body:<?xml version="1.0" encoding="UTF-8"?><CreateSecurityGroupResponse>
<requestId>request-id</requestId><return>true</return><groupId>sg-5ec80f37</groupId>
</CreateSecurityGroupResponse>
VPC
10.0.0.0/16
Subnet
10.0.0.0/24
AWS Cloud
Internet
gateway
router
Destination Target
10.0.0.0/16 local
0.0.0.0/0 igw-id
VPC
10.0.0.0/16
Subnet
10.0.0.0/24
security group
AWS Cloud
Internet
gateway
router
Destination Target
10.0.0.0/16 local
0.0.0.0/0 igw-id
Demo
$
Important Takeaways
• --debug
• Search for Traceback’s
• Leverage knowledge of AWS CLI
architecture
The Effective AWS CLI User Tenets
The effective AWS CLI user:
Is resourceful with tooling
• BASH: variables, pipes, functions
• BASH: variables, pipes, functions
• UNIX commands: xargs, sort, tail
• BASH: variables, pipes, functions
• UNIX commands: xargs, sort, tail
• External tools: jp, jpterm, aws-shell
• BASH: variables, pipes, functions
• UNIX commands: xargs, sort, tail
• External tools: jp, jpterm, aws-shell
• DIY (Do it yourself) tooling
alias
[alias]st = statusci = commitbr = branchco = checkout
~/.gitconfig
[alias]st = statusci = commitbr = branchco = checkout
~/.gitconfig
$ git co some-branch
[alias]st = statusci = commitbr = branchco = checkout
~/.gitconfig
$ git co some-branch
[alias]st = statusci = commitbr = branchco = checkout
~/.gitconfig
$ git co some-branch $ git checkout some-branch
VPC
10.0.0.0/16
Subnet
10.0.0.0/24
security group
AWS Cloud
Internet
gateway
router
Destination Target
10.0.0.0/16 local
0.0.0.0/0 igw-id
VPC
10.0.0.0/16
Subnet
10.0.0.0/24
security group
AWS Cloud
instance
Internet
gateway
router
Destination Target
10.0.0.0/16 local
0.0.0.0/0 igw-id
role
Demo
$
Important Takeaways
• alias
• Leverage BASH and external
tooling
The Effective AWS CLI User Tenets
The effective AWS CLI user:
Understands performance implications
Performance Implications
Client-side vs. server-side filtering
Pagination
aws s3 cp
Performance Implications
Client-side vs. server-side filtering
$ aws ec2 describe-images \
--query 'Images[?starts_with(to_string(Name),`amzn-ami-hvm-`)]'
$ aws ec2 describe-images \
--query 'Images[?starts_with(to_string(Name),`amzn-ami-hvm-`)]'
$ aws ec2 describe-images \
--query 'Images[?starts_with(to_string(Name),`amzn-ami-hvm-`)]'
$ aws ec2 describe-images \
--query 'Images[?starts_with(to_string(Name),`amzn-ami-hvm-`)]'
$ aws ec2 describe-images \--filters Name=name,Values='amzn-ami-hvm-*' --query 'Images'
$ aws ec2 describe-images \
--query 'Images[?starts_with(to_string(Name),`amzn-ami-hvm-`)]'
$ aws ec2 describe-images \--filters Name=name,Values='amzn-ami-hvm-*' --query 'Images'
$ aws ec2 describe-images \
--query 'Images[?starts_with(to_string(Name),`amzn-ami-hvm-`)]'
$ aws ec2 describe-images \--filters Name=name,Values='amzn-ami-hvm-*' --query 'Images'
Performance Implications
Pagination
$ aws ec2 describe-snapshots
$ aws ec2 describe-snapshots
{"Snapshots”: [{"SnapshotId": "snap-0",...
},...
]} }
$ aws ec2 describe-snapshots
{"Snapshots”: [{"SnapshotId": "snap-0",...
},...
]} }
{"Snapshots”: [{"SnapshotId": "snap-1000",...
},...
]} }
$ aws ec2 describe-snapshots
{"Snapshots”: [{"SnapshotId": "snap-0",...
},...
]} }
{"Snapshots”: [{"SnapshotId": "snap-1000",...
},...
]} }
{"Snapshots”: [{"SnapshotId": "snap-2000",...
},...
]} }
$ aws ec2 describe-snapshots
{"Snapshots”: [{"SnapshotId": "snap-0",...
},...
]} }
{"Snapshots”: [{"SnapshotId": "snap-1000",...
},...
]} }
{"Snapshots”: [{"SnapshotId": "snap-2000",...
},...
]} }
{"Snapshots”: [{"SnapshotId": "snap-0",...
},...{"SnapshotId": "snap-1000",...
},...{"SnapshotId": "snap-2000",...
},...
]} }
$ aws ec2 describe-snapshots --output text --query 'Snapshots[].SnapshotId'
{"Snapshots”: [{"SnapshotId": "snap-0",...
},...
]} }
{"Snapshots”: [{"SnapshotId": "snap-1000",...
},...
]} }
{"Snapshots”: [{"SnapshotId": "snap-2000",...
},...
]} }
$ aws ec2 describe-snapshots --output text --query 'Snapshots[].SnapshotId'
{"Snapshots”: [{"SnapshotId": "snap-0",...
},...
]} }
{"Snapshots”: [{"SnapshotId": "snap-1000",...
},...
]} }
{"Snapshots”: [{"SnapshotId": "snap-2000",...
},...
]} }
{"Snapshots”: [{"SnapshotId": "snap-0",...
},...
]} }
{"Snapshots”: [{"SnapshotId": "snap-1000",...
},...
]} }
{"Snapshots”: [{"SnapshotId": "snap-2000",...
},...
]} }
$ aws ec2 describe-snapshots --output text --query 'Snapshots[].SnapshotId'
snap-0...
snap-1000...
snap-2000...}
Performance Implications
aws s3 cp
upload
Thread pool
Thread
1
Thread
2
Thread
3
upload
Thread pool
6 5 4 3 2 1
Thread
1
Thread
2
Thread
3
Parts
upload
Thread pool
6 5 4
3
2
1Thread
1
Thread
2
Thread
3
Parts
upload
Thread pool
6 5
3
4
1Thread
1
Thread
2
Thread
3
Parts
upload
Thread pool
6
5
4
1Thread
1
Thread
2
Thread
3
Parts
upload
Thread pool
5
4
6Thread
1
Thread
2
Thread
3
upload
Thread pool
Thread
1
Thread
2
Thread
3
[default]region = us-east-1
~/.aws/config
[default]region = us-east-1s3 =
max_concurrent_requests = 20multipart_chunksize = 16MBmultipart_threshold = 64MBmax_queue_size = 10000
~/.aws/config
[default]region = us-east-1s3 =
max_concurrent_requests = 20multipart_chunksize = 16MBmultipart_threshold = 64MBmax_queue_size = 10000
~/.aws/config
max_concurrent_requests
Thread pool
6 5 4 3 2 1
Thread
1
Thread
2
Thread
3
Parts
VPC
10.0.0.0/16
Subnet
10.0.0.0/24
security group
AWS Cloud
instance
Internet
gateway
router
Destination Target
10.0.0.0/16 local
0.0.0.0/0 igw-id
role
VPC
10.0.0.0/16
Subnet
10.0.0.0/24
security group
AWS cloud
instance
Internet
gateway
router
Amazon
S3
Destination Target
10.0.0.0/16 local
0.0.0.0/0 igw-id
role
Demo
$
Important Takeaways
• Use server-side filtering when
possible
• Pagination with --output text
• s3 configuration options
The Effective AWS CLI User Tenets
The effective AWS CLI user:
The Effective AWS CLI User Tenets
The effective AWS CLI user:
Uses an iterative workflow
Example: --generate-cli-skeleton output
The Effective AWS CLI User Tenets
The effective AWS CLI user:
Troubleshoots well
Example: --debug
The Effective AWS CLI User Tenets
The effective AWS CLI user:
Is resourceful with tooling
Example: alias
The Effective AWS CLI User Tenets
The effective AWS CLI user:
Understands performance implications
Example: s3 configurations
Remember to complete
your evaluations!
Thank you!