©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved

Transparency and Auditing on AWS

Dave Walker – Specialised Solutions Architect, Security and ComplianceAmazon Web Services UK Ltd

28/01/16

The AWS Compliance “Display Cabinet”Certificates: Programmes:

ISO 27001Certified

ISO 9001CertifiedMPAA

Compliance: How to work with AWS Certifications• “The magic’s in the Scoping”

• If a Service isn’t in scope, that doesn’t necessarily mean it can’t be used in a compliant deployment• …but it won’t be usable for a purpose which touches sensitive data• See Re:Invent sessions, especially "Navigating PCI Compliance in the

Cloud”, https://www.youtube.com/watch?v=LUGe0lofYa0&index=13&list=PLhr1KZpdzukcJvl0e65MqqwycgpkCENmg

• Remember the Shared Responsibility Model• “we do our bit at AWS, but you must also do your bit in what you build

using our services”• Our audit reports make it easier for our customers to get approval

from their auditors, against the same standards• Liability can’t be outsourced…

Compliance: How to work with AWS Certifications• Time-based Subtleties:

• PCI, ISO: point-in-time assessments• SOC: assessment spread over time, therefore more rigorous assessment

of procedures and operations• (AWS Config allows you to make a path between these, for your own

auditors)• FedRAMP: Continuous Monitoring and Reporting – important proof

• If a service for defined sensitive data isn’t in scope of an audit report, can this be designed around?• Eg standing up a queue system on EC2 as a substitute for SQS…

• Be careful of what elements of a Service are in scope, too…• Metadata is typically “out”

SOC 1• Availability:

• Audit report available to any customer with an NDA• Scope:

• CloudFormation, CloudHSM, CloudTrail, DirectConnect, DynamoDB, EBS, EC2, Elastic Beanstalk, ELB, EMR, ElastiCache, Glacier, IAM, KMS, RDS, Redshift, Route 53, S3, SES, SimpleDB, SQS, Storage Gateway, SWF, VM Import / Export, VPC, Workspaces

• Sensitive data:• N/A

• Particularly good for:• Datacentre management, talks about KMS for key management and

encryption at rest, discusses Engineering bastions• Downsides:

• None

SOC 2• Availability:

• Audit report available to any customer with an NDA• Scope:

• CloudFormation, CloudHSM, CloudTrail, DirectConnect, DynamoDB, EBS, EC2, Elastic Beanstalk, ELB, EMR, ElastiCache, Glacier, IAM, KMS, RDS, Redshift, Route 53, S3, SES, SimpleDB, SQS, Storage Gateway, SWF, VM Import / Export, VPC, Workspaces

• Sensitive data:• N/A

• Particularly good for:• Risk assessment considerations, management visibility and process,

organisational structure• Downsides:

• None

PCI-DSS• Availability:

• Audit report available to any customer with an NDA• Scope:

• EC2, Auto-scaling, ELB, VPC, Route 53, Direct Connect, S3, Glacier, EBS, RDS, DynamoDB, SimpleDB, Redshift, EMR, SWF, IAM, CloudTrail, CloudHSM, SQS, CloudFront, CloudFormation, Elastic Beanstalk, KMS

• Sensitive data:• CVV, PAN

• Particularly good for:• Forensics cooperation, breach disclosure, explaining Shared

Responsibility in depth; also Hypervisor-based instance separation assurance

• Downsides:• None (since the August 2015 update, when KMS was added)

ISO 27001• Availability:

• Certificate is public at http://d0.awsstatic.com/certifications/iso_27001_global_certification.pdf, Statement of Applicability is normally not available externally

• Scope:• CloudFormation, CloudFront, CloudHSM, CloudTrail, Direct Connect, Directory Service,

DynamoDB, EBS, EC2, ECS, EFS, Elastic Beanstalk, ELB, EMR, ElastiCache, Glacier, IAM, KMS, RDS, Redshift, Route 53, S3, SES, SimpleDB, SQS, Storage Gateway, SWF, VM Import / Export, VPC, WAF, WorkDocs, WorkMail, Workspaces

• Sensitive data:• N/A

• Particularly good for:• A broad-ranging “backstop” and important “tick box item” – ISMS considerations (see

“Technical and Organisational Measures” later)• Downsides:

• No detailed audit report available

ISO 27018• Availability:

• Certificate available at https://d0.awsstatic.com/certifications/iso_27018_certification.pdf

• Scope:• CloudFormation, CloudFront, CloudHSM, CloudTrail, Direct Connect,

Directory Service, DynamoDB, EBS, EC2, ECS, EFS, Elastic Beanstalk, ELB, EMR, ElastiCache, Glacier, IAM, KMS, RDS, Redshift, Route 53, S3, SES, SimpleDB, SQS, Storage Gateway, SWF, VM Import / Export, VPC, WAF, WorkDocs, WorkMail, Workspaces

• Sensitive data:• PII

• Particularly good for:• Assurance of protection of PII in AWS environments

• Downsides:• No detailed audit report available

Others (and Resources):• ISO 27017: Cloud security recommended practices• ISO 9001: Quality control• UK G-Cloud / CESG Security Principles, gov.uk “Cyber Essentials”:

• See me and our whitepaper at https://d0.awsstatic.com/whitepapers/compliance/AWS_CESG_UK_Cloud_Security_Principles.pdf

• IT-Grundschutz: Workbook at https://d0.awsstatic.com/whitepapers/compliance/AWS_IT_Grundschutz_TUV_Certification_Workbook.pdf

• MTCS, IRAP, …: “Other People’s Geos” – we can put you in touch with AWS Specialist Security and Compliance SAs there as needed, there are also some whitepapers.

• SEC OCIE Workbook: https://d0.awsstatic.com/whitepapers/compliance/AWS_SEC_Workbook.pdf

• EU Data Protection Guidance: https://d0.awsstatic.com/whitepapers/compliance/AWS_EU_Data_Protection_Whitepaper.pdf

Detailed Billing

• Billing Information logged Daily in S3• Also Visible in the Billing Console• Alarms can be set on Billing Info to Alert on

Unexpected Activity

Sample Records

ItemDescriptionUsageStartDate

UsageEndDate

UsageQuantity

CurrencyCode

CostBeforeTax

Credits

TaxAmount

TaxType

TotalCost

$0.000 per GB - regional data transfer under the monthly global free tier

01.04.14 00:00

30.04.14 23:59

0.00000675 USD 0.00 0.0

0.000000

None

0.000000

$0.05 per GB-month of provisioned storage - US West (Oregon)

01.04.14 00:00

30.04.14 23:59

1.126.666.554USD 0.56 0.0

0.000000

None

0.560000

First 1,000,000 Amazon SNS API Requests per month are free

01.04.14 00:00

30.04.14 23:5910.0 USD 0.00 0.0

0.000000

None

0.000000

First 1,000,000 Amazon SQS Requests per month are free

01.04.14 00:00

30.04.14 23:594153.0 USD 0.00 0.0

0.000000

None

0.000000

$0.00 per GB - EU (Ireland) data transfer from US West (Northern California)

01.04.14 00:00

30.04.14 23:59

0.00003292 USD 0.00 0.0

0.000000

None

0.000000

$0.000 per GB - data transfer out under the monthly global free tier

01.04.14 00:00

30.04.14 23:590.02311019USD 0.00 0.0

0.000000

None

0.000000

First 1,000,000 Amazon SNS API Requests per month are free

01.04.14 00:00

30.04.14 23:5988.0 USD 0.00 0.0

0.000000

None

0.000000

$0.000 per GB - data transfer out under the monthly global free tier

01.04.14 00:00

30.04.14 23:593.3E-7 USD 0.00 0.0

0.000000

None

0.000000

AWS CloudTrail

CloudTrail can help you achieve many tasks• Security analysis• Track changes to AWS resources, for example

VPC security groups and NACLs• Compliance – log and understand AWS API call

history• Prove that you did not:

• Use the wrong region

• Use services you don’t want

• Troubleshoot operational issues – quickly identify the most recent changes to your environment

AWS CloudTrail logs can be delivered cross-account

CloudTrail can help you achieve many tasks• Accounts can send their trails to a central

account• Central account can then do analytics• Central account can:

• Redistribute the trails

• Grant access to the trails

• Filter and reformat Trails (to meet privacy requirements)

AWS Config

AWS Config is a fully managed service that provides you with an inventory of your AWS resources, lets you audit the resource configuration history and notifies you of resource configuration changes.

©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved

Continuous ChangeRecordingChanging Resource

s

AWS ConfigHistory

Stream

Snapshot (ex. 2014-11-05)AWS Config

Resource

• A resource is an AWS object you can create, update or delete on AWS

• Examples include Amazon EC2 instances, Security Groups, Network ACLs, VPCs and subnets

Amazon EC2Instance, ENI...

Amazon EBSVolumes

AWS CloudTrailLog

Amazon VPCVPC, Subnet...

Resources

Resource Type Resource

Amazon EC2 EC2 InstanceEC2 Elastic IP (VPC only)EC2 Security GroupEC2 Network Interface

Amazon EBS EBS Volume

Amazon VPC VPCsNetwork ACLsRoute TableSubnetVPN ConnectionInternet GatewayCustomer GatewayVPN Gateway

AWS CloudTrail Trail

Relationships

• Bi-directional map of dependencies automatically assigned

• Change to a resource propagates to create Configuration Items for related resources

Example: Security Group sg-10dk8ej and EC2 instance i-123a3d9 are “associated with” each other

Relationships

Resource Relationship Related ResourceCustomerGateway is attached to VPN ConnectionElastic IP (EIP) is attached to Network Interface  is attached to InstanceInstance contains Network Interface  is attached to ElasticIP (EIP)  is contained in Route Table  is associated with Security Group  is contained in Subnet  is attached to Volume  is contained in Virtual Private Cloud (VPC)InternetGateway is attached to Virtual Private Cloud (VPC)… …. …..

Configuration ItemAll AWS API configuration attributes for a given resource at a given point in time, captured on every configuration change

Component Description Contains

Metadata Information about this configuration item

Version ID, Configuration item ID, Time when the configuration item was captured, State ID indicating the ordering of the configuration items of a resource, MD5Hash, etc.

Common Attributes Resource attributes Resource ID, tags, Resource type. Amazon Resource Name (ARN)Availability Zone, etc.

Relationships How the resource is related to other resources associated with the account

EBS volume vol-1234567 is attached to an EC2 instance i-a1b2c3d4

Current Configuration Information returned through a call to the Describe or List API of the resource

e.g. for EBS VolumeState of DeleteOnTermination flagType of volume. For example, gp2, io1, or standard

Related Events The AWS CloudTrail events that are related to the current configuration of the resource

AWS CloudTrail event ID

Configuration Item

Config Rules• Essentially, “Lambda Integration for Config”• See

https://aws.amazon.com/blogs/aws/aws-config-rules-dynamic-compliance-checking-for-cloud-resources/

• Apply detailed checks to the state of your configuration, at the point when it changes

• Raise alerts if anything is outside compliance with your defined policy• Eg if there’s unencrypted non-root EBS volumes• …or eg if any taggable resources aren’t tagged appropriately

• We have a small (currently) library of pre-built rules – or build your own• See also Re:Invent (SEC308) “Wrangling Security Events in the Cloud”

(https://www.youtube.com/watch?v=uc1Q0XCcCv4 )• Feature is in Preview right now – see

https://aws.amazon.com/config/preview/ and sign up!

New post-Re:Invent!

Full visibility of your AWS environment• CloudTrail will record access to API calls and save logs in your S3

buckets, no matter how those API calls were made

Who did what and when and from where (IP address)• CloudTrail support for many AWS services and growing - includes EC2,

EBS, VPC, RDS, IAM and RedShift• Easily Aggregate all instance log information – CloudWatch Logs agent

scrapes files from EC2 instances and sends them to S3• Also enables alerting with SNS on “strings of interest”, just like regular

CloudWatch• CloudWatch Logs used as delivery mechanism for Flow Logging

Out of the box integration with log analysis tools from AWS partners including Splunk, AlertLogic and SumoLogic

Monitoring: Get consistent visibility of logs

Elasticsearch, Kibana and CloudWatch Logs integration• Push CloudTrail to CloudWatch Logs:

http://docs.aws.amazon.com/awscloudtrail/latest/userguide/send-cloudtrail-events-to-cloudwatch-logs.html

• Push CloudWatch Logs to Elasticsearch: http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/CWL_ES_Stream.html

• Put a Kibana front-end on it: https://aws.amazon.com/blogs/aws/cloudwatch-logs-subscription-consumer-elasticsearch-kibana-dashboards/

Also new post-Re:Invent!

Firewall Requirements

• Based on NIST SP-800, PCI-DSS and others– Anti-Spoofing

– Packet-Filtering (minimum) stateful/stateless

– Segregation of Duties at the management side

– Logging/Audit capabilities on the management side

– Event-Logging on processed traffic

Security Group

IAM

AWS Config CloudTrail

FlowLogs

VPC Flow Logs

CloudWatchLogs

LogGroup

ENI-LogStream

ENI-LogStream

ENI-LogStream

ENI-LogStream

ENI-LogStream

ENI-LogStream

ENI-LogStream

VPC Flow Logs in Context

route restrictively

lock down on network level

isolate concerns

lock down on instance level

Flows

Flow Log Record Structure

Event-Version

Account Number

ENI-ID

Source-IP

Destination-IP

SourcePort

Destination-Port

Protocol Number

Number of Packets

Number of Bytes

Start-Time Window

End-Time Window

Action

State

2 123456789 eni-31607853 172.16.0.10 172.16.0.172 80 41707 6 1 40 1440402534 1440402589

ACCEPT OK

Flow Log Sampling

Flow Logs are STATISTICAL reports of activity over a window of time

Start-Time Window End-Time Window

Number of Packets Number of Bytes Action

Statistical Sampling and Spikes

Time

Src/Dst IP/Port Tuple

?

Example

How To Access

CloudWatchLogs

LogGroup

ENI-LogStream

ENI-LogStream

ENI-LogStream

ENI-LogStream

ENI-LogStream

ENI-LogStream

ENI-LogStream

Start

End

TIME

Logs→metrics→alerts→actions

AWS Config

CloudWatch / CloudWatch Logs

CloudWatch alarms

AWS CloudTrail

Amazon EC2 OS logs

Amazon VPC Flow Logs

Amazon SNS

email notification

HTTP/S notification

SMS notifications

Mobile push notifications

API calls from most

services Monitoring data from

AWS services

Custom metrics

Further Log Sources• ELB access logs

– Delivered to an S3 bucket

• CloudFront access logs– Delivered to an S3 bucket

• Redshift logs– Delivered to an S3 bucket

• RDS logs– Delivered to an S3 bucket or CloudWatch Logs

©2015, Amazon Web Services, Inc. or its affiliates. All rights reserved

Dave Walker – Specialised Solutions Architect Security/ComplianceAmazon Web Services UK Ltd

22/10/15

Thank You