Embed Size (px)
Citation preview
Industry Leading Platform For Machine Data
Machine Data: Any Location, Type, Volume
Online Services Web
Services
ServersSecurity GPS
Location
StorageDesktops
Networks
Packaged Applications
CustomApplicationsMessaging
TelecomsOnline
Shopping Cart
Web Clickstreams
Databases
Energy Meters
Call Detail Records
Smartphones and Devices
RFID
On-Premises
Private Cloud
Public Cloud
Platform Support (Apps / API / SDKs)
Enterprise Scalability
Universal Indexing
Answer Any Question
DeveloperPlatform
Report and
analyze
Custom dashboards
Monitor and alert
Ad hoc search
Add-on for AWS
Splunk App for AWSExplore Analyze Dashboard Alert
CloudTrail
ECS
EMR
VPC ELBCloudfront
Lambda
ConfigCloudWatch
S3
Kinesis
Splunk Insights for AWS Machine Data
EC2
RDS IoT
Inspector
Why Splunk for AWS?
Security Intelligence (Cloudtrail, Config Cloudwatch, Inspector, VPC)
Operational Intelligence (Cloudwatch, Config, RDS, ELB, EC2, S3, Cloudfront)
DevOps Intelligence (Cloudwatch, Lambda)
Big Data Insights (Kinesis, EMR, IoT, S3)
Service Billing & Usage
Billing &
Other
Services
Ingests Data From Heterogeneous Data Sources
perf
shellAPI
Mounted File Systems\\hostname\mount
syslogTCP/UDP
Active Directory
syslog hostsand network devices
Unix, Linux and Windows hosts
Local File MonitoringSplunk Forwarder
virtualhost
Windows
Scripted or Modular Inputsshell scripts
API subscriptions
Mainframes*nix
Wire DataSplunk App for Stream
HTTPHTTP Event Collector
Splunk App for AWS: The Data
5
• AWS Cloudtrail– Service that records AWS API calls for your account
and delivers activity logs – Provides data to enable security analysis, resource
change tracking, compliance auditing, and troubleshooting
• AWS Config & Config Rules– Service that provides resource inventory, configuration
history and configuration change notifications– Config Rules enables creation of rules to auto-check
AWS configurations – Provides data to enable resource discovery, service
relationships, change tracking & troubleshooting
• Amazon Cloudwatch– Service that collects AWS system metrics and log files– Offers ability to stream logs via Amazon Kinesis– Provides data to enable utilization & health reporting
for services such as EC2, EBS, & RDS
• Amazon Cloudwatch VPC Flow Logs– Service that enables capture of IP traffic information
to/from VPC network interfaces– Data stored and accessible from AWS Cloudwatch Logs– Provides data used to troubleshoot undesired traffic
behavior for both operational and security use cases
• Amazon Inspector– Automated security assessment service to help improve
security and compliance of apps on AWS– Provides data from knowledge base and security findings
based on security best practices
• AWS Access Logs– Elastic Load Balancing (ELB) – Provides data on load
balancer requests to anlayze traffic patterns– Cloudfront CDN – Provides data about every user request
received from Cloudfront– S3 – Provides data about a single access request and can be
used for security and access audits
• AWS Billing– Current Month via Cloudwatch metrics– Monthly Detailed Billing for Capacity Management
Getting Started!
6
• Create a Splunk account: https://www.splunk.com/page/sign_up
• Access Splunk AMIs on AWS Marketplace and then set-up Splunk App for AWS & AWS Technology Add-On
*or*Access Splunk Cloudformation template by following these directions. This environment will include the Splunk App for AWS and Splunk TA for AWS
• Be sure to take self-paced Using Splunk tutorial + Review Splunk>Docs and Splunk>Apps
• Automate your deployment:– Puppet: https://forge.puppet.com/tags/splunk– Chef: https://github.com/chef-cookbooks/chef-splunk
Splunk & Amazon Web Services
June 2016
7
Why is Splunk Important For AWS Customers?
8
“You can’t protect what you can’t see.”
Best Practices for Securing Workloads in Amazon Web Services
Gartner, April 2015
Neil MacDonald, Greg Young
“Security monitoring will make or break a technology risk management program.”
“Security requires visibility.”
Assessing the Risk: Yes, the Cloud Can Be More Secure Than Your On-Premises Environment
IDC, July 2015
Pete Lindstrom
Amazon Web Services
“Intro to AWS Security”
2015 AWS Summit Series
Extrapolating for Other AWS Use Cases…
9
“You can’t operate what you can’t see.”
“You can’t manage cost for what you can’t see.”
“You can’t gain business analytics for what you cant’ see.
Splunk’s AWS Credentials• AWS Advanced Technology Partner
• AWS Big Data Competency
• AWS Security Competency
• AWS Government Competency
• AWS SaaS Sales Alignment Program (Internal Program)
• AWS MSP Technology Provider
• AWS Marketplace Partner
• AWS IoT Launch partner for IoT analytics
• AWS Security by Design Program Partner
• 1st partner with published Blueprints for AWS Lambda
• 1st partner to pass SaaS extension for Well Architected framework
Splunk Portfolio of Cloud/AWS Solutions
11
Splunk Core + Enterprise Security & ITSI available
100% Uptime SLA
SOC2 Type II Certified
Starts at $90 / Month
App for AWS Support @ .conf16!
Cloud Apps & Solutions
AWS Specific Integrations
Splunk App for AWS, ServiceNow, SFDC, Box, more
AWS Lambda: First partner blueprint
AWS IoT: Featured analytics platform
AWS Kinesis: TA & Mod Input
AWS EC2 Container Service: Splunk Driver
Enterprise on AWS
AMIs & Cloudformation
Splunk Analytics for AWS Elastic MapReduce
(Hadoop/HDFS)
Hunk for AWS EMR
Available on AWS & Cloud
AMI for Splunk Enterprise
AMI for Splunk Light
AMI for Hunk
Cloudformation Templates
Add-on for AWS
Splunk App for AWSExplore Analyze Dashboard Alert
CloudTrail
ECS
EMR
VPC ELBCloudfront
Lambda
ConfigCloudWatch
S3
Kinesis
Splunk Insights for AWS Machine Data
EC2
RDS IoT
Inspector
Why Splunk for AWS?
Security Intelligence (Cloudtrail, Config Cloudwatch, Inspector, VPC)
Operational Intelligence (Cloudwatch, Config, RDS, ELB, EC2, S3, Cloudfront)
DevOps Intelligence (Cloudwatch, Lambda)
Big Data Insights (Kinesis, EMR, IoT, S3)
Service Billing & Usage
Billing &
Other
Services
13
How FINRA Uses SplunkCloud for Security
“Splunk Cloud gives you applications
which let you get huge amounts of
value from your data.”
• Transforms third-party threat intelligence information into security alerts
• Leverages the Splunk App for AWS
• Extends solution to report on AWS Cost Optimization
— Sr. Director of Information Security
Better Code, Faster Development and Migration to Cloud
• Reduced error rates by 2 ordersof magnitude in a couple of weeks
• Rapidly found and fixed one line ofcode responsible for 30,000+ errors
• Real-time dashboards on errorrates and production impact
• In-depth visibility as they strategicallymigrate apps to AWS Cloud
14
Supporting Global Websites
• Real-time insight ensures an optimum customer experience, even during peak sales periods
• Proactive troubleshooting results in faster resolution of issues
• Real-time monitoring ensures confidence in the cloud
“When I look at the e-commerce chain from customer service, through to the warehouse and
even in the physical stores—there’s opportunity to drive value with Splunk everywhere.”
— E-Commerce Systems Architect, Kurt Geiger
Splunk App for AWSv4.2
Splunk App for AWS: The Data
17
• AWS Cloudtrail– Service that records AWS API calls for your account
and delivers activity logs – Provides data to enable security analysis, resource
change tracking, compliance auditing, and troubleshooting
• AWS Config & Config Rules– Service that provides resource inventory, configuration
history and configuration change notifications– Config Rules enables creation of rules to auto-check
AWS configurations – Provides data to enable resource discovery, service
relationships, change tracking & troubleshooting
• Amazon Cloudwatch– Service that collects AWS system metrics and log files– Offers ability to stream logs via Amazon Kinesis– Provides data to enable utilization & health reporting
for services such as EC2, EBS, & RDS
• Amazon Cloudwatch VPC Flow Logs– Service that enables capture of IP traffic information
to/from VPC network interfaces– Data stored and accessible from AWS Cloudwatch Logs– Provides data used to troubleshoot undesired traffic
behavior for both operational and security use cases
• Amazon Inspector– Automated security assessment service to help improve
security and compliance of apps on AWS– Provides data from knowledge base and security findings
based on security best practices
• AWS Access Logs– Elastic Load Balancing (ELB) – Provides data on load
balancer requests to anlayze traffic patterns– Cloudfront CDN – Provides data about every user request
received from Cloudfront– S3 – Provides data about a single access request and can be
used for security and access audits
• AWS Billing– Current Month via Cloudwatch metrics– Monthly Detailed Billing for Capacity Management
Splunk App for AWS: The Value
18
• Increase visibility into AWS resource utilization & user activity across all accounts
• Ensure adherence to security and compliance standards with a audit reporting
• Understand AWS environmental dependencies via interactive topology visualization
• Monitor VPC traffic utilization for additional patterns & security insights
• Cost Optimization through Monthly and Detailed Billing Dashboards
Overview for Splunk App for AWS
• The overview page shows you on one screen information about:• Configuration changes• Compute• Storage• Billing• ELB• Cloudfront• Security
• Notable CloudTrail Activity is highlighted on the map.
• Drill down on any event and gain detailed information.
AWS Topology
Topology view gives you a holistic view of your current or historical AWS deployment using AWS Config
Maps out relationships between all the components, giving you a clear view into the environment
Clickable layers adds additional visual queues for high CPU or network traffic
Snapshot feature allows for topology to be saved for future use
Config
AWS Topology - Expanded Visuals
CloudWatch
CloudTrail
Config
AWS Topology - IAM
IAM Topology view uses AWS Config to provide a comprehensive view of Identity and Access Management Information
Provides visual way to manage IAM Users, Groups and Policies
Select entity of interest to see IAM relationships
AWS Usage Overview
• In one glance, instantly see your EC2 usage and EBS Volume data info via Cloudwatch metrics
• Click through dashboards for details on individual EC2 instancesand EBS Volumes
• Drill down into raw search for even more detailed views on your instances.
VPC Flow Data - Traffic
• Utilizes VPC Flow Logs from Cloudwatch for Traffic Analysis
• Visualize VPC traffic by interface, time, and location
VPC Flow Data - Security
• Utilizes VPC Flow Logs from Cloudwatch for Security Analysis
• Drill down into rejected vs. accepted traffic
• View top Source Country and City information
• See top source / destination and IP Addresses and ports
AWS Billing & Capacity Planning
Utilizes Billing Logs from Cloudwatch for Month-to-Date billing and End-of-Month projections
Detailed Historical Billing Dashboard available using Monthly AWS Detailed billing reports
Capacity Planner gives additional clarity on AWS On-Demand instance spending
AWS S3 Access
S3 Access logs provide visibility on the health, requests, and traffic volume handled by your S3 bucket objects across all accounts.
Aggregations by requester, user-agent, and error codes give insights for troubleshooting, security and general product/business analytics.
AWS Elastic Load Balancer
ELB dashboards provide visibility on the health, latency and request volume of your load balancers
Client and server side errors are surfaced (HTTP 4XX-5XX errors) by account and region
AWS Cloudfront CDN
Cloudfront Dashboards displays visitor information per edge location, referrers, cache hits/misses and traffic volume
Provides operational utility by adding visibility to errors, latency, distribution
Provides business insights such as geo location of visitors, user agents and referrers.
AWS User & IAM Activity
• Utilizes Cloudtrail data to quickly see the number of active users logged into the system
• Get alerted on Unauthorized user activities and create additional alerts for any user action
• See what ARN’s are being used to access services and the correlated functions
AWS Key Pairs Activity
Utilizes Cloudtrail data to quickly see number of In-Use Key Pairs, Error events and actions
Reports on Key Pair usage by Region and activity over time
Getting Started!
32
• Create a Splunk account: https://www.splunk.com/page/sign_up
• Access Splunk AMIs on AWS Marketplace: https://aws.amazon.com/marketplace/search/results/ref=lbr_navgno_search_box?page=1&searchTerms=splunk and then set-up the Splunk App for AWS & AWS Technology Add-On
*or*Access Splunk Cloudformation template by following these directions. This environment will include the Splunk App for AWS and Splunk TA for AWS
• Be sure to take self-paced Using Splunk tutorial + Review Splunk>Docs and Splunk>Apps
AWS Extras
34
• Utilizes new Splunk HTTP Event Collector
• Enabling Developers by Monitoring Lambda functions
• Use Lambda to pipe events from services like Kinesis to Splunk
• Configure in the AWS Console or use our JavaScript and Java logging libraries
Splunk & AWS Lambda
http://dev.splunk.com/goto/awslambda
Splunk & AWS IoT• Visibility into
Connected Devices communicating with Cloud Apps
• Enables advanced analytics & insights for IoTdeployments
36
Hunk & AWS Elastic Map Reduce (EMR)
• Gain insights - Explore, analyze, and visualize Amazon EMR and Amazon S3 data at massive scale
• Unlock the business value of stored data Preview search results before MapReduce
jobs finish Quickly conduct sophisticated analytics
• Easily provision Hunk from AWS EMR Console Use for only as long as you need it Charged by the hour
Splunk Enterprise on AWS Deployment Guidelines
•Search Heads (8+ users)
•c4.4xlarge 16 vCPU, 30 GB RAM
•c4.8xlarge 36 vCPU, 60 GB RAM
•Indexers (50-250GB/day/indexer)
•c4.4xlarge 16 vCPU, 30 GB RAM
•d2.4xlarge 16 vCPU, 122 GB RAM
•c4.8xlarge 36 vCPU, 60 GB RAM
CloudFormation Templates
•Consistent, repeatable deployments for Splunk on AWS
•Abstract away details of configuring distributed Splunk
•Extensible and customizable to fit any need
CF Templates On GitHub
Workload = Searching + Indexing
Storage- Ephemeral or EBS
- Data Retention Dependent
Compute- Best Available
Archiving- S3
Best Practices for Sizing
Splunk on AWS Tech Brief
Splunk Cloudformation Templates
Splunk Admin Docs
• 37
