Embed Size (px)
Citation preview
What’s Up With That Airplane?
Visualizing DoD Knowledge Using Splunk Dashboards
Ken Mattern
� Ken Mattern � Senior Systems Analyst
� Data Miner
� Aranea Solutions, Inc. � Huntsville, Alabama � Department of Defense System Integrator
� Operating Environment � Splunk � Liferay Portal � Custom Built Software to Manage Data Movement
10/2/14 2
� Introduction of Terms, Data Types, and our Customers
� History and Goals of the Project
� Development of Splunk Dashboards
� Simplifying Things with Liferay
� Code Samples and Other Resources
10/2/14 3
Today's Agenda
Mid Tier 1
A Co.
B Co.
C Co.
A Co.
B Co.
C Co.
A Co. C Co.
Mid Tier 3
Mid Tier 2
Splunk Dashboards
FLEET Top Tier • Automated push/pull
FLEET Structure
B Co
What Kind of Data � Aircraft Data
� Bus data (Black Box)
� Health and usage � Vibration analysis
� Engine metrics
� Safety of flight messages
� Technical documents and media
� Maintenance data
� SQL Server databases � Derived data files
� Data movement logs
� IIS � Custom Windows event logs
� Liferay portal
10/2/14 5
Who is the Customer � Aircraft program management offices
� Subject Matter Experts (SMEs)
� Maintenance officers � Webmasters
� Data customers � Aircraft manufacturers
� Logistics and support agencies
10/2/14 6
The Challenge � Aggregate terabytes of disparate data from remote, tactical locations
� Transform data into actionable information
The Solution – Splunk � Provide management consoles for operational decision making
� Close to real-time metrics
� Granular Situational Awareness
� To unit level
� Historical Data for Trend Analysis
� Enable visibility of portal site user interaction
10/2/14 7
What You Will See � Demonstration version of our DoD operations,
Maintenance, and management portal – FLEET � Sanitized data
� Decommissioned weapon system � F-14D Tomcat
� Approved for public release
10/2/14 8
10/2/14 9
Where We Started
Where We Are Now � Sun Java System Portal Server 7.2
� Transitioning this month to Liferay
� Windows Server & IIS data transfer platform � Custom developed data transfer software
� F5 LDAP SSO authentication � Common access card
� Direct link to Splunk dashboards
10/2/14 10
Where We Are Now � Splunk platform
� Version 6.1.3 � Sideview Utils
� Nick Mealy � Whoami
� Brian Wooden � Google Maps � Customized Cascading Style Sheets � Splunk Database Connector (DBX) � Splunk SDK for Java and PHP
10/2/14 11
10/2/14 12
10/2/14 13
Typical Stats Dashboard
10/2/14 14
10/2/14 15
Generating Version Info The Search <param name="search"> | inputlookup VersionInfo.csv | where view=“F14" | fields view,version,app | eval version=if(view="F14",version,"") | eval app=if(view="F14",app,"") | table version, app </param> The HTML <table width=650><tr> <td><div class="style1"><img src="/static/app/common/images/FleetLogo.png"></div></td> <td><div class="style0">Version: $results[0].version$ </div></td> </tr></table>
Where We Are Going � Liferay web portal
� Utilizing Liferay portlets
� Splunk access through Java SDK � Splunk data displayed in a single portlet
10/2/14 16
Stats with FLEET Top Tier
10/2/14 17
Stats Portlet
10/2/14 18
Liferay Advantage � One script versus 24 advanced XML dashboards
� Elimination of all Splunk users in LDAP and F5
� Elimination of all Splunk Groups
� One secure login for all
� Much lighter IA impact
� Less to maintain � Only five searches to worry about rather than over 100
� Which would you chose?
10/2/14 19
We Also Do… � Visualization of aggregated data into dashboards
that provide actionable information � Such as Mid-Tier Status
10/2/14 20
Mid-Tier Activity and Data Movement
10/2/14 21
Mid-Tier Activity and Data Movement
10/2/14 22
| table cs_username, path, Status, MidTierName, Account, "Download MBs", "Upload MBs"
OK.png Monitor.png Contact.png
<module name="HTML" group="row.fields.Status"> <param name="html"><![CDATA[ <img src="/static/app/Mid-TierAndLogbookStats/images/$row.fields.Status$.png"> ]]></param> </module>
10/2/14 23
10/2/14 24
Internal User Access � Aranea personnel
� System Engineering � Developers
� Sustainment � Splunk administrator
� Fielding � Team that builds and deploys Mid-Tier networks
� Support
10/2/14 25
The Landing Zone
10/2/14 26
Mid-Tier Activity and Data Movement
10/2/14 27
<param name="search"> | whoami fieldname=user | fields user | lookup UserDashboards.csv user output Dashboard, url | stats first(url) as url by Dashboard | table Dashboard, url </param> … <module name="Table" layoutPanel="panel_row2_col1_grp1"> <param name="hiddenFields">url</param> <param name="count">10</param> <module name="Redirector"> <param name="popup">True</param> <param name="url">$row.fields.url$</param> </module> </module>
Fleet Management
10/2/14 28
10/2/14 29
10/2/14 30
Aircraft Mission Status
10/2/14 31
10/2/14 32
10/2/14 33
10/2/14 34
10/2/14 35
10/2/14 36
10/2/14 37
10/2/14 38
10/2/14 39
What’s Up With That Airplane?
Visualizing DoD Knowledge Using Splunk Dashboards
Ken Mattern
For More Information Contact Aranea Solutions, Inc.
5030 Bradford Dr. Bldg. 1, Suite 210
Huntsville, AL 35805 256-430-0995
http://www.araneasolutions.com [email protected]
Code Samples
10/2/14 41
In the following pages the note sections include expanded code samples of information discussed in the presentation
Version Info Detail Slide 15
10/2/14 42
The notes section of this slide includes the XML, search and Sideview HTML module code used to generate the Version Info, Logout and Help page links.
Portlet PHP Prototype Detail Slide 18
10/2/14 43
The notes section of this slide includes a snippet of the PHP code used to generate the first Liferay Portlet. This sample shows the search for the display of the number of page views and distinct users
Displaying Status Icons Detail Slide 22
10/2/14 44
The notes section of this slide includes a snippet of the XML, search and Sideview HTML module code used to generate the Mid-Tier Status icons
Displaying Dashboards and Searches Detail
Slide 27
10/2/14 45
The notes section of this slide includes a snippet of the XML, search and Sideview HTML module code used to generate the list of favorite dashboards and saved searches for the user in the Landing Zone dashboard
