Upload
donga
View
220
Download
1
Embed Size (px)
Citation preview
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved.
TIBCO Spotfire Analytics for Energy
MICHAEL O’CONNELL, PhD
Sr Director, Analytics
TIBCO
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
• Introductions & Spotfire® Basics
• Production demonstration
• Predictive Analytics
• Upstream exploratory – multiple use cases
• ESRI Integration
• Portfolio optimization
• Forecasting (time series)
• Sales & marketing
• Health & safety
• Social and mobile
• Tibbr® and collaborative computing
• iPad
• Wrap up and questions
• Deployment approaches
Outline
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Automation Cloud Event
Processing Analytics Social
• Messaging
• SOA
• BPM
• MDM
• Events/CEP
• Rules
• In-memory
• Visual Analysis
• Data Discovery
• Statistics
• Private/hybrid
Cloud Platform
• Apps
• Deployment
Option
• Enterprise
Social Media
• Collaboration
Key Components of 21st Century Platform
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Dimension-Free
Data Exploration
Data Mashup
Predictive &
Event Driven
Contextual
Collaboration
Enterprise-
Class
• Visual • Interactive • No Constraints
• Combine Data Sources • No Scripting • IT-Free
• Data at Rest & Data In Motion
• Open Source & 3rd Party
• “Two Second Advantage”
• Bookmarks • Guided Apps • Portals & Social Platforms
• Unmatched Performance
• Massive Scalability
• 24x7 Expertise
Spotfire Areas of Focus
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Telco, Media, Networks • Customer life-cycle management: social churn • Mobile marketing & advertising • Fraud detection
Finance / Insurance • Customer insight • Risk management & Trading analytics • Fraud and Compliance
Consumer Goods & Retail • Category/brand – value/share analysis & forecasting • Market – product / price analytics, segmentation • Execution – promotion effectiveness
Predictive Analytics Data
TIBCO Industry Analytics
Energy • Exploration, drilling analytics; asset recovery prediction • Trading and financial analytics • Portfolio optimization
Attributes [Customer]
Context / Facts
Behavior [Real-time]
Analytic Solutions
Life Sciences • Research chemistry, biology • Clinical development • Post-marketing / Healthcare
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
• Supervised Learning
• Regression
• Linear / non-linear models - continuous response (Gaussian underlying errors)
• Generalized linear models – discrete response (e.g. Poisson, binomial errors)
• Trees, Nets, SVM, Additive Models, Discriminant Functions, Nearest Neighbors, …
• Ensembles – Forests
• Fit a bunch of models and average (continuous) or vote (categorical)
• Data Mining – Unsupervised Learning
• Hierarchical and K-means clustering
• Identify segments in the space of X variables
• Association Rules
• Identify associations e.g. between products, documents
• Simulation and Optimization
• Monte Carlo methods and (linear) optimization (mixed integer)
• Unstructured Data Mining
• String matching e.g. bipartite graph algorithm
• Network analytics
Modeling and Data Mining
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
EUR Predn; Decline/Type
P10/P50/P90 Estimation
Oil and Gas Reserves Analysis
Business Mix Operations Optimization
Profiling Fraccs
Well Economics Unconventional
Methods Optimization
Predictive Analytics in Energy
Analytics Components
• Non-linear modeling (exponential mixtures for recovery estimation)
• Distribution and frequency function estimation (eCDF): P10, P50, P90
• Constrained resource optimization based on well characteristics
• EUR models: effects of extraction parameters and spatial regression
• Optimizing unconventional plays
Real-Time Decisions
Portfolio
Reserves
Dashboards
A&D Analytics
Operations/ BPM
Geospatial Regression
Operational Health & Safety Safety
Portfolio Mgt Portfolio
Operating Wells
Environmental Effects
Land/Lease Potential
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
• Decline / Type curves
• Algorithms: Power law loss ratio, modified hyperbolic; sophisticated pre-processing / automation / over-rides
• P10, P50, P90
• Distributions: Log-normal, Weibull etc; Bootstrap (non-parametric)
• Geospatial regression
• Kriging, loess/splines; completions analysis / optimization
• ESRI Integration
• Shape file import/export
• Capital planning / portfolio optimization
• Portfolio
• Well / pad optimization
• Sales and marketing
• Health and safety analytics
• Text analysis
• Corpus analysis / networks
• Social and mobile applications
• Collaborative computing / iPad deployment and applications
Analytics Use Cases
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Arkoma Basin Data
Extract from IHS Enerdeq
Production on ~1300 wells
Production data 2005-2010
Completion information by frac
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Shale Gas
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
#1. Decline/Type Curves: EUR Estimation • Response
• Oil/gas production data across time
• Analysis
• Fit production decline and predict ultimate production
• Preprocess to remove outliers eg zero production days
• Backtest, report to SEC
Model
Arp’s equation: hyperbolic
qt = qi (1 + bDi t)^(-1/b)
b = reservoir decline exponent
Hyperbolic – exponential
Transition to exponential decline at user=specified time
Power law loss-ratio decline
qt = qi exp(-Dy t - Di t^n)
Dy = decline rate at infinite time
EUR via economic limit e.g. 1 mcf/d (0.03 10^3m^3/d) or time limit
References: Ilk et al, SPE 114947, SPE 116731; Texas A&M
New methods readily added via S+/R scripts
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Decline and Type Curves
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Decline and Type Curves • Pre-processing
• Response
• Oil/gas production data across time
• Analysis
• Fit monotone smoothing spline (2 df)
• Assess median absolute deviation from smoothing
spline and remove outliers from rest of pack at low
p-value eg 0.0001
Process / Application
• Works well when large number of wells and/or
noisy data (eg zero production days) - most of
time!!
• Manual over-ride is built-in i.e. if user doesn’t like
automated time point removal, can manually set
the points to be included in the decline curve fit
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Decline and Type Curves
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Decline and Type Curves
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
#2. EUR P10, P50, P90 Estimation
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Oil and Gas Reserves: EUR P90, P50, P10
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
#3. Geospatial Regression • Response
• Production, Environmental variables (groundwater)
• Explanatories
• Spatial locations
Model
• y = f (x, b) + e
• f : spatial regression, kriging, splines, loess
• packages : S+spatial, R-fields
• Thin plate splines:
• yk = f (x, k) + ek min ||y- f||2 + l int f ‘‘(x)2 dx
• Kriging:
• yk = P(x, k) + Z(x, k) + ek
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Geospatial Regression
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Geospatial Regression
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
#4. Integration with ESRI ArcMap
• Two way interactivity between TIBCO Spotfire and ArcMap
• Load data from an ArcMap project into TIBCO Spotfire
• Push a Map image from ArcMap into TIBCO Spotfire
• Interactivity between multiple concurrent TIBCO Spotfire and ArcMap sessions
• Re-establishment of the connectivity between previously integrated TIBCO Spotfire and ESRI ArcMap sessions with minimal user input
• Perform ad hoc analysis on spatial and tabular project data with immediate corresponding display in ArcMap
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Spotfire Connection
ESRI-Spotfire Integration
• Data associated with “layers”
• Point (e.g. Cities, Wells)
• Polygon (e.g. Countries, Field Reservoirs)
• Attribute table associated with each layer
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
After a connection has been set up, marking records in Spotfire will select the corresponding features in ArcMap
Spotfire & ArcMap Interactivity
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Correspondingly, selecting features in ArcMap will mark the matching records in Spotfire
Spotfire & ArcMap Interactivity
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
The extension also allows a MapChart plot to be added to the Spotfire session with background taken from the ArcMap session
The MapChart background can be exported either as a shape file for the selected layer or as a raster image based on what is currently displayed in ArcMap
Push a Map image from ArcMap to Spotfire
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
• Publish
• OpenSpirit Scan Utility
• Publish all G&G spatial data to GIS repository (and keep it synchronized)
• Interact with Desktop
• OpenSpirit ArcGIS Extension
• Use ArcMap as a data selector and viewer to interact with G&G data and other OpenSpirit enabled apps
• Interact with Web
• OpenSpirit Web Services
• “Publish” and interact with G&G spatial data through GIS Web portal using OpenSpirit web services
Spotfire – OpenSpirit Integration
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
#5. Resource (Compression) Optimization
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
S+ NuOpt • S-PLUS module: Premier Toolkit for numerical optimization
Large-scale optimization problem solving, including:
Linear programming
Mixed integer programming
Quadratic programming
Unconstrained nonlinear optimization
Multi-objective programming
NuOpt comprises a collection of optimization methods:
primal-dual interior point method with higher order
correction for Linear Programming (LP) models. [higher]
simplex method for Linear Programming (LP) and mixed
integer programming (MIP) models. [simplex]
primal-dual interior point method based on line search for
general Convex Programming (CP) models. [line]
primal-dual interior point method based on trust region
method for general Non-Linear Programming (NLP) models.
[trust]
primal-dual interior point method based on quasi-Newton
method for general Non-Linear Programming (NLP) models.
[BFGS]
NuOpt for S+ is a combination of the S+ programming environment, the NuOpt optimizer and the SIMPLE modelling language.
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
#6. Portfolio Management
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
#7. Sales and Marketing
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Sales Forecasting
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
S+ FinMetrics
• S-PLUS module: 500+ functions for time series analysis, econometric modeling and prediction
• Many other related functions in base S-PLUS e.g. Fourier analysis
Vector Autoregressive models (VAR)
Cointegration and Vector Error Correction Model (VECM)
Univariate and multivariate GARCH Volatility Modelling
State Space Modelling
Factor Models for Portfolio Allocation
Fixed Income Analytics (rate conversion, yield curve fitting …)
Extreme Value Theory and Copula Analysis\
Time Series Manipulation and Visualisation
Rolling Statistics and Technical Indicators
Rolling Estimation and Backtesting Strategies
Time Series regression (static, dynamic regression)
Systems of equations estimation (SUR)
Classical Time Series Analysis (ARIMA, unit root ...)
Long-memory modelling
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Forecasting: Parameter Selection
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
#8. Forecasting Commodity Prices
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
#9. Health and Safety
• Near real-time safety data: gives organization as much time as possible to take action and reduce incidents
– Had been doing monthly/quarterly reviews
– Web deployment
• Move from lagging indicators to leading
– Track incident pre-cursors, prevent injuries before they happen
• Easy to use for adoption and speed
– Increase participation through higher data interactivity
• Depth and Detail
– Make it clear how the organization should respond by providing high level trends and incident level details
– Transparency allows easy spotting of
errors
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
#10. Social and Mobile
• Social
• Collaborative computing
• Tibbr®
• Mobile
• iPad application in
Spotfire 4.5
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
• tibbr® and Apache • Solution for geographically distributed
teams on drilling projects • share articles, meeting notes, operating
procedures, documentation on tools
• ideas on ways to improve technology, processes and team objectives.
• Drillers can upload pictures from the field and crowdsource feedback to be shared in the community
• improves the quality of responses to issues and reduces trouble costs
• tibbr streams can be integrated in SharePoint - context of relevant projects
Social and Mobile
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Social and Mobile
Drilling Site #4
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Social and Mobile
Drilling Site 4 Drilling Site 5
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Social : Spotfire-to-Tibbr®
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Social : Spotfire-to-Tibbr®
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Social : Tibbr®
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
#11. Spotfire® Deployment Kit for Apple iPad
• Deploy white-labeled analytic apps to mobile devices
– Without mobile development team
– Configure & deploy in under an hour, no coding
• Spotfire’s mobile engineering team monitors & adapts to Apple’s rapidly changing APIs and policies
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Display a custom URL on startup • Spotfire app: • Custom example:
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Customize the Analysis Control in
Deployment Kit
Control in .dxp / Web
Player
Author Once,
Deploy Anywhere
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
SE
RV
ER
S
CLI
ENTS
Zero install web client Visual, Analytic Hybrid In-Memory Engines
Spotfire
Web Player
Spotfire
Enterprise Player
Spotfire
Professional
DA
TA
SO
UR
CE
S
XML RDBMS
Spotfire
Developer
IT / DBA
Development,
Solutions,
Integration Flat
Files
Web
Services Spread-
sheets
Spotfire S+ (R)
Managers, Consumers,
Executives
Analysts
Independents
Configurators
Analysts
Informaticians
Statisticians
SAP R/3
Managers, Consumers,
Executives
Spotfire Miner
In-Memory Engine
Spotfire
Web Player
Server
Administration & Integration
Spotfire Server
App Data Connectivity
Spotfire
Advanced Data
Services Real-Time Connectivity
Complex Event
Processing
Services
Computation Engine
Spotfire Statistics
Services
Informaticians
Statisticians
Oracle
E-Business Siebel
eBusiness
EMS, FTL
Batch Analysis
Spotfire
Automation
Services
SDKs
Product (Spotfire) Stack
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Product (Spotfire) Stack
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Statisticians
Develop analytic
Matlab/SAS Spotfire Miner R/S+
Integrating Statistics into Spotfire
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Statisticians
Develop analytic
Spotfire Statistics Services
Deploy and register analytic
Spotfire Server
Make analytic available to
application authors
Integrating Statistics into Spotfire
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Statistician
Develop analytic Analysts and Application Authors
Configure Application to use analytic No coding required
Spotfire Statistics Services
Spotfire Server
Deploy and register analytic
Integrating Statistics into Spotfire
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Statistician
Develop analytic Analysts, Application Authors
Configure Application to use analytic
Managers, Consumers, Executives
One-click deployment of web applications
Centrally-managed application with governance managed by statistics /
analytics team
Deploy and register analytic
Spotfire Statistics Services
Spotfire Server
Integrating Statistics into Spotfire
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Fastest to Actionable Insight Instantly turn insight into action by enabling anyone to rapidly discover
hidden insights and quickly collaborate
in context
Universal Adaptability Leverage a single analytics and data discovery platform to empower
anyone, anywhere to make insightful decisions
Self-Service Discovery Freely explore data to any level of detail, radically accelerating decision
making, while dramatically reducing dependence on IT
Visibility Into the Unknown Discover unexpected insights hidden in Big Data and Real-Time Events to
immediately identify strategic business opportunities or threats.
Spotfire Value Drivers
© Copyright 2000-2012 TIBCO Software Inc. All rights reserved. TIBCO Confidential & Proprietary Information.
Michael O’Connell, PhD
Sr. Director Analytics
TIBCO
Contact Info
David Meade
Sr. Solution Consultant
TIBCO
Peter Shaw, PhD
Sr. Consultant
TIBCO
Tim Crocker
Sr. Solution Consultant
TIBCO