Embed Size (px)
Citation preview
Methods and Technologies forBusiness Process Monitoring
Josef Schiefer
Vienna, June 2005
2
Agenda
»Motivation/Introduction»Real-World Examples»Technology Perspective»Web-Service Based Business Process Monitoring»Adaptive Business Process Monitoring with Sense & Respond»Modeling Sense & Respond»Business Process Monitoring with Senactive» “Diplomarbeitsthemen” in the area of business process
management & monitoring»Q&A
3
MotivationMotivation
4
Motivation
"What one can measure, shall bemeasured; what cannot be measured,
shall be made measurable."Schmalenbach (1963)
"If you cannot measure it, you cannotcontrol it. And if you cannot control it,
you cannot manage it."Harrington (1991)
"Large scale production workflow systems withthousands of interactive tasks per day will generate
megabytes of audit trail data for analysis."
What do you do with this information?
McLellan (1996)
5
Why Business Process Monitoring?
» Automatically extracts and generates performance data from enterprise processes
» Uncovers weaknesses in process handling
» Optimize throughput times, resource assignments
» Warning system by monitoring business processes
» Benchmarking based on measurable process indicators -> identification of best practices
» Process-driven analysis of business data (e.g. break down metrics by business activities)
» Past activities can be used to drive prospective interactions with the customer
» Continuous process improvement, closed loop analysis
6
IntroductionIntroduction
7
Business Process Management Lifecycle
BusinessProcess
Analysis &Control
BusinessProcess
Execution
BusinessProcess
ModellingImplementation
BPMEnd-to-End
Tooling
8
How well is Performing a Business Process?
Task 1 Task 2 Task 3 Task 4 Task 5 ProcessOutcomes
Customer
Sub Process 1Sub Process 2
M3
M4 M4
M3
M4 M4 M4
M3 M1M1
Internal
ExternalM2
Manager ObjectivesM3
- Reduce service costs- Reduce order processing time- Improve service quality- Optimize resource assignments
Workgroup ObjectivesM4
- Reduce defects/waste/delays- Improve customer rating- Automate processing steps- Early error detection
Vice Pres./Director ObjectivesM2
- Shorten delivery and supply time- Increase Return on Investment- Increase delivery performance
Executive ObjectivesM1
- Fully exploit customer potential- Improve profitability- Customer satisfaction
9
Process Related Business Questions
CustomerContact
Processing
ProcessCustomerRequest
CreateCustomer Offer
CreateContract
Sample ProcessCustomer Request Processing 1/3
Analysis of the process (time, quality, costs...) and its activities:
How many customer requests result in a contract?Which activities are the bottlenecks in the customer request processing?How satisfied are customers with the current process?Show the customer requests with the longest processing time?Show the average processing time of customer requests for the company XY?How did the processing time and process cycle time change compared to the lastquarter / last year?How did an introduction of an CRM software package impact the processing of thecustomer requests?
Business Questions
10
Process Related Business Questions
CustomerContact
CustomerContact
Processing
No Need forFurther Contact
CustomerRequestreceived
CustomerRequest from
Contact
ProcessCustomerRequest
Export NotPermitted
Offer MustBe Created FromCustomer. Req.
Customer Req.Is Profitable
Customer Req.Is Canceled
CreateCustomer Offer
Offered Items orService Cannot
Be Delivered
Contract MustBe CreatedFrom Offer
Customer OfferIs Sent
CreateContract
Contract isCanceled Contract
Confirmation isSent
FulfillmentRequestCreated
Customer Request Processing 2/3
Analysis of the control flow and the events:
How many customer contracts originate from customer contacts and howmany are direct customer queries?Which customer requests couldn't be processed because of exportrestrictions?Which customer requests were turned back because of less profitability?Which customer requests were canceled? Why were they canceled?Which customer requests were not processed because the items orservices could not be delivered?Which customer requests were canceled during the contract processing?
Business Questions
11
CustomerContact
CustomerContact
Processing
Sales Team No Need forFurther Contact
CustomerRequestreceived
CustomerRequest from
Contact
ProcessCustomerRequest
Sales Team
SalesData
Export NotPermitted
Offer MustBe Created FromCustomer. Req.
Customer Req.Is Profitable
Customer Req.Is Canceled
CreateCustomer Offer
Offered Items orService CannotBe Delivered
Contract MustBe CreatedFrom Offer
Customer OfferIs Sent
Sales Team
OfferData
CreateContract
ContractProcessingDepartment
Contract isCanceled Contract
Confirmation isSent
FulfillmentRequestCreated
ContractData
Process Related Business Questions
Analysis of the process context (organization, customer, channel...)Which sales teams initiated and processed the most customer requests? Which salesteams are initiating customer contract with a contract volume of more than 100.000$?What are the Top/Bottom 10 contract volumes by channels which were used for thecustomer contacts.Which sales teams do the most profitable customer request processing?Which customer requests result in long-term contracts?Which customer requests cannot be fulfilled and why?How long is the average customer request processing per sales team?How much time does the contract processing take with a contract volume of more than100.000$ compared with last quarter / year?
Business Questions
Customer Request Processing 3/3
12
Real-World ExamplesReal-World Examples
13
Example: Ordering a Book from Amazon
Shopping Checkout Process Post-Checkout Processing
14
Example: Ordering a Book from Amazon
Use
r Ses
sion
Sta
rted
Book
Sea
rch
Add
Boo
k to
Sh
oppi
ng C
art
Rem
ove
Boo
k fro
m
Sho
ppin
g C
art
Che
ckou
t
Iden
tify
Use
r
Ship
ping
Info
rmat
ion
Cap
ture
d
Paym
ent I
nfor
mat
ion
Cap
ture
d
Ord
er P
lace
d
Paym
ent C
lear
ed
Ord
er C
onfir
mat
ion
Sen
t
Even
ts fr
om W
ebsi
te
Business ProcessMonitoring
15
Management Portal for Supply Chains
(c) IBM Watson Research
16
Management Portal for Supply Chains – Detailed Exception View
(c) IBM Watson Research
17
Technology Perspective ofBusiness Process MonitoringTechnology Perspective of
Business Process Monitoring
18
Technology Evolution forGaining Insight into the Business
Approaches for Insight into Business Data
Data Warehousing
Business Process Warehousing
Real-Time Business Monitoring
Adaptive Monitoring Environment
Tech
nolo
gy E
volu
tion
19
Business Process Monitoring – Technology Landscape
Business Process &Workflow Management
Business ProcessModeling & Analysis Network Systems
Management
Integration BrokerSuites, EAI
Business Intelligence,Data Analysis
Data Warehouse,Databases
Business SoftwareERP, CRM, SCM…
Data IntegrationSuites, ETL
Tibco, Vitria,IBM, SeeBeyond
BMC, CA, HP
Informatica,Ascential, Data Mirror
Oracle,IBM, Microsoft
Business Objects,Cognos, Hyperion
SAP, Oracle,Microsoft
LombardiIBM,FileNet,Staffware (Tibco)
IDS-Scheer,CaseWISE,
Popkin
Business Process MonitoringTechnology Mix
20
Web-Service BasedBusiness Process Monitoring
Web-Service BasedBusiness Process Monitoring
21
Monitoring of a DistributedSupply Chain Process
• Monitoring of intra-organizationalbusiness processes
• Shared monitoringinfrastructure forbusiness partners
• Monitoring ofbusiness processesas on demand service
22
Business Process Monitoring: Four Web Service Types
Define Web Service
• Setup of monitoringinfrastructure
• Identifies web servicesfor monitoring andtheir owners
• Defines the webservices states andstate transition rules
• Defines performanceobjectives
• Defines securitypolicies
Log Web Service
• Captures audit traildata from varioussource systems
• Logs state changesof a businessprocess
• Calculates businessprocess metrics andstores them in adatabase
• Correlation ofprocess events
Analyze Web Service
• Query interfaces forretrieving performancedata
• The performance dataincludes
• web serviceenactments
• process metrics• service level
violations• Access to analytical
agents• Dependent on policies
defined by the webservice owner
Monitor Web Service
• Senses performancedata for exceptionalbusiness situations
• Responds toexceptional businesssituations (e.g. bycalling another web-service)
• Manages service levelagreements
23
Monitoring BPEL4WS Processes
BPEL4WSProcess
BPEL4WSProcess
<invoke>
<receive>
<invoke>
<receive>
<reply>
Port Type
audit<invoke>
audit<invoke>
...
...
BPEL4WSProcess
BPEL4WSProcess
<receive>
<reply>...
<receive>
<reply>
<receive>
...
MonitoringInfrastructure
Log WebService
log item
EventStream
Processor
Business PartnerBusiness Processes
Organisation with MonitoredBusiness Processes by SMWS
Real-timeData Store
DataWarehouse
24
Monitoring Process Managed By Workflow Management Systems
25
Monitoring with Probes in Operational Systems
Obs
erve
dB
usin
ess
Proc
ess
System 1
System 2
System 3
System 4
Observable BusinessOperations
Probes
Audit Data Flows
Logging Daemon/SMWS Proxy
Business Operations
MonitoringInfrastructure
Log WebService
log itemEvent
StreamProcessor
Log Files, Log Tables
Real-timeData Store
DataWarehouse
26
Event Stream Processor
Event Stream Processor
Metric Evaluation
Event FlowManagement
Notifications &Alerting
Event Correlation
Event Condensation
Event StandizationMetric Calculation
Event and MetricPersistence
Services forEvent Processing
Receive/Unify Events Event Processing Evaluation/Alerting
�
��
Event StreamQuerying
Log WebService 1
Log WebService 2
Log Web Services
Log WebService 3
Log WebService 4
BPEL4WSProcess
BPEL4WSProcess
BPEL4WSProcess
BPEL4WSProcess
Raw Logging Items
27
AdaptiveBusiness Process Monitoring
with Sense & Respond
AdaptiveBusiness Process Monitoring
with Sense & Respond
28
Sense & Respond adds intelligence when the execution of a business operation depends on» State of a business process (e.g. delays of process activities)» Complex event pattern (e.g. a fraud event patterns)» Assessment of opportunities, risks, capacity utilization, quality of service» Real-time key performance indicators, which can be highly aggregated business metrics
(e.g. cycle times, processing costs, response time to customer)» Predictable indicators (e.g. demand forecasts, production plans)
Business Process Management and Sense & Respond
Monitored Environment
IT Systems Business Processes
…
BusinessOperations
Sense & Respond adds intelligence to when and how a business operation is executed
Events
Adapt
29
Sense & Respond Loops
RespondDecide
Analyze
Interpret
Sense
Acc
eler
ate
Responding
Sensing» Extracts and collects relevant event data
from the business environmentSense
Interpret
Analyze
Decide
Respond
» Giving the event data a meaning» Transforms event data into information
(e.g. KPIs, business situations and exceptions)
» Determines root causes of identifiedbusiness situations
» Predicts the performance and assess the risksfor changing the business environment
» Selects the best option for improvingthe business situations
» Determines most appropriate action for a responseto the business environment
» Communicating the decision as a command or suggestion» Executing the business actions in target systems
30
Business Activity Monitoring with Senactive InTime
Real-time BusinessPerformance Data
Business ActivityMonitoring
Production System ERP SystemE-Commerce System
SenseEvents of orderprocess
Response
Process Metrics
Notifications
SenseEvents of production
processesUpdates ofproduction plan
ResponseTrigger re-orderingof products and materials
eMail fororder confirmations
ProductionManagement
1
23
4
5
31
Sense & Respond and Data Warehousing
Sens
e
Inte
rpre
t
Ana
lyze
Dec
ide
Res
pond
Traditional approach forintegrating data into thedata warehouse
32
Sense & Respond – Logistics & Transportation Management
Facility to FacilityTransportation
Sens
e
Int e
rpre
t
Ana
lyze
Dec
ide
Res
pon dSense
(Shipment Events)
Response &Notifications(Change route,transporation mode,priorities, ...)
33
ModelingSense & Respond
ModelingSense & Respond
34
Event Driven Process Chains and Sense & Respond Processes
Outside View of a Sense &Respond Process
35
Event Driven Process Chains and Sense & Respond Processes
Inside View of a Sense &Respond Process
BPEve
nt1
36
Event Driven Process Chains and Sense & Respond Processes
Example for Inside View of aSense & Respond Process
Select Orders thatshould be examed
Decide on Discount
Order Request
Give Discount
Analyze CustomerPayment History
Order Event
NewCustomer
Rating
Deny Order
FilteredOrder Event
Customer PaymentHistory Accepted
Deny OrderEvent
External CreditCheck
Deny OrderEvent
Update CustomerRating
Credit RatingAccepted
CustomerDiscount
Applicable
37
Web-Service Based & AdaptiveBusiness Process Monitoring
Web-Service Based & AdaptiveBusiness Process Monitoring
38
Steps to Go…
Senactive InTime Sense & Respond Loops
InTime S&R Service
MonitoredEnvironment
IT Systems
Business Processes
…
1
Probing / Sensing
2
Transmit Events
3
Unify Events
Transform Events intoBusiness Indicators4 Detect Situations or
Exceptions5
6 Analyze Situationsor Exceptions
Make Decisions7Put Responseinto Action8
39
Supply Chain Monitoring of BPEL4WS Processes
40
Monitoring of BPEL4WS Processes with Senactive
Service Provider TypeService Provider Type
BPEL4WSProcess
BPEL4WSProcess
<invoke>
<receive>
<invoke>
<receive>
<reply>
Port Type
audit<invoke>
audit<invoke>
...
...
BPEL4WSProcess
BPEL4WSProcess
<receive>
<reply>...
<receive>
<reply>
<receive>
...
MonitoringService Provider Type
Log WebService
log item
ProcessWarehouse
Sense &RespondSystem
<invoke>
41
Monitoring of Non-BPEL4WS Processes with Senactive
Obs
erve
dB
usin
ess
Proc
ess
System 1
System 2
System 3
System 4
Observable BusinessOperations
Probe Points
Audit Data Flows
Web Service Proxy
Business Operations
MonitoringService
Log WebService
log itemProcess
WarehouseSense &RespondSystem
Response
42
Sense & Respond System
Sense & Respond System
Metric Evaluation
Event FlowManagement
Notifications &Alerting
Event Correlation
Event Condensation
Event StandizationMetric Calculation
Event and MetricPersistence
S&R Services
Receive/Unify Events Event Processing Evaluation/Alerting
�
��
Event StreamQuerying
Log WebService 1
Log WebService 2
Log Web Services
Log WebService 3
Log WebService 4
Event Adapters Event Service Evaluators
BPEL4WSProcess
BPEL4WSProcess
BPEL4WSProcess
BPEL4WSProcess
Raw Logging Items
43
Sense & Respond Monitoring with Senactive InTime
BusinessEnvironment
ComputerSystems
Logistics,Transportation
BusinessProcesses
External Systems,Business Partners
Production
Sens
e In
terf
ace
Res
pond
Inte
rfac
e
Event ServiceAdapter
HubAdapter
Hub
Event Service
Event Service
...
...
Adapter
Hub
Event Service
Event Service
Event Service
Adapter
Adapter
Adapter...
Sense
Interpret
Analyze
Decide
Respond
Senactive InTime Key Features
» Visualization of Sense & Respond Processing
» Modular and Service-Oriented Architecture
» Generation of Real-Time Key PerformanceIndicators
» Real-Time Data Analysis for ControllingBusiness Processes
» Instant Response on Business Situations andExceptions
Sense & Respond Model
44
Research Topics for “Bakk.-Arbeiten”, “Diplomarbeiten” etc.
Research Topics for “Bakk.-Arbeiten”, “Diplomarbeiten” etc.
45
Research Topics (see also http://www.ifs.tuwien.ac.at/~js)
Business Process Management» Web Service Based Auditing of BPEL4WS Processes (� Web Services, BPEL4WS, Messaging)» Scalable Messaging Infrastructure for Monitoring Business Processes (� Messaging, MSMQ, MQSeries, JMS,…)» Analysis Services with XMLA for Business Process Monitoring (� OLAP, Data Mining, SQL Server 2005)
Event Management» Event Models for Monitoring Business Processes (� XML Schema for events)» Event Simulation Model for Business Processes (� Generating consistent XML events)» Discovering Patterns in Event Streams, Event-Driven ECA Rules (� Rule Engines, XML Rules)» Correlation and Synchronization of Event Streams (� XPath, Concurrency, Distributed Computing)
Data Management» Data Management for Event Streams (� Star Schemas, Data Warehousing, SQL Server 2005)» Real-time Analytics & Data Stream Analysis with OLAP, Neuronal Network, Decision Trees, Clustering
Techniques, Association Rules (� Data Warehousing, Business Intelligence, SQL Server 2005)
User Interfaces» Visualization/Modeling of Sense & Respond Processes (data flows, control flows, dependencies)
(� Visualization and manipulation of Graphs, GUI design, Visual Studio 2005, C#)
46
Q&AQ&A
Methods and Technologies forBusiness Process Monitoring
Josef Schiefer
Vienna, June 2005
