Benchmarking of Transactions with DEA: Analyses on Process Level at the Example of a Banking Process F r a n k f u r t S c h o o l. d e Andreas Burger Jrgen Moormann 20 minutes presentation F r a n k f u r t S c h o o l. d e7th Int. Conf. on DEA, Conference Presentation, Philadelphia, PA, Agenda 1.Introduction 2.Research positioning / contribution 3.Definitions 4.Methodology 5.Case study 6.Results 7.Conclusion F r a n k f u r t S c h o o l. d e7th Int. Conf. on DEA, Conference Presentation, Philadelphia, PA, Introduction 1 Process performance matters Performance gaps are existent Process performance is strongly linked to execution quality (a) process design and (b) execution quality Variation is all-around Basic idea of this work: intrinsicinefficiency (1) Detect (intrinsic) inefficiency in process execution performance (2) Evaluate the performance gap (inefficiency best practice) (3) Derive measures for improving the process to best practice level e.g. process modeling / simulation Companys performance An overview e.g. Deming, Six Sigma identify gaps find ways to close gaps F r a n k f u r t S c h o o l. d e7th Int. Conf. on DEA, Conference Presentation, Philadelphia, PA, Research positioning / contribution 2 Process analysis Data Envelopment Analysis (DEA) Process Performance Measurement Research gap Non parametric methods Operations Management 3 Research area Scientific field Process diagnosis Research strategy: Design science (Hevner et al.) Effective search for a method Research contribution / Whats new? Concept of intrinsic inefficiency for analyzing the process execution quality Benchmarking of Transactions: An innovative DEA-based approach for process performance analysis Generating first results from an case-study application in Banking Research contribution / Whats new? Concept of intrinsic inefficiency for analyzing the process execution quality Benchmarking of Transactions: An innovative DEA-based approach for process performance analysis Generating first results from an case-study application in Banking Using transaction logs for analyzing the process execution quality. Process Management Performance measurement Business Process Intelligence Efficiency Measurement F r a n k f u r t S c h o o l. d e7th Int. Conf. on DEA, Conference Presentation, Philadelphia, PA, Definitions 3 Distance: A - A*Intrinsic inefficiency A*- BExtrinsic inefficiency B - C Theoretical inefficiency ___________________________ A - C Total inefficiency + + = Intrinsic Inefficiency A concept to determine the inefficiency in executing a given process! What is the appropriate benchmark? Main source for intrinsic inefficiency is variation in performing process activities. This variation affects the performance (e.g. time, cost, quality) of a transaction as well as of the process. Intrinsic inefficiency on process level = Performance gap in comparison to best practice execution! F r a n k f u r t S c h o o l. d e7th Int. Conf. on DEA, Conference Presentation, Philadelphia, PA, Methodology 4 Object of Analysis: TX the thing WPAID Unique reference Process instance Trader Exchange Type Currency Consideration [] Error codes Outcome characteristics Resource utilization [t] TX = Transaction performance Process performance Benchmarking of Transactions TX*Find best practice 1 TX Compare against 2 TX* Calculate individual EFF-score for every, ( = 1) 3 TX TX* Determine target levels for improvement for every 4 TX Methodological approach: (1)Detect and determine intrinsic inefficiency for all transactions of a process by benchmarking and (2) Aggregate the results to determine the inefficiency profile of the process itself! (Input factor) (Output factor) (additional information, for pattern description) Looking inside the performance of a process Analysis steps: via DEA analysis Analyze the pattern of (in)efficiency and reveal new insights of the performance 5 Factors of performance (Basis for Input-Output- Model definition) Performance of a TX vs. Process Performance F r a n k f u r t S c h o o l. d e7th Int. Conf. on DEA, Conference Presentation, Philadelphia, PA, Method / Methodology 4 Productive view (EFF 1) Performance view (EFF 2) Profitability view (EFF 3) Question? Relationship between production effort and output quantity. A transaction would be classified as efficient when it is produced with minimal effort. Productivity analysis with multiple variables. Question? Relationship between production effort and the performance achieved. A transaction would be classified as efficient if it achieves the focused performance with minimal effort. Theoretical, but integrated point of view Question? Relationship between costs spent and revenues generated. A transaction would be classified as efficient if it reaches the maximum profitability. Performance equals a simple ratio- based profitability analysis. Man time (h) 1 Inputs: Outputs: Auto time (h) TIME-Index [%] CpTx-Index [%] Rev.-Index [%] STP-Index [%] CpTx () Revenue [] Input-Output-Models (Basis for DEA optimization) TX Inputs: Outputs: Inputs: Outputs: TX Focus: Engineering Factors: Time Three views on performance: Modeling performance on TX level: Focus: Engineering, economic Factors: Time, cost, quality, profitability Focus: Economic Factors: Cost, profitability F r a n k f u r t S c h o o l. d e7th Int. Conf. on DEA, Conference Presentation, Philadelphia, PA, Scope of the analysis Trade Capture and Execution Process steps: Enrichment Validation Measuring points t Time of capture Data source: 1 23 Data sensors: Time of reception In COWIAS IT-Systems: Bloomberg GL343 COWIAS GL343 Time of last status change Time of last statement Max. Time of last booking Max. Time of last settlement Max. WKAAAA WKBRAA D3UMGN D3ULGN Daten werden in GL 343 weitergegeben ad e2e2 g2g2 f2f2 h Timestamps Clearing and Settlement Internal bookings 54 End WKAAAA Duration N = Bond-buy/sell transaction May 2008 Trade capture in Bloomberg without notes Case study / Example: Securities processing 5 F r a n k f u r t S c h o o l. d e7th Int. Conf. on DEA, Conference Presentation, Philadelphia, PA, Transaction: Transaction (TX) 1. WPAID / Ordernummer 2. Geschftstag (GESCHTAG) 3. Valuta (VALUTA) 4. Geschftsart, Kauf oder Verkauf (WPGSHART) 5. Handelsplatz (HDPLATZ / HDPLBEZ) 6. Abrechnungswhrung (ABRWHR) 7. Nennwert (NENNWERT) 8. Maklernummer (NRMAKLER) 9. Whrung ausmachender Betrag (WAUSMBETR) 10. Wert des ausmachenden Betrags (AUSMBETR) 11. -Wert des ausmachenden Betrags (AUSMBETR ) 12. Hndler-ID (HNDLRNR) 13. Geschftsbereich (GESCHBER) 14. Liefersystem, Ebene 1 (BAORDLS0) 15. Liefersystem, Ebene 2 (BAORDLS1) 16. Liefersystem, Ebene 4 (BAORDLS4) 17. WP-Bezeichnung (WPKBEZ) 18. Wertpapieridentifikator (ISWPKNR / WPNRISIN) 19. Wertpapierkurswhrung (WPKRSWHR) 20. Wertpapierwhrung (WPWAEHR) 21. Status-Info Ergnzung (BAORDSTA) 22. Zusatzinformation 1 (ORDZUSTX1) 23. Zusatzinformation 2 (ORDZUSTX2) (a) Erfassungszeitpunkt (ERFTS) (b) Statuswechsel in Ergnzung (BAORDEZP/05) (c) Statuswechsel in Ergnzung (BAORDEZP/00) (d) Bekanntwerden in COWIAS (WPAANLTS) (e 1 ) Erste Abrechnung (WPAABRTS-MIN) (e 2 ) Letzte Abrechnung (WPAABRTS-MAX) (f 1 ) Erste Buchung (DPBUCHTS-MIN) (f 2 ) Letzte Buchung (DPBUCHTS-MAX) (g 1 ) Erstes Settlement (DEPZKLTS-MIN) (g 2 ) Letztes Settlement (DEPZKLTS-MAX) (h) Letzte Statusnderung in COWIAS (WPASTATS) Trading info: Intrument: Reference: Enrichment: Processing: the thing Process instance Case study 5 F r a n k f u r t S c h o o l. d e7th Int. Conf. on DEA, Conference Presentation, Philadelphia, PA, Results / Production view 6 :0,5912 Min.:0,2018 Max.:1,0000 :0,1486 N=11869 Inputs: I 2 : Cycle time 2 I 3 : Cycle time 3 I 4 : Cycle time 4 I 5 : Cycle time 5 Outputs: O 1 : = 1 TOP 5 Benchmark transactions: # TX Additional 57 Benchmark transactions can be found; thereof 4 are self-identifying. Distribution of EFF 1 Efficient TX (EFF=1) Relationship between processing effort and output quantity Model assumption Input-Output-model Findings: Intrinsic inefficiency of the production view equals ~40,8%. 62 Benchmark transactions. Distribution of EFF 1 close to normal distribution Potential for improvement of 48% of cycle time can be identified. Input-slacks sum up to 2,7% of total cycle time. Results of the analysis are considered plausible. F r a n k f u r t S c h o o l. d e7th Int. Conf. on DEA, Conference Presentation, Philadelphia, PA, Results / Traget view 6 Inputs: I 6 : Cycle time-index Outputs: O 2 : Unit cost-index O 3 : Revenue-contribution-index IDs of Benchmark transactions: # TX :0,0801 Min.:0,0001 Max.:1,0000 :0,0789 N=11869 Distribution of EFF 2 Efficient TX (EFF=1) Model assumption Input-Output-model Findings: Relationship between processing effort and target achievement from an operative point of view Intrinsic inefficiency of the target view equals ~92,0%. 2 Benchmark-TX. Well balanced consideration of cost and revenue aspects in the specific view. Only very limited and marginal slacks can be found. Results of the analysis are considered plausible F r a n k f u r t S c h o o l. d e7th Int. Conf. on DEA, Conference Presentation, Philadelphia, PA, Results / Profitability view 6 IDs of Benchmark- transactions: # TX :0,0055 Min.:0, Max.:1,0000 :0,0321 N=11869 Relationship between unit cost and revenue contribution Inputs: I 4 : Unit cost Outputs: O 5 : Revenue contribution Distribution of EFF 3 Efficient TX (EFF=1) Model assumption Input-Output-model Findings: Intrinsic inefficiency of the profitability view equals ~99,5%. 1 Benchmark-TX. Significant variation across single transactions. Consideration amount is dominant factor: EFF 3 = 0,46 for top 20 Big Tickets. Revenue contribution almost not correlated with unit cost (Factor: 0,036) No slacks. Results of the analysis are considered plausible. F r a n k f u r t S c h o o l. d e7th Int. Conf. on DEA, Conference Presentation, Philadelphia, PA, Korr: -0,382 TX-Analyse fr Ansatzpunkte fr Verbesserung 1 Outlier- Analyse 2 Korr: -0,490 3 TX-Analyse fr Ansatzpunkte fr Verbesserung Outlier- Analyse 4 Results / Additional analysis 6 F r a n k f u r t S c h o o l. d e7th Int. Conf. on DEA, Conference Presentation, Philadelphia, PA, Conclusion 7 1) Benchmarking of Transactions offers new insights into the process performance / patterns of inefficiency can be revealed to improve process. 2) Alternative instrument for process analysis and diagnosis on DEA basis. 3) Concept of intrinsic (in)efficiency supports determining the execution quality of a given proess. Benchmarking of Transactions / Key characteristics: (1)Focus on single transactions. (2)Best practice-Benchmarking, rather than comparing averages. (3)Single performance measure including multiple variables. (4)Process analysis on the basis of input-output models. (5)Strong methodological basis in comparison to subjective analytical methods. Key results of the typical banking process application are: (a)Large variance in performance on transaction level exists. (b)Significant intrinsic inefficiency can be detected. (c)The pattern of inefficiency differs across the three views of performance (d)Critical areas for improving the performance can be identified.