View
1.265
Download
0
Tags:
Embed Size (px)
DESCRIPTION
P.I. Elsas & J. Gangolly: Enterprise-level Process Documentation incorporating Automatic Audit Analytics, Biennial Deloitte / University of Kansas Auditing Symposium, Lawrence, Kansas, USA, May 2008 (invited keynote)
Citation preview
Enterprise-level Process Documentationincorporating Automatic Audit Analytics
Philip Elsas, ComputationalAuditing.com Jagdish Gangolly, SUNY-Albany
Lawrence, Kansas May 2-3, 2008
2008 Deloitte / University of Kansas Auditing Symposium
Assessing Audit Risks in an Evolving Assurance Environment
ComputationalAuditing.com
Introduction• Since 2003: Company - Canada, Netherlands
• 1988 - 2003: Deloitte. with Bakkenist intermezzo, sold to Deloitte.
• 1990 - 1996: PhD Computational Auditing
- Principal, Chief Architect & inventor of Smart Audit Support - Smart Audit Support is since 1994 key in Deloitte’s worldwide audit practice. Currently integrated in “The Deloitte Audit”- System blueprint in Chapter 5 of …
- PhD in Mathematics & Computing Science on Financial Auditing - Parallel to Smart Audit project, 30% part-time - Directly after appearance awarded with the biennial Alfred Coini Prize for the best publication in Auditing
Offering software and consultancy services to audit practices and audit software firms
1
Used in 2003 by Dutch Tax Office as Frame of Reference to compare Big 4 planning and decision-support models & systems to investigate how to improve audit productivity (57 page report). Considers Smart Audit Support “leader of the pack”.
ComputationalAuditing.com
Agenda
Enterprise-level Process Documentationincorporating Automatic Audit Analytics
• Modern Auditing: Challenge, Criteria & Solution Approach (8)
• What can you do with it? Examples of major analytics (8)
• More on positioning this Doc Technique & Tooling (4)
2
• What is it? How doc looks and what it actually is (4 + movie)
• How to prepare it? Making doc in safeguarding tool (1)
ComputationalAuditing.com
On both Client Engagement Level & on Template Level
Modern Auditing: Challenge
1. Focus on Client’s Processes
3
While bridging the gap in the Audit Process between:
2. Risk Analysis on Process Assertions: identify, assess & respond
3. Items in the Financial Statements
In a modern top-down, risk-based Audit Approach with a focus on client’s processes the challenge boils down to:
a. How to understand Client’s Top-level Business Process
b. How to guide and document getting this understanding
c. How to guide and document using this understanding
- Ruling standards &
- Audit software 1 2
3
Client’s Occurrence Risk
Auditor’s Detection Risk
ComputationalAuditing.com
Deloitte’sInternationalAudit Approach
- “40.000 feet”, nineties
- Role of Doc: all phases
PERFORM PRE-ENGAGEMENTACTIVITIES
Assess Engagement Risk
Establish Terms of Engagement
Perform Preliminary Analytical Procedures
Understand the Client's Business
Understand the Accounting Process
Determine Planning Materiality
Develop Client-Service Objectives
Understand the Control Environment
Assess Risk at the Account and Potential-Error Level
Rely on Controls ? Control Reliance Strategy ?
Identify ControlsIdentify Controls and,if Efficient, Establisha Rotation Plan
Test Controls
Perform FocusedSubstantive Tests
Perform Basic Levelof Substantive Tests
Perform IntermediateLevel of
Substantive Tests
Evaluate Results of Tests
Perform Financial Statement Review
Perform Subsequent Events Review
Obtain Management Representations
Report on Financial Statementsand Render Management Letter
PERFORMPRELIMINARYPLANNING
ASSESSRISK
DEVELOPAUDITPLAN
PERFORMAUDITPLAN
CONCLUDEANDREPORT
That Mitigate Risk
Specific Identified Risk No Specific Identified Risk
NO YES YES NO
p.62
Deloitte’sAudit Processat EngagementLevel (1 of 3)
4
ComputationalAuditing.com
Doc Index
Planningdocs arepart ofSmart AuditSupport
Deloitte’s Audit Process at Engagement Level (2 of 3)
p.336
5
ComputationalAuditing.com
Inside aplanning document
“Player”system
Player of what?GuidanceModel
Where does thatModelcome from?
(=investment)
Deloitte’s Audit Process at Engagement Level (3 of 3)
p.337
6
Guidance is:- Easy-to-use &- Powerful
Easy-to-use:- Familiar interface: form-based- Answering multiple-choice questions that guide & document the audit, and…
as a tacit side-effect of answering: safeguards the correct (de)activation of other questions, & “how to” approaches to risk assessments & responses
Here questions can only be answered
Powerful:- Effective: conditionally relevant risks cannot be overlooked &- Efficient: risks conditionally not relevant cannot be assessedYearly ROI guess: 20K man-yrs/yr
x $10K cost reduction/man-yr
ROI
Return is:- Relevant Doc & Planning, no more no less- Easy & strict way to get it
Documentation = Specification
Executable Specification = Source CodeExecutable Specification of
“Auditor’s Evidence Acquisition Strategy” -David Budescu, Mark
Peecher & Ira Solomon -
Integrated in Interactive Documentation
ComputationalAuditing.com
KSTDM
APM AEM
APPM
KST
7Deloitte’sSmartAudit Support(1)
p.324
Proven Architecture for Interactive Documentation
& Guidance
Audit Plan Performance Module(blueprint only)
Audit Evaluation Module(blueprint only)
Smart Audit Support (2)
Audit Planning Module
KST Definition Module
Knowledge Specification Tool
one per engagement team
one in Deloitte
one per country
National Tailoring
ClientTailoring
AssuranceEnvironment
ComputationalAuditing.com p.334
8
Defining aplanning documentwith itsbehavior
“Builder”system
Builder of what?GuidanceModel
Builder’sprimitivescome fromtheory
Here questions are made and connected
Documentation = Specification
Executable Specification = Source Code
Deloitte’sAudit Processat TemplateLevel (1 of 1)
Guidance is:- Easy-to-use &- Powerful
Easy-to-use:- Familiar interface: form-based- Dialog box transactions: to stepwise specify an interconnected questionnaire to guide and document the audit, and…
as a tacit side-effect of every step: safeguarding a technically correct (de)activation structure for questions & their answer choice’s impact on audit planning
Powerful:- Correctness by Construction- Domain-specific Language
Executable Specification of “Auditor’s Evidence Acquisition Strategy”
-David Budescu, Mark
Peecher & Ira Solomon-
Integrated in Interactive Documentation
ComputationalAuditing.com
Challenge & Criteria9
In a modern top-down, risk-based Audit Approach with a focus on client’s processes the challenge boils down to:
a. How to understand Client’s Top-level Business Process
b. How to guide and document getting this understanding
c. How to guide and document using this understanding
3
Now we have key criteria for modern guidance in process documentation:
Guidance:- Easy-to-use &- Powerful
Easy-to-use:- Familiar interface: close to flowcharts- Dialog box transactions to stepwise specify client’s business process, and…
as a tacit side-effect of every step: safeguarding a technically correct business specification, allowing powerful automatic audit analytics “on-the-fly” & on the result
Powerful:- Correctness by Construction- Audit-specific Diagram Language
Engagement Level & Template Level
- Effective- Scalable- Cost-Efficient
ComputationalAuditing.com
Solution Approach10
Powerful system that supports practice and is founded in theory:- The world’s strongest Process-oriented Auditing Theory: Classical Dutch Auditing Theory- & Its Best-fitting rigorous Process Theory: Petri nets tailored for the Auditing Domain
Top Benefits
Major examples of Powerful Audit Analytics, impossible with old-style approaches:1. X-Raying a body of authorizations on immunity to major classes of fraud2. Deriving a model of enterprise-wide checks & balances,
basis for automatically generating executable scripts for data analysis tools3. Feasible: Petri net reachability analysis from initial to trial/final balance
Stringent Application of a Correct Systematic Approach: Clarifying & Refreshing
50% added value, E&Y
Typology with structured classification of audit approaches per type of industry
Proven in theory & practice
ComputationalAuditing.com
Agenda
Enterprise-level Process Documentationincorporating Automatic Audit Analytics
• Modern Auditing: Challenge, Criteria & Solution Approach
• What can you do with it? Examples of major analytics
• More on positioning this Doc Technique & Tooling
11
• What is it? How doc looks and what it actually is
• How to prepare it? Making doc in safeguarding tool
ComputationalAuditing.com
What is it? Elementary Trade Example
12
Top-down, Leveled Diagram
Enterprise-wide: Integral & Unifying
Static: State Balance ItemS
Dynamic: Transaction
Profit & Loss Item
T
Top-level is a Supercycle: one level up & connecting US cycles
200100
Normative (‘Soll’) & Representative (‘Ist’)
Mental Model =Executable Model
Flow of Money
Flow of Goods
ComputationalAuditing.com
What is it? Trade Diagram in detailed Audit Net 13
http://www.ComputationalAuditing.com/images/Kring.swf
1. Purchase2. Accept3. Sales4. Deliver & Collect5. Pay6. Collect
Process Steps
ComputationalAuditing.com
Auditing Laws of Starreveld & Frielink14
The computational interpretation of these Laws leads to the Audit Invariant: used as preventive safeguard
1. Law of Relation between Produced & Consumed
Illustrated by movie: A rational, normative relation between frequencies of business transactions in the supercycle and generated margin
2. Law of Relation between State & EventIllustrated by movie: BETA-equation for every State:End – Begin – Inflow + Outflow = 0, except Money > 0
ComputationalAuditing.com
CLASSIFICATION EXAMPLES
organizations with-out a technical trans-
trade organizations deliveringmainly to other industries
wholesalers, importers, exporters
formation process trade organizations deliveringmainly to final consumers
shops, retailers
industrial organiza-tions with homoge-
(flowingly) rotatinghomogeneous massproduction
gas-works, power stations, sugar-factories, oil refineries, paper-mills
neous mass produc-tion
(intermittently) par-celling homogene-ous mass production
brick-works, breweries, tanneries,lime-kilns, wire-drawing mills
industrialindustrial organiza-tions with heteroge-
singular heteroge-neous mass produc-tion
glass-works, potteries, wall-paperfactories, preserving factories
organizations neous mass produc-tion
compound heteroge-neous mass produc-tion
factories for: shoes, ready-madeclothing, audio devices, bicycles,cars
industrial organiza-tions with (serial)
(unique) pieceproduction
cloth tailoring, house building con-tractors, shipyards, engineeringworks
piece production serial pieceproduction
builders of: sisterships, ship mo-tors, railroad passenger cars
agrarian and extractive organizationsagriculture, animal husbandry,horti-culture, forestry, miningindustry, fishing industry
some flow of goodsowned by the orga-nization
pubs, coffeehouses, restaurants,publishers of newspapers
service organiza-tions with flow ofgoods
flow of goods own-ed by others
auctioneers, laundries, dye-works,repair-works, transporters, store-houses (goods)
service
delivery of goodsvia fixed pipes orwires (is: outflow)
gas, electricity and water suppliers,telephone exploiters, radio and tele-vision broadcasters,
organizations
service organiza-tions offering space-time capacity
specific reservationof space-time capa-city
house exploiters, hospitals, hotels,storehouses (see also above), trans-porters of passengers over relati-vely long distance (e.g. aviation,shipping)
unspecific reserva-tion of space-timecapacity (via quasi-goods, e.g. tickets)
entertainment providers, swimmingpools, theaters, transporters of pas-sengers over relatively short dis-tance (e.g. train, bus, taxi-cab)
other service organizations and professions(time capacity / number of performed tasks)
professional services, cleanupservices
banks general banks, savings banks,mortgage banks
financialinstitutions
special finance institutions venture capital companies,investment companies (trusts)
intermediates in stock exchange stockbrokersinsurance organizations life insurance and indemnity
insurance companies
organizations producing (and/oroffering services) directly for theirmembers, i.e. without mediation
governmental agencies and public corporatebodies (as excluded above)
government (central, provincial,municipal), public corporate bodies(possibly belonging to organiza-tions which produce for the market)
of the market private corporate bodies(as excluded above)
foundations, societies, religiouscommunities
Table 1: Auditee Typology
15
ComputationalAuditing.com
StarreveldAuditeeClassification
Based onRigor in theSupercycle
Audit Pack Platform
Drill-down tree with downloadable packs
Every node contains asupercycle pack &client-tailoring guidance
Uploader, downloader & broker
Client Side: “Information Rules”
Pack Trade
Roll Upward
Roll Forward
- Effective- Scalable- Cost-Efficient
Audit Pack Platform
Real softwareRelease 0.5April 2008
ComputationalAuditing.com
Agenda
Enterprise-level Process Documentationincorporating Automatic Audit Analytics
• Modern Auditing: Challenge, Criteria & Solution Approach
• What can you do with it? Examples of major analytics
• More on positioning this Doc Technique & Tooling
16
• What is it? How doc looks and what it actually is
• How to prepare it? Making doc in safeguarding tool
ComputationalAuditing.com
Qualitative Audit Analytics: Segregation of Duties (1 of 3) 17
50 600
5
2
3
60
10
5
300
15
40
5
S f
F m
F t
B f w
F m
B m fB f w
M fF m
F s
W m t
W m t
W m t
T m
F m b
F m s
Everything for SoD analysis
Real case:International Network of Accountants and Auditors, INAA, SRA
M: Majority Owner-ManagerS: Sales departmentB: Buy/Purchase departmentF: Financial administratorT: Technical staff managerW: Warehouse manager
Agent Legend
Capital: Authorization - Small: Ability
ComputationalAuditing.com
INA
A,
SR
A C
ase O
utp
ut:
Solo
-Fra
ud
Base
18Potential Solo-FraudQualitative Audit Analytics (2 of 3)
Con
cep
tual P
rim
itiv
es
Why is this class relevant?
ISA 240
Isn’t this only interesting for
SME?
ComputationalAuditing.com
19
Qualitative Audit Analytics - SoD (3 of 3)
X-Raying Segregation of Duties: Support to Illuminate an Enterprise’s Immunity to Solo-Fraud
UWCISA presentation on:http://artsms.uwaterloo.ca/accounting/uwcisa/symposium_2007/Program.htm
Paper with discussions and response, appearing in the International Journal of Accounting Information Systems, June 2008
ComputationalAuditing.com
Quantitative Audit Analytics: Check Model (1 of 5) 20
Real case:Ernst & Young
Everything for Check Model
225
25 200
225
500
25
25
1,000400
400100
20
20
20
20
500
400
Book & Course flow:1-1 normative
Materiality
Coverage of registration points
in SoD: S & T
Quantitatively motivated process decomposition
ComputationalAuditing.com
21Quantitative Audit Analytics: Enterprise-level Check Model, Output E&Y Case (2 of 5)
1. Debtors ‘+’ Deb : DebI (Sales)*1000 + DebB – DebE DebO (Collect)*40*25
2. Sales Fee ‘-’ sFee : sFeeO (GrantFee)*400 + sFeeE – sFeeB sFeeI
(Sales)*4003. Course Orders ‘-’ cOrd : cOrdO
(DeliverCourse) + cOrdE – cOrdB cOrdI (Sales)
4. Book Orders ‘-’ bOrd : bOrdO (DeliverBook) + bOrdE – bOrdB bOrdI
(Sales)5. Teacher Hours ‘+’ tHour : tHourI
(EmployTeacher)*20 + tHourB – tHourE tHourO (DeliverCourse)*20
6. Room Hours ‘+’ rHour : rHourI (RentRoom)*20 + rHourB – rHourE rHourO
(DeliverCourse)*207. Course Books ‘+’ Books : BooksI
(BuyBook) + BooksB – BooksE BooksO (DeliverBook)
8. Salaries ‘-’ Sal : SalO (PaySalaries)*500 + SalE – SalB SalI
((GrantFee)*400+(EmployTeacher)*100)9. Creditors ‘-’ Cred : CredO
(PayCreditors)*225 + CredE – CredB CredI ((BuyBook)*25+(RentRoom)*200)
10. Cash ‘+’ : CashI (Collect)*40*25 + CashB – CashE CashO
((PayCreditors)*225+(PaySalaries)*500)B : Beginning I : Inflow E : End O : Outflow
Spanning Reconciliation Checks
Asset (‘+’) Buffer: I + B - E = O Liability (‘-’) Buffer: O + E - B = I
Correctness = Isn’t it overstated? Completeness = Isn’t it understated?
Algebraic deduction
1st interpretation: Bold font = Completeness Regular font = Correctness
2nd interpretation: Bold font = Correctness Regular font = Completeness
1st interpretation: Completeness of stated debtor revenues Historical: owner-ordered audit
2nd interpretation: Correctness of stated debtor revenues Historical: management-ordered audit
Today: Management-ordered audit on behalf of both current (1st) and future (2nd) owners/shareholders
“Over-constrained”
ComputationalAuditing.com
22
Frielink et al
ClassicalDutchAuditingEducationLiterature
Three ExampleEnterprise-levelProcess CheckModels
QuantitativeAudit Analytics (3 of 5)
Auditor’s EvidenceAcquisition Strategy
-David Budescu, Mark
Peecher & Ira Solomon
ComputationalAuditing.com
23
Automatically generating executable scripts for data analysis tools
QuantitativeAudit Analytics (4 of 5)
Case provided by Tom Koning,author of: “The Auditor’s New Clothes”
ComputationalAuditing.com
Quantitative Audit Analytics: Reachability (5 of 5)24
A System of Spanning Reconciliation Checks, the Check Model, corresponds to the Flow Matrix of the normative Petri net
Petri Net Reachability Analysis from Initial to Trial/Final Balance goes a step further then detailed Spanning Reconciliation Checks by taking into account Time Stamps in Event Registrations
- Interrelating all buffer contents on a day-to-day basis - Reconciled with day-to-day external evidence- Shows deviations and associated risks Trial Balance
Spanning Reconciliation Checks can be applied in Totals or in Detail per parameter
ComputationalAuditing.com
Agenda
Enterprise-level Process Documentationincorporating Automatic Audit Analytics
• Modern Auditing: Challenge, Criteria & Solution Approach
• What can you do with it? Examples of major analytics
• More on positioning this Doc Technique & Tooling
25
• What is it? How doc looks and what it actually is
• How to prepare it? Making doc in safeguarding tool
ComputationalAuditing.com
Stringent application of correct systematic approach 26
Large model is built and used at Dutch Post Office
Guidance is:- Easy-to-use &- Powerful
Easy-to-use:- Familiar interface: close to flowcharts- Pop-up box transactions to stepwise specify client’s business process, and…
as a tacit side-effect of every step: safeguarding a technically correct business specification, allowing powerful automatic audit analytics
Powerful:- Correctness by Construction- Audit-specific Diagram Language
Engagement Level & Template Level
100 200
100 200
ComputationalAuditing.com
Agenda
Enterprise-level Process Documentationincorporating Automatic Audit Analytics
• Modern Auditing: Challenge, Criteria & Solution Approach
• What can you do with it? Examples of major analytics
• More on positioning this Doc Technique & Tooling
27
• What is it? How doc looks and what it actually is
• How to prepare it? Making doc in safeguarding tool
ComputationalAuditing.com
Con’s & Response
1 - Large model is cumbersome to make, making it only suitable for SME- A lot of information is required
- Reuse & extend already existing models- Gives good and visible foundation to opine upon, improving documentation quality & applicability
2 Only supercycle related, and not everything is in the supercycle
- ‘Type of industry’ is essential- A lot is attributable to the supercycle- Gives focus on determining normative relations
3 Support is too immature To be finalized for clients & content providing expert auditors
4 Normative gross margin is fixed - Qualitative: margin size has no influence on number and structure of pot. fraud constructs- Quantitative: tolerance is allowed, but leads to weaker numerical checks, to be compensated
5 Authorizations on:- Root data: price lists, employee lists...
- Filters in record keeping chain
Integrate these as ‘pre-processing’ transactions in client’s business model
28
ComputationalAuditing.com
Pro’s 29
“The stringent application of a correct systematic approach will without any doubt improve audit quality” A.B. Frielink, Lead author of Dutch Auditing literature, personal correspondence regarding the Computational Auditing thesis
- “Mapping out the supercycle is considered clarifying and refreshing: establishing a wider look than traditional cycles”
- “The schema technique is not too complex and can be well understood”
- “Guides the input preparation process by a systematic framework”
- “The support is feasible in practice” Hans Verkruijsse & team, Partner Ernst & Young, National Director Audit Technique, Evaluation report regarding the diagram technique and application for SoD analysisMore prominent references:
Hans Blokdijk, Emeritus Auditing Professor, ex-KPMG partnerRuud Veenstra, former Chairman of Deloitte NetherlandsHarold Kinds, National Director Audit Technique, INAA NetherlandsPeter Waas, National Audit Coordinator, Dutch Tax Office
ComputationalAuditing.com
Comparison
Audit-SpecificDiagram Language
Yasper/Prom
(Deloitte & TUE)Audit net
Editor
Criteria
Tool
+
Flowchartsoftware
30
Correctness byConstruction
Underlying Rigor
Deloitte’s Smart Audit Support
–
++
–+
+ +
–
+
ComputationalAuditing.com
31
ComputationalAuditing.com
Continuation
Correctness by Construction
Script Generator
Typology Platform
Supercycle
200100
You are an expert auditor?Why not have a facilitator to leverage
your guidance impact for your audience?
1. Smart Audit Planning Forms
2. Generating Checking Scripts
3. Smart Flowcharts
All Pack-based & Web-based