Management System Auditing by David Hoyle

7/24/2019 Management System Auditing by David Hoyle

1/35

David Hoyle 2012

Management System

AuditingAssessment of auditing methodsAssessment of auditing methods


2/35

David Hoyle 2012

Objective

To assess hethe! cu!!ent auditingTo assess hethe! cu!!ent auditingmethods ill hold bac" o! stimulatemethods ill hold bac" o! stimulate

develo#ments to $SO %001develo#ments to $SO %001


3/35

David Hoyle 2012

Audit methods

&lause based&lause based De#a!tment basedDe#a!tment based &ont!act based&ont!act based '!ocess based'!ocess based

(ehaviou! based(ehaviou! based

Select amethod that

fulfils you!objectives andmatches theinfo!mation and

!esou!cesavailable) *+AO,

Dete!mine hatyou a!e going to

-OO. AT andhen you find ithat you a!e

going to -OO./O

Alays "nohat you a!eattem#ting to

establish!elative to you!

objective


4/35

David Hoyle 2012

A##!oach

amine the methodamine the method $dentify assum#tions$dentify assum#tions valuate the methodvaluate the method


5/35

David Hoyle 2012

&lause based a##!oach$SO %001 Select clause of the standa!dSelect clause of the standa!d

&hoose de#a!tment&hoose de#a!tment

Assess confo!mity ithAssess confo!mity ith!e3ui!ements by!e3ui!ements by

As"ing sea!ching 3uestions of eachAs"ing sea!ching 3uestions of each#e!son#e!son

evealing evidence of confo!mity ithevealing evidence of confo!mity ithclauses of standa!dclauses of standa!d

'!oduce !e#o!t of nonconfo!mities'!oduce !e#o!t of nonconfo!mitiesagainst clausesagainst clauses

?

Audite#o!t

Objective

To dete!minethe etent tohich the 4MSconfo!ms to!e3ui!ements of$SO %001


6/35

David Hoyle 2012

Assum#tions De#a!tments a!e eclusively !es#onsible fo! meetingDe#a!tments a!e eclusively !es#onsible fo! meeting

ce!tain clausesce!tain clauses vidence of confo!mity in one De#t is indicative ofvidence of confo!mity in one De#t is indicative of

confo!mity in othe! De#tsconfo!mity in othe! De#ts

&onfo!mity ith clauses is evidence of ca#ability&onfo!mity ith clauses is evidence of ca#ability &o!!ecting non5confo!mity ill im#!ove system&o!!ecting non5confo!mity ill im#!ove system

effectivenesseffectiveness 'eo#le o#e!ate f!om the same causality as natu!e

and machines


7/35

David Hoyle 2012

valuation1)1) &onfo!mity6&onfo!mity62)2) $nhe!ent !is"s6$nhe!ent !is"s67)7) 'e!fo!mance6'e!fo!mance64.4. Capability?Capability?

8)8) fficiency6fficiency69)9) ffectiveness6ffectiveness6

:):) $m#!ovements6$m#!ovements6;);) &onfidence6&onfidence6%)%)


8/35

David Hoyle 2012

&lause based a##!oach$SO %001 Select clause of the standa!dSelect clause of the standa!d


Assess confo!mityAssess confo!mityith !e3ui!ements byith !e3ui!ements by

As"ing sea!ching 3uestions of eachAs"ing sea!ching 3uestions of each#e!son#e!son

evealing evidence of confo!mity ithevealing evidence of confo!mity ithclauses of standa!dclauses of standa!d


?

Audite#o!t

Objective



9/35

David Hoyle 2012

De#a!tment based a##!oach

Select clauses of the standa!dSelect clauses of the standa!dthat a##lythat a##ly


Assess confo!mityAssess confo!mityith !e3ui!ements byith !e3ui!ements by As"ing sea!ching 3uestions in each a!eaAs"ing sea!ching 3uestions in each a!ea /olloing a t!ail th!ough each/olloing a t!ail th!ough each

de#a!tmentde#a!tment evealing evidence of confo!mity ithevealing evidence of confo!mity ith

clauses of standa!dclauses of standa!d


$SO %001

Audite#o!t

?

Objective



10/35

David Hoyle 2012

Assum#tions &onfo!mity ith clauses is evidence&onfo!mity ith clauses is evidence

of ca#abilityof ca#ability

&o!!ecting non5confo!mity ill&o!!ecting non5confo!mity illim#!ove system effectivenessim#!ove system effectiveness 'eo#le o#e!ate f!om the same

causality as natu!e and machines

De#a!tments a!e eclusively !es#onsibleDe#a!tments a!e eclusively !es#onsiblefo! meeting ce!tain clausesfo! meeting ce!tain clauses

vidence of confo!mity in one De#t isvidence of confo!mity in one De#t isindicative of confo!mity in othe! De#tsindicative of confo!mity in othe! De#ts


11/35

David Hoyle 2012





12/35

David Hoyle 2012

De#a!tment based a##!oach

Select clause of the standa!dSelect clause of the standa!d


Assess confo!mityAssess confo!mityith !e3ui!ements byith !e3ui!ements by As"ing sea!ching 3uestions in each a!eaAs"ing sea!ching 3uestions in each a!ea /olloing a t!ail th!ough each/olloing a t!ail th!ough each

de#a!tmentde#a!tment evealing evidence of confo!mity ithevealing evidence of confo!mity ith

clauses of standa!dclauses of standa!d


$SO %001

Audite#o!t

?

Objective



13/35

David Hoyle 2012

&ont!act based a##!oachSelect !andom sam#le of cont!acts=o!de!s andSelect !andom sam#le of cont!acts=o!de!s andc!itical cha!acte!isticsc!itical cha!acte!istics

Assess confo!mity ithAssess confo!mity ithcont!acts and clauses bycont!acts and clauses by As"ing sea!ching 3uestions at each stageAs"ing sea!ching 3uestions at each stage /olloing a t!ail th!ough each #!ocess/olloing a t!ail th!ough each #!ocess

f!om in#ut to out#utf!om in#ut to out#ut evealing evidence of confo!mity ithevealing evidence of confo!mity ith

custome! !e3ui!ements and ith clausescustome! !e3ui!ements and ith clausesof standa!dof standa!d

'!oduce !e#o!t shoing the deg!ee'!oduce !e#o!t shoing the deg!eeof confo!mity ith !e3ui!ementsof confo!mity ith !e3ui!ements

Audite#o!t

?

&ont!act=O!de!

Obtain o! #!oduce flo cha!t of activitiesObtain o! #!oduce flo cha!t of activitiesf!om cont!act=o!de! to fulfilmentf!om cont!act=o!de! to fulfilment

$SO%001

Objective

To dete!minehethe! the o!gani>ationhas the ability toconsistently #!ovide#!oduct that meetscustome! !e3ui!ements


14/35

David Hoyle 2012

Assum#tions

&onfo!mity ith clauses is evidence of&onfo!mity ith clauses is evidence ofca#abilityca#ability

&onfo!mity ith $SO %001 ill enable&onfo!mity ith $SO %001 ill enableconsistent #!ovision of confo!mingconsistent #!ovision of confo!ming#!oducts#!oducts

&o!!ecting non5confo!mity ill im#!ove&o!!ecting non5confo!mity ill im#!ove

system effectivenesssystem effectiveness 'eo#le o#e!ate f!om the same causality as

natu!e and machines


15/35

David Hoyle 2012





16/35

David Hoyle 2012

&ont!act based a##!oachSelect !andom sam#le of cont!acts=o!de!s andSelect !andom sam#le of cont!acts=o!de!s andc!itical cha!acte!isticsc!itical cha!acte!istics

Assess confo!mity ithAssess confo!mity ithcont!acts and clauses bycont!acts and clauses by As"ing sea!ching 3uestions at each stageAs"ing sea!ching 3uestions at each stage /olloing a t!ail th!ough each #!ocess/olloing a t!ail th!ough each #!ocess

f!om in#ut to out#utf!om in#ut to out#ut evealing evidence of confo!mity ithevealing evidence of confo!mity ith

custome! !e3ui!ements and ith clausescustome! !e3ui!ements and ith clausesof standa!dof standa!d

'!oduce !e#o!t shoing the deg!ee'!oduce !e#o!t shoing the deg!eeof confo!mity ith !e3ui!ementsof confo!mity ith !e3ui!ements

Audite#o!t

?

&ont!act=O!de!

Obtain o! #!oduce flo cha!t of activitiesObtain o! #!oduce flo cha!t of activitiesf!om cont!act=o!de! to fulfilmentf!om cont!act=o!de! to fulfilment

$SO%001

Objective

To dete!minehethe! the o!gani>ationhas the ability toconsistently #!ovide#!oduct that meetscustome! !e3ui!ements


17/35

David Hoyle 2012

So hat?s the #!oblem6

'eo#le don?t o#e!ate f!om the samecausality as natu!e and machines

@e thin" e can design o!gani>ationsin the same ay e design machines

A management system is not a set of

documents


18/35

David Hoyle 2012

O!gani>ationis a closed

system

O!gani>ationSystem of effectivelymanaged #!ocessesDesi!ed Out#uts

SatisfiedSta"eholde!s


19/35

David Hoyle 2012

eality chec"

&onfo!mity is only #a!t of the #ictu!e&onfo!mity is only #a!t of the #ictu!e 'eo#le ma"e choices'eo#le ma"e choices

'eo#le a!e influenced by images events and'eo#le a!e influenced by images events andthei! inte!action ith othe!sthei! inte!action ith othe!s

Many invisible !is"s a!e hidden in behaviou!Many invisible !is"s a!e hidden in behaviou!

The im#act of behaviou! is hat effectsThe im#act of behaviou! is hat effectsoutcomes and ca#abilityoutcomes and ca#ability


20/35

David Hoyle 2012

Alte!native A##!oaches

'!ocess management audit'!ocess management audit (ehaviou! assessment(ehaviou! assessment


21/35

David Hoyle 2012

'!ocess based a##!oach

$dentify #!ocesses and sub5#!ocesses that$dentify #!ocesses and sub5#!ocesses thatachieve these objectivesachieve these objectives

Assess #!ocess effectiveness byAssess #!ocess effectiveness by

'!oduce !e#o!t shoing the'!oduce !e#o!t shoing the

effectiveness of the system ofeffectiveness of the system of!ocesses!ocesses

Audite#o!t

?

$dentify o!gani>ation?s mission objectives and$dentify o!gani>ation?s mission objectives andsuccess measu!essuccess measu!es

$SO%001

Objective

To dete!mine hethe!the o!gani>ation?s#!ocesses a!e beingmanaged effectively

MissionObjectivesMeasu!es

As"ing sea!ching 3uestions at eachAs"ing sea!ching 3uestions at eachstagestage

evealing ho #!ocesses a!e beingevealing ho #!ocesses a!e being

managedmanaged evealing evidence of ca#abilityevealing evidence of ca#ability

against objectives and measu!esagainst objectives and measu!es


22/35

David Hoyle 2012

A '!ocess Audit

'!ocessobjectives BMeasu!es

$m#!oved

effectiveness

'!ocess

im#!ovement

'!ocess !evie

'!ocess

monito!ing

$m#!oved

#e!fo!mance

$m#!oved

efficiency

$n#uts esou!ces

T

4

T

T

S

Out#uts

Demands

SatisfiedDemands'!oduce

'!oductDelive!'!oduct

Su##o!t'!oduct

'lan'!oduction

Ho do you"no you a!e

doing it !ight6

Ho do you "no

you a!e doing itin the best ay6

Ho do you"no you a!e

doing the !ightthing6

@hat a!e you t!yingto do6

B@hat ill success

loo" li"e6

Ho do youma"e it ha##en6


23/35

David Hoyle 2012

Assum#tions

The system as !evealed th!oughThe system as !evealed th!oughsystematic en3ui!y is the system that issystematic en3ui!y is the system that is

#!oducing the !esults#!oducing the !esults The info!mation #!esented by theThe info!mation #!esented by the

o!gani>ation is legitimate and has not beeno!gani>ation is legitimate and has not been

fab!icatedfab!icated


24/35

David Hoyle 2012

valuation1)1) &onfo!mity6&onfo!mity62)2) $nhe!ent !is"s6$nhe!ent !is"s67)7) 'e!fo!mance6'e!fo!mance64.4. Capability?Capability?8)8) fficiency6fficiency69)9) ffectiveness6ffectiveness6

:):)

$m#!ovements6$m#!ovements6;);) &onfidence6&onfidence6%)%)


25/35

David Hoyle 2012

'!ocess based a##!oach

$dentify #!ocesses and sub5#!ocesses that$dentify #!ocesses and sub5#!ocesses thatachieve these objectivesachieve these objectives

Assess #!ocess effectiveness byAssess #!ocess effectiveness by

'!oduce !e#o!t shoing the'!oduce !e#o!t shoing the

effectiveness of the system ofeffectiveness of the system of!ocesses!ocesses

Audite#o!t

?

$dentify o!gani>ation?s mission objectives and$dentify o!gani>ation?s mission objectives andsuccess measu!essuccess measu!es

$SO%001

Objective

To dete!mine hethe!the o!gani>ation?s#!ocesses a!e beingmanaged effectively

MissionObjectivesMeasu!es

As"ing sea!ching 3uestions at eachAs"ing sea!ching 3uestions at eachstagestage

evealing ho #!ocesses a!e beingevealing ho #!ocesses a!e being

managedmanaged evealing evidence of ca#abilityevealing evidence of ca#ability

against objectives and measu!esagainst objectives and measu!es


26/35

David Hoyle 2012

(ehaviou! based a##!oach

$dentify a !ange of behaviou!s as outcomes$dentify a !ange of behaviou!s as outcomesthat !eflect levels of matu!ities and c!ossthat !eflect levels of matu!ities and c!oss!ef clauses and a##ly eighting!ef clauses and a##ly eighting

'!oduce !e#o!t shoing sco!ed !is"s to'!oduce !e#o!t shoing sco!ed !is"s tothe achievement of objectives andthe achievement of objectives and

d!ive!sd!ive!s

Audit

e#o!t

Matu!ityC!id

CoalsObjectives

D!ive!s

$dentify the system objectives and$dentify the system objectives and

#e!fo!mance d!ive!s#e!fo!mance d!ive!s

$SO%001

Objective

To measu!e the

inhe!ent level of !is"to o#timum businessoutcomes andgove!nance issuescaused by the im#actof eve!yday #atte!nsof behaviou!

$dentify #eo#le ho e#e!ience hat is$dentify #eo#le ho e#e!ience hat isha##eningha##ening

$nvite on5line$nvite on5line 790790 #a!tici#ation#a!tici#ationbased on the #a!t they #lay Ebased on the #a!t they #lay E

confidentialconfidentialun analysis engine to com#ute !esultsun analysis engine to com#ute !esults

Mainframe


27/35

David Hoyle 2012

/ulfilment #!ocess !esults *#a!t,

FFOutcomeOutcome Av Sco!eAv Sco!e

11 We know the objectives that have to be achieved and how ourWe know the objectives that have to be achieved and how our

performance will be measuredperformance will be measured3.!3.!

22 We know the activities that need to be carried out to achieve theWe know the activities that need to be carried out to achieve theobjectivesobjectives

3".#3".#

77 $he necessary physical and human resources are provided when$he necessary physical and human resources are provided when

neededneeded

%&.%%&.%

GG 'rovisions made to minimi(e risk are successful'rovisions made to minimi(e risk are successful %&.!%&.!

88 Work commences on timeWork commences on time 3!.#3!.#

99 Work is e)ecuted in accordance with policies and plansWork is e)ecuted in accordance with policies and plans 3*.!3*.!

:: Work flows without unplanned interruptionWork flows without unplanned interruption 4".+4".+

;; When thin,s ,o wron, we put them ri,htWhen thin,s ,o wron, we put them ri,ht 3.33.3

%% -o work is released until found in conformity with all reuirements-o work is released until found in conformity with all reuirements 43.443.4

1010 Outputs delivered on timeOutputs delivered on time %+.*%+.*


28/35

David Hoyle 2012

Matu!ity C!id fo! outcome ;OutcomeOutcome @hich statement matches you! e#e!ience the closest6@hich statement matches you! e#e!ience the closest6

@hen@hen

things gothings go

!ong e!ong e

#ut them#ut them!ight)!ight)

@hen things@hen things

go !ong ego !ong e

#ut them#ut them

!ight as best!ight as beste can ande can and

get on ithget on ith

the jobthe job

@e no!mally@e no!mally

co!!ectco!!ect

mista"es asmista"es as

they occu!they occu!folloing ou!folloing ou!

#!ocedu!es#!ocedu!es

but ebut e

ouldn?touldn?t

!eco!d these!eco!d these

eco!dseco!ds

sho ou!sho ou!

#!ocedu!es#!ocedu!es

have beenhave beena##lied toa##lied to

co!!ectco!!ect

mista"esmista"es

andand

#!event#!eventthemthem

ha##eningha##ening

again)again)

@e usually@e usually

!evie all!evie all

mista"esmista"es

fo!mally info!mally inline ith ou!line ith ou!

#!ocedu!e#!ocedu!e

and ta"eand ta"e

effectiveeffective

action toaction to#!event#!event

themthem

!ecu!!ing!ecu!!ing

eco!ds ofeco!ds of

all mista"esall mista"es

sho thatsho that

it is !a!eit is !a!efo! thefo! the

samesame

mista"e tomista"e to

ha##enha##en

ticetice

-evel-evel 0)000)00 1)001)00 2)002)00 7)007)00 8)008)00

&lauses&lauses

lin"edlin"ed

;)8)2 &o!!ective action;)8)2 &o!!ective action

G)2)G &ont!ol of !eco!dsG)2)G &ont!ol of !eco!ds


29/35

David Hoyle 2012

C!a#hical e#!esentation

G

9

(y clause

G 4uality management system8 Management !es#onsibility9 esou!ce management: '!oduct !eali>ation; Measu!ement analysis B

im#!ovement

;

:

8

20

40

(y sta"eholde!

1 '!ocess one!2 '!ocess o!"e!s7 '!ocess su##lie!G '!ocess custome!

2

7

G0

20

1

G

(y business #!ocess

G0

20

1

G

1 Mission management2 esou!ce management7 Demand c!eationG Demand fulfilment

7

2


30/35

David Hoyle 2012

Assum#tions

The system as !evealed th!ough systematicThe system as !evealed th!ough systematicen3ui!y is the system that is #!oducing theen3ui!y is the system that is #!oducing the!esults!esults

'atte!ns of behaviou! a!e lead indicato!s'atte!ns of behaviou! a!e lead indicato!sof !is"of !is"

&o!!elation beteen clauses and business&o!!elation beteen clauses and business

d!ive!s !elative to #e!fo!manced!ive!s !elative to #e!fo!mance The statements don?t #!oduce ildlyThe statements don?t #!oduce ildlydiffe!ent inte!#!etationsdiffe!ent inte!#!etations


31/35

David Hoyle 2012

valuation1)1) &onfo!mity6&onfo!mity62)2) $nhe!ent !is"s6$nhe!ent !is"s67)7) 'e!fo!mance6'e!fo!mance64.4. Capability?Capability?8)8) fficiency6fficiency69)9) ffectiveness6ffectiveness6:):)

$m#!ovements6$m#!ovements6;);) &onfidence6&onfidence6%)%)


32/35

David Hoyle 2012

(ehaviou! based a##!oach

$dentify a !ange of behaviou!s as outcomes$dentify a !ange of behaviou!s as outcomesthat !eflect levels of matu!ities and c!ossthat !eflect levels of matu!ities and c!oss!ef clauses and a##ly eighting!ef clauses and a##ly eighting

'!oduce !e#o!t shoing sco!ed !is"s to'!oduce !e#o!t shoing sco!ed !is"s tothe achievement of objectives andthe achievement of objectives and

d!ive!sd!ive!s

Audit

e#o!t

Matu!ityC!id

CoalsObjectives

D!ive!s

$dentify the system objectives and$dentify the system objectives and

#e!fo!mance d!ive!s#e!fo!mance d!ive!s

$SO%001

Objective

To measu!e the

inhe!ent level of !is"to o#timum businessoutcomes andgove!nance issuescaused by the im#actof eve!yday #atte!nsof behaviou!

$dentify #eo#le ho e#e!ience hat is$dentify #eo#le ho e#e!ience hat isha##eningha##ening

$nvite on5line$nvite on5line 790790 #a!tici#ation#a!tici#ationbased on the #a!t they #lay Ebased on the #a!t they #lay E

confidentialconfidentialun analysis engine to com#ute !esultsun analysis engine to com#ute !esults

Mainframe


33/35

David Hoyle 2012

&onfidence level&lause based&lause based +ot fo! 7+ot fo! 7!d!d'a!ty Audits'a!ty Audits

De#a!tment basedDe#a!tment based +ot fo! 7+ot fo! 7!d!d'a!ty Audits'a!ty Audits

&ont!act based&ont!act based Acce#table fo! 2Acce#table fo! 2ndnd'a!ty'a!ty

AuditsAudits

'!ocess based'!ocess based

Acce#table fo! 7Acce#table fo! 7!d!d'a!ty Audits'a!ty Audits

(ehaviou! based(ehaviou! basedAcce#table hen combinedAcce#table hen combinedith othe! audit methodsith othe! audit methods


34/35

David Hoyle 2012

&onclusion@hich methods ill hold bac" o!@hich methods ill hold bac" o!

stimulate develo#ments to $SO %0016stimulate develo#ments to $SO %0016

&lause based&lause based Hold bac"Hold bac"

De#a!tment basedDe#a!tment based Hold bac"Hold bac"

&ont!act based&ont!act based Hold bac"Hold bac"

'!ocess based'!ocess based StimulateStimulate

(ehaviou! based(ehaviou! based StimulateStimulate


35/35

David Hoyle 2012

(ehaviou! Assessment contact

ian)!osamthe5h#o)com

'!ocess Audit contact

hoylet!ansition5su##o!t)com

Than" you and have a safe jou!ney home

/o! /u!the! DetailsI
mailto:[email protected]:[email protected]:[email protected]:[email protected]

Documents

Management System Auditing by David Hoyle