ARCcompliantEAmaturityAppraisal

8/2/2019 ARCcompliantEAmaturityAppraisal

1/32

Enterprise Architecture Maturity Assessment- An ARC-compliant Approach

Amit Bhagwat

NDIA 7t h Annual CMMI Technology Conference& User Group:12-15 Nov 2007


2/32

2 Amit Bhagwat amit_buk-ndia1107[at] yahoo[dot ]co[dot ]ukNDIA 7th Annual CMMI Technol ogy Conference& User Group:12-15 Nov 2007

Agenda

Introduction to the Presenter Context of the Appraisal

Appraisal Design

Appraisal Execution

Appraisal Results Presentation (illustrations)

Concluding Remarks


3/32


The Presenter

Current ly Consult ing Deputy to CTO @ a City of London

Institution leading in Europe in its specialty Enterpr ise Archit ect ure Pract ice Lead Responsible for corporat e IT Policy def init ion

Background Key roles

Chief Designer / Design Authority System Engineering & Enterprise Architecture

Chief / Lead / Principal St rategist Mentor Auditor

CMM-CMMI adopt ion background in high-maturit y (3-5) organizat ions

Predominantly in India Worked on / advised resources in a number of Public sector programs (principally in the UK) Commercial sector (principally financial and ancillary services)

Regular and recognized speaker and wr it er on Enterprise Archit ect ure andSyst ems Engineering St rategy


4/32


The Context

I was approached By a leading IT research and st rat egy consult ing f irm In early 2007 As EA st rat egist , TOGAF expert and audit or aware of t he concepts

of Capability and Maturity To plan and lead Archit ect ure Mat urit y Appraisal in a large UK

publ ic sect or organizat ion The organizat ion

Endeavored to follow TOGAF as it s EA f ramework

Had heard the sound of CMM Was aware of t he Archit ect ure-CMM referred in context of US DoC and

some state government bodies


5/32


The Appraisal Team

Self (acted as Lead Appraiser)

A Subject Matter Manager from the research and consultancyorganizat ion (referred to as the Director)

A Sector Lead from the research and consultancy organization(referred to as t he Managing Part ner - MP)

Quality Assurance Manager of the public sector organization,deputy to the CTO

A senior Enterprise Architect with consulting and SI background,

freshly recruited in t he public sector organizat ion


6/326 Amit Bhagwat amit_buk-ndia1107[at] yahoo[dot ]co[dot ]ukNDIA 7th Annual CMMI Technol ogy Conference& User Group:12-15 Nov 2007

Key Stakeholders

CIO

CTO

Funct ional Unit Heads (DCEO)

Funct ional Units IT Heads (DCIO)


7/327 Amit Bhagwat amit_buk-ndia1107[at] yahoo[dot ]co[dot ]ukNDIA 7th Annual CMMI Technol ogy Conference& User Group:12-15 Nov 2007

Background Work

DoC and other Architecture CMM(?) approaches (including thoseobtained from three leading IT research and strategy consulting

organizations) Had EA-relevant subject l ist (l ike Process Areas) Had a subject -based capabil i t y rat ing scale Individual rat ings were associat ed wit h adj ect ives (l ike CMMI levels)

However Rat ings were subject ive

Did not look for PIIs

Concept of Mat urit y as an evolut ionary plat eau building on charact erist icsof lower levels was absent

It was usually eit her average or weighted average of subject scores Dependence of subj ect s (PAs) on other subj ect s was not considered, t houghusual ly obvious t o EA pract it ioners

Scores (usually being averages) ended up being f ract ional t hus not givingany real meaning or clear adject ive, t o focus improvement


8/328 Amit Bhagwat amit_buk-ndia1107[at] yahoo[dot ]co[dot ]ukNDIA 7th Annual CMMI Technol ogy Conference

& User Group:12-15 Nov 2007

Appraisal Purpose Stakeholders View

To Underst and st rengt h and weakness across areas of t he EA pract ice

Obt ain advice on EA st rat egy Plan EA improvement over t he next 12 months Be able to make year-on-year comparison Gain great er sel f -conf idence in operat ions and del ivery

Not to Be indust ry-wide exemplar Serve as prest ige point for t he st akeholders Sat isfy specif ic indust ry assessment f ramework

Achieve / exhibit credibil i t y in ot hers eyes




Appraisal Design Key Decisions

Make Appraisal Obj ect ive Record Type A, and Type B or C PIIs

Decide relevant Subj ects (PAs) t hrough consensus among appraisers and

stakeholders (with subj ect mat ter background) and present individualsubject rating

Decide relevant PA capability levels and associate each level withmeaningful adject ive through consensus among appraisers and

stakeholders (with subj ect mat ter background) Where meaningful, use adject ives/ levels used in t he wider indust ry

Create lucid practice statements for PAs to correlate with capabilitylevel adjectives

Design an appraisal framework t hat a competent team of appraisers canreliably repeat to obtain results that may be compared

Keep ARC 1.2 compliance in mind in Appraisal Method Design




Appraisal Execut ion Key Decisions

A PA is deemed to be at a capability level when practice statementsassociated with that level and all the levels below are remarked to be

t rue and veri f ied as such by all relevant samples appraised E.g. Business unit cust omers groups could be spared providing t ype A PII onAppli cat ions Archit ect ure Development , but not on Archit ect ure Communicat ion

Cover dif ferent groups (Business Units IT Funct ions, IT St rategyFunct ion, IT Common Services Funct ion, Business Unit Customers)

adequately and provide their trends Provide PA-based capabil it ies in the form of relevant adject ives, not ing

key weaknesses (and improvement recommendat ions) for each PA

Conduct appraisal with rigor of ARC Class B, or greater

Typical t eam size 4 (minimum 2) Evidence as t ype A and B or C All organizat ion unit s in cont ext covered




Implementat ion - PAs

Process Areas / Subjects (inspired f rom various federal sources,

including DoC, and TOGAF) Archit ect ure Process

Archit ect ure Development

Business Linkage

Senior Management Involvement

Operat ing Unit Underst anding and Acceptance of EA

EA Consist ency, Representat iveness, Cont ribut ion across Operat ing Unit s Archit ect ure Communicat ion seeded t hrough Process and Framework

Documentat ion

Archit ect ure Communicat ion act uat ed t hrough Passive Broadcast ing Mechanisms

Act ive dif fusion of EA ideas t hrough Educat ion and Communit ies

IT Securit y

EA Governance

IT Invest ment and Acquisit ion St rategy

IT Transformat ional Governance - Impact , Change and Migrat ion




Implementat ion - Levels

Levels and their adj ectives (inspired from various federal sources,

including DoC, and TOGAF) 0 = None

1 = Init ial

2 = Under Development

3 = Defined

4 = Managed

5 = Measured (t herefore Quant i t at ively Managed and Predictable important qual if iers,

else Measured can be vague or misinterpreted) 6 = Opt imizing


13/32


Implementation

Each interviewee is shown the statements characterizing

Practices Implementation for a PA and asked to state Which st at ements are t rue at t he moment , provide evidence

Statements are so worded that the next level statement would not be true

wit hout the previous level

What subset of consecut ive pract ice st at ement s would t hey l ike (and

real ist ical ly consider possible) t o be t rue in the next 12 months, how What priority (on a scale of 0-5, 5 being highest priority) would they give

t o improvement work on a process area in the next 12 months (t his

answer was also normal ized for relat ive priorit y, un-normal ized answers

gave relat ive import ance across groups)


14/32


Procedure

Pract ice statements were rated for their implementat ion as: Ful ly Implemented (FI)

Direct artifacts present and appropriate Support ed by indirect art if act and/ or aff irmat ion

No weaknesses not ed Largely Implemented (LI) Direct artifacts present and appropriate Support ed by indirect art if act and/ or aff irmat ion One or more weaknesses noted

Part ial ly Implement ed (PI) Direct art ifacts absent or j udged inadequate Artifacts or affirmations indicate some aspects of the practice are implemented

One or more weaknesses noted Not Implemented (NI)

A capability level was considered achieved when all Practice statementsunder that Process Area that represented all levels, up to and including thatlevel, were fully implemented

A level was considered achieved conditionally when all Practice statementsunder that Process Area that represent all levels, up to and including thatlevel, were fully or largely implemented; the condition being thatweaknesses, noted in the verdict Largely Implemented, were eliminated


15/32

Results

Presentation Approach

(All examples used here are illustrative and do not

reflect actual results which are confidential)


16/32


Results were presented as

Grouped, PII-backed, averaged perception, aspiration and priority This was present ed as indicat or of self -discovery This was emphasized as

Only to be used for f inding t rends and level of self -understanding

Averaged fract ional numbers not t o be mistaken for fuzzy capabil it y level

PII-backed verdict on capability of each PA along with summary of keyweaknesses w.r.t . achieving next capabilit y level, as presented by SMEappraisers

Dependency map across PAs A verdict on overall maturity, taking into account capability scores and

PA dependencies

Detailed improvement recommendations and priorities for each PA

A roadmap detailing temporal progression through improvements acrossvarious aspects of PA

Detailed steps for each roadmap section

Recommendat ions on future EA appraisal


17/32


Grouped, PII-backed, averaged perception,aspirat ion and prior ity (il lust rat ion, not actual)

Averaged Scores

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

Architecture Process

Architecture Development

Business Linkage


Operating Unit Understanding and Acceptance of EA

EA Consistency, Representativeness, Contribution across

Operating Units

Architecture Communication seeded through process and

framework documentation

Architecture Communication actuated through Passive Broadcasting

Mechanisms

Active diffusion of EA ideas through Education and Communities

IT Security

Governance

IT Investment and Acquisition Strategy

Impact, Change and Migration

Averaged evidence-supported perception

Averaged aspiration for next year

Averaged Priority


18/32


Perception by Groups (il lust rat ion, not actual)

Average Perceived Scores by Groups

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00



Business Linkage




Operating Units



Architecture Communication actuated through Passive

Broadcasting Mechanisms


IT Security

Governance



X

Y

Z


19/32


Aspirat ion by Groups (il lust rat ion, not actual)

Average Aspired Scores by Groups

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00



Business Linkage




Operating Units


framew ork documentation




IT Security

Governance



X

Y

Z


20/32


Prior ity by Groups (il lust rat ion, not actual)

Average Perceived Priorities by Groups

0.00

0.50

1.00

1.50

2.00

2.50

3.00



Business Linkage




Operating Units






IT Security

Governance



X

Y

Z


21/32

Results

Substant iated Levels

(il lust rat ion, not actual)


22/32


PIIs Indicate(il lust rat ion, not actual)

Process Area Verdict Most Notable Aspect toFocus on

Architecture ProcessDefined, withweaknesses Process flows, client group processes

Archit ecture Development Under Development Requirements

IT SecurityDefined, withweaknesses Staffing and cross-unit dialogue

GovernanceDefined, withweaknesses Make policies and dispensation transparent

IT Investment and Acquisition Strategy Under Development Business process analysis, port folio approach

Impact, Change and Migrat ion Initial Role definition and Infrastructure


23/32

Results

Process Areas



24/32


Architecture Process(il lust rat ion, not actual)

Key weaknesses Whereas a f ramework and compliance st ruct ure exist s, t he f ramework is

not necessari ly real ized / ext ended into each business unit Document at ion wordy and representat ions Lego-based, t hus l imit ed senseof int eract ion / process

Areas to focus on through the next 12 months Go beyond Architecture building block blueprint and try to modelfunct ional f lows in and across business unit syst ems

Creat e more specif ic checkpoints as part of t he compliance regime to helpbusiness unit s deli ver on non-f unct ional requirement s. Consider developing

non-functional specializations (similar to security, already in place thoughunder-resourced) Work wi t h business and associated cl ient IT groups t o model business and

ident ify pat t erns in business processes Develop ful l -f ledged archit ect ure process relevant t o specif ic business unit s

while keeping a mapping wit h the EA f ramework


25/32


Architecture Development(il lust rat ion, not actual)

Key weaknesses The axel of t he ADM wheel requirement (def init ion, management ,

development ) i s weak and does not always l ink to Archit ect ureDevelopment

Areas to focus on through the next 12 months Requirement Management (granularit y, quant if icat ion,traceability, hierarchy, dependency)

Requirement t o Archit ect ure Linkage Requirement modeling & development

Requirement t o Governance Linkage


26/32


Impact , Change and Migrat ion (il lust rat ion, notactual)

This area is at Initial level, thus weak overall

Areas to focus on through the next 12 months Define roles associat ed wi t h Configurat ion and Change Management Est ablish Impact assessment (which may involve ADM for

archit ect urally signif icant changes) and CCM processes Est ablish dependable CCM infrast ruct ure Identify configurable items


27/32

Results

Roadmap & Priorities



28/32


Roadmap (il lust rat ion, not actual)

Understand and engage more closelywith the Business

Manage, relate, quantify anddevelop Requirements

Build Configuration and Change

Management capability

Build ADM

Build strategic capability within the team and benchmark it periodically

Broaden governance beyondConsent and Compliance

Build Non-functional specialities

Attain role consistency /mapping

Way Forward E.g.

U d t d d E l l


29/32


Understand and Engage more closelywith t he Business (il lust rat ion, not actual)

Observat ions The assessment indicat es signif icant change in percept ion across groups (X,

Y, Z) Business is deeply federated and suscept ible to changes effect ed by volat i lepol it ical wil l

Archit ect ure groups are t hin on business / syst em behavior engineering andNFRs

Archit ect ure t eams are business-enablers, but can lead st ruct ured analysis

Actions Engage more closely wit h t he business on Archit ect ure Communicat ion,

including Educat ion and Communit ies Look for pat t erns in business processes and be wil l ing t o develop cloned

services owned by business unit s (where budget wi l l come more readily for

t he prototype), rather than common services with central ownership


30/32


Business Architecture (il lust rat ion, not actual)

Consider How t he business (or various business l ines) i s used by it s user What improvements / addit ional facil i t ies are likely in t his usage

Define and own j oint ly with the Business Key business abst ract ion boundaries, ent it ies, processors Business scenarios Business pat t erns

Business component & services

Analyze How what t he business is for, maps t o how i t does what i t does

Consider with the Business scope for Business Reengineering Business Aut omat ion Business process Reusabi l i t y Improved Governance


31/32


Conclusion

Effectiveness of Appraisal depends on Clear obj ect ives communicat ed and owned across t he organizat ion

being appraised Clarity, sufficiency of definition and repeatability of the appraisal

mechanism Object ivit y coupled wit h pragmat ism

Dependencies across Process Areas and Pract ices and theircumulative effect taken into account Result s present ed wi t h dif ferent perspect ives, purposes and

granularity Result s fed into st rat egy and reflect ed in improvement roadmap


32/32

32NDIA 7th Annual CMMI Technol ogy Conference

Comments and Questions

Get in touch amit _buk-ndia1107[at ]yahoo[dot ]co[dot ]uk +447886782022, +447773364043

Documents

ARCcompliantEAmaturityAppraisal