BiZZdesign Enterprise Architecture Capability Assessments:

Get a Grip on Your EA Practice!

Marc LankhorstApril 17, 2014

About Marc Lankhorst

• Enterprise Architecture Service Line Manager @ BiZZdesign– Managed the development of the ArchiMate®

standard

– TOGAF® and ArchiMate® certified

– Board member of Netherlands Architecture Forum

• Some publications:– Enterprise Architecture at Work (2005-2012),

the book on ArchiMate

– Agile Service Development (2012)

• m.lankhorst@bizzdesign.com

2

BiZZdesign: Integrated Service Offering

Enterprise Architecture Management

Business Modeling

Lean & Business Process

Management

Consulting services

Tooling

Training

Best practices

3

BiZZdesign Worldwide: Offices and Partners

BiZZdesign officesOfficial BiZZdesign resellers

Some Clients Worldwide

5

Delivering Business Value

• Enterprise architecture MUST deliver business value and do it now

• Companies understand the need for architecture in general, but what are the specific benefits?

• Architecture processes are sometimes perceived as slow and bureaucratic

• Architects are often insufficiently connected to strategic investment decisions on the one hand, and realization processes on the other

• Architects have difficulty expressing their added value

6

The Problem: The Value of EA

• How effective is your enterprise architecture capability?

• What is the business value it delivers?

• What should the next steps in its development be?

7

EA Adds Value in Context

Enterprise Architecture

8

EA Adds Value in Context

Enterprise Architecture

9

Approach: An EA Assessment

1. Enterprise architecture Capability quick scan– Rapid, high-level assessment of your architecture capability

2. Enterprise architecture Realization assessment– Provides thorough insight into effectiveness of your EA practice

– Based on principles of COBIT governance framework

– Aligned with the phases of TOGAF’s Architecture Development Method

3. Enterprise architecture Value assessment– Insight into the (actual and perceived) business value offered

by your enterprise architecture practice

– Based on Balanced Scorecard

10

Assessment Benefits

• Independent review of the effectiveness and results of your architecture practice

• Concrete guidance for improvement, focused on the most valuable and urgent steps

• Based on BiZZdesign’s extensive experience and broad knowledge of the architecture practice

11

ENTERPRISE ARCHITECTURE CAPABILITY QUICK SCAN

12

Assessing the Enterprise Architecture Capability

• Rapid assessment of the EA capability– Quickscan, to gain first important insights in status and issues

– Using the familiar 5 levels of maturity

• EA cannot add value on its own, but functions in collaboration with:– IT governance: EA must fit within the governance structures of

the organization

– Project portfolio management: EA realizes the enterprise vision via projects and programs

13

Approach & Effort

1. Interviews with experts from enterprise architecture, portfolio management and IT governance– Using an assessment instrument with a structured questionnaire

of some 70 questions

2. Collating and analyzing findings– Intermediate report to interviewees

3. Final report and presentation to management

• Effort:– Lead time: 1-3 weeks, depending on availability of interviewees

– Typical effort: 2-6 days, depending on nr. of interviews

14

Example Questions

Awareness and CommunicationIs the need for having an EA recognized and supported by business and IT management at relevant organizational levels?

Responsibility and AccountabilityIs the responsibility for the architectural fit of projects with the EA explicitly assigned?Does each IT system have a business owner?

Goal Setting and Measurement

Are the goals, objective and scope of the EA program clear and well-defined?Is the contribution of the architecture function measured and used for improving its working practices?

Policies, Standards and ProceduresIs the architectural compliance of projects explicitly managed, e.g. via building permits?

Tools and AutomationDo the architects actively seek and employ international standards and best practices for their way of working, methods and tools?

15

Maturity Levels

• Level 1: Initial– Starting point, need is recognized

• Level 2: Managed– People are starting to take responsibility

• Level 3: Defined– Standardized ways of working

• Level 4: Quantitatively Managed– Measuring the outcomes

• Level 5: Optimizing– Forward-looking approach, continuous improvement of

practices

16

Example Outcome

ITGovernance

PortfolioManagement

EnterpriseArchitecture

Integration

Awareness andCommunication

Responsibilityand

Accountability

Goal Settingand

Measurement

Policies,Standards and

Procedures

Tools andAutomation

Graphs show that most attention should be directed to IT governance and Integration, and Goal setting and Measurement

17

Course of Action

• Based on this quickscan, you can identify pain points and areas for improvement

• Start with those areas that are least mature

• The quickscan provides guidance to select relevant approaches and instruments to help you get to the next maturity level– In our example outcome, to get from level 2 to 3 in Goal setting

and Measurement, recommendations would e.g. include

• a business case handbook

• a portfolio management tool

• a corporate EA repository

18

ENTERPRISE ARCHITECTURE REALIZATION ASSESSMENT

19

Enterprise Architecture Realization Assessment

• How do we assess the effectiveness of the EA capability in an organization?

• Focus on architecture goals and outcomes– NOT just the process

• Provides clear insight into current capability and the needs for improvement

20

Architecture Goals & Outcomes

1. Vision (phase A in TOGAF ADM)

• Is the goal part of the architecture vision?

2. Architecture design (B, C, D)

• Can the goal be achieved with the solutions described by the Business, Information Systems & Technology architecture?

3. Migration (E, F)

• Have the right projects been started to achieve the goal?

4. Project result (G)

• Can the goal be reached with the project (start) architectures?

• Have the relevant architectures been realized by the project?

5. Operational outcome (H)

• Have the project results realized the goal in practice?

Per organization goal, we measure the following outcomes:

21

Measurement

Per outcome, we measure three aspects:architecture Product, Scope and Acceptance

Product

• Does the product contribute to realizing the goal?

Scope

• Does the product cover the desired scope?

Acceptance

• Do the relevant stakeholders know and use the product?

22

Assessment Approach

Step 1. Preparation

• Setup

• Select interviewees

• nr. of interviews depends on org. size

• Way of reporting

• Lead time: 1-2 weeks

Step 2. Interviews

• 1,5 hours per interview

• Intermediate report

• Lead time: 2-4 weeks

• depends on availability of interviewees

Stap 3. Final report

• Writing report

• Presentation to management

• Lead time: 1 week

Total lead time 4-8 weeks

23

Example Outcome

Deliverable

Acceptance

0

2

4

6

8

10

#1 DefineVision

#2 DevelopArchitecture(s)

#3 DevelopMigration Plan

#4 Transfer theEA to

Implement.projects

#5 GovernImplementation

projects

EA Realization Score

Conclusion The example organization scores very high on vision and migration,below average on architecture development and transfer to projects,and very low on implementation governance

Legend

Deliverable Acceptance

10 =Deliverable is available,quality is good, scope is right

10 = Contents of the deliverableare well-known and actively used by all relevant stakeholders

1 = Deliverable is not available

1 =Deliverable is completely unknown to relevant stakeholders

24

Effort

• Total effort required depends on:– Size of organization, nr. of interviewees

– Goals to be evaluated

– Size of architectures and project deliverables in scope

• Lead time: 4-8 weeks, typical effort: 5-15 days

• A modular setup is also possible– Evaluating outcomes only for specific phases in the approach

• Vision, Architecture design, Migration, Project result, Operational outcome

• Specific quality analysis can be part of the Architecture design phase

25

ENTERPRISE ARCHITECTURE VALUE ASSESSMENT

26

Value of Enterprise Architecture

• Questions:

– How can EA benefits be classified?

– Are the benefits measurable?

– How can the benefits be measured?

– What are the measured benefits?

Changes in Operations

Organizational Goals

EA Goals

Effects of these changes

due to

terms

assessment

EA activitiesValue

created?

27

Classification Principles

• Our principles for measuring EA value:

– Value should be measured in terms of business goals

– Value can change (increment) with time

– Value has a subjective dimension as well: perceived value which should be taken into account

Balanced Scorecard

Architectural lifecycle

Different scales for measured and perceived value

28

The EA Value Classification Scheme

PerspectivePhase

Financial Customer Internal Learning & Growth

Development

Realization

Use

Re-use

Improved decision making

Business goals (Balanced Scorecard)

Low

er IT co

sts

Improved processes

29

Questionnaire

• In order to measure benefits, a questionnaire is used– Based on the classification

scheme

– Covers reported benefits

– Focuses on measurements made in the organization (“objective data”)

Any benefits in cell X?

Measurable benefits?

Measured benefits?

Regularly or ad-hoc?

30

Example Questions

Phase Customer’s PerspectiveDevelopment In developing the architecture, have the consequences for the

customers and the market been taken into account? Where? How is this translated to the realization phase?

Realization Has the impact of the migration on the customers and the market been established? Are there any analyses and/or scenarios developed and if yes, what were the expected results?

Use Has the market share increased as a result of the changes? How is this measured?

Has the customer satisfaction changed as a result of the changes? How is this measured?

Re-use Has a strategic advantage in the market been reached by virtue of the architecture? How is this evaluated?

Has the ability of the organization to react on changes in the environment (like market changes, changes in customers’ needs, etc.) increased? How is this evaluated?

31

Approach

Preparation with the responsible manager

Goal selection, identification of stakeholders, plan, ...

Report outcomes

Documents Collect evidence

Interviews With stakeholders

32

Measurability Maturity Scale

• Question: Are there enough factual data for objective measurements? Introduction of a measurability maturity scale

1. Ad-hoc: sometimes

2. Measurable: systematic, but not everywhere

3. Measured: systematic and enough data available

4. Managed: used in managing EA activities

33

Perceived Value

• When there are not enough factual data available: use the perceived value (gathered by the interviews)

• Extend withadditional questionnaireif necessary

Preparation

Measurability Perceived value

Report

Documents Interviews

34

Results

• The EA value assessment is a powerful tool for measuring benefits of EA

• Quick indication where insufficient data are available to establish the actual value

• Quick assessment of the perceived value of EA

• Subjective judgements may be quite different from objectivemeasurements

• Effort:– Lead time: 2-5 weeks, depending on availability of interviewees

– Typical effort: 4-10 days, depending on nr. of interviews (organization size, architecture size)

35

ARCHITECTURE QUALITY ANALYSIS

36

Quality Analysis

• Analyzing specific qualities of architectures and architecture products

• Can be performed as part of enterprise architecture realization assessment (phase 2: architecture design)– But can also be done as a separate review or second opinion

• Approach can be tailor-made to specific client questions and architecture content– E.g. cost analysis, agility, feasibility of a roadmap, security, etc.

• Example: Enterprise agility assessment

37

General Enterprise Agility Assessment

• General assessment of agile capabilities, in 3 aspects:*– Process agility: how does the organization use agile ways of

working?

– System agility: how does the organization foster the agility of its organizational and technical service systems?

– Business agility: how is agility part of the organization’s strategy and daily business?

• Assessment instrument with structured questionnaire, used in expert interviews– Ca. 1,5 hours per interview

– Total assessment can be done in 2-5 days, including report, depending on organization size

*M. Lankhorst (ed), Agile Service Development, Springer, 2012 38

Agile Capabilities

BusinessAgility

ProcessAgility

SystemAgility

Agility CapabilitiesAwareness andCommunication

Individuals andTeams

Goal Setting andMeasurement

Way of Working

Tools andAutomation

Capability Aspects

Graphs show that most attention should be directed to System Agility, Tools & Automation and the Way of Working

39

Business Agility

• Based on the Business Model Canvas*

• Shows hotspots where the business needs to be agile, but current agility is lacking• Darker colors = more

agility needed

*A. Osterwalder & Y. Pigneur, Business Model Generation, Wiley, 2010 40

CONCLUSIONS AND NEXT STEPS

41

Conduct an Assessment of Your EA Practice!

• Know the strengths and weaknesses of your EA practice

• Build a solid foundation for your EA capability– Create awareness & establish responsibilities

– People & skills

– Methods & tools

• Use independent expertise– Credibility of independent assessment

– Comparison with other organizations

• Get concrete guidance for improvement– Increase the business value that your EA capability provides

– Focus on the most urgent and valuable steps

42

Determine Which Assessment to Conduct

• Capability quickscan: – To get a quick overview of your EA capability

– What are the most important points for improvement?

– Does not look at EA products

• Realization assessment: – Does your architecture realize the desired goals and outcomes?

– In-depth assessment of both EA processes and products

• Value assessment: – To assess the measured and perceived value of EA

– Do the EA outcomes really add business value, and is this seen by your stakeholders?

43

BiZZdesign Can Help!

• BiZZdesign can assist you in all of these aspectswith our extensive EA knowledge and experience

• We provide:– Consultancy, e.g. in setting up your architecture practice

– Training on TOGAF, ArchiMate and more

– Tools for enterprise architecture, business modeling, Lean, and business process management

• Free trials of our solutions:

LeanCoach

44

For More Information on Assessments and Other Questions, Please Contact:

• Marc LankhorstService Line Manager Enterprise Architecturem.lankhorst@bizzdesign.com

• Brenda CowieVP Client Services, Director – BiZZdesign North Americab.cowie@bizzdesign.com+1 416 821 0297

Webinars, blogs, e-books, customer stories, training portfolio, software and more on www.bizzdesign.com

More on our EA Assessments

45

©BiZZdesign. All rights reserved.BiZZdesign and BiZZdesign logos are registered trademarks of BiZZdesign Company.