Auditing SDLC(NT20071212)

1

Auditing Systems Development Lifecycle

Audit Guidelines On How To Review SDLC Framework

By

Nandasena T(NT) Hettigei CISA, CISSP, CITP, CPA, CA

Copyrights © NTH 2007

Presented at ISACA Roundtable on 12/12/2007 at KPMG Training Center, Wells Fargo Building Minneapolis . MN

2Auditing the Systems Development Lifecycle - By NT Hettigei © 2007BY NT HETTIGEI © 2007- Presented at ISACA Roundtable on 12/12/2007 at KPMG Training Center, Wells Fargo Building Minneapolis . MN.USA

Section (1) - Introduction

• Introduction• Big Picture• What is SDLC

• Audit Approach• Audit Scope & Objectives• Auditing SDLC Framework


Section (2) – Audit Process

• Evaluate Adequacy• Waterfall Model• Iterative Model• Agile Model

• Validate Effectiveness• Validate Common Components• Project Management• Auditor’s Role


Section 1

Introduction to

Systems Development Life Cycle


Introduction

• Big Picture Blueprint– Oversight– Project management – Development Life Cycle (SDLC)

• What is SDLC– System or Software?– How to add value?

SDLC is a methodology/framework that provides a systematic approach to develop information systems/software while ensuring quality


SDLC Audit Approach (1) Audit Scope and Objectives

• Evaluate adequacy of the methodology – Ensure system development follows a proven methodology

to maintain consistency, effectiveness and efficiency of the systems development process in order to maintain the quality of the outcome.

• Validate effectiveness of the methodology– Validate by testing and substantiating that risks are

mitigated effectively by consistently adhering to the methodology/controls.


SDLC Audit Approach (2) Frameworks/Models

• Traditional phase by phase model– Waterfall model (linear and sequential)

• Iterative model– RAD (Rapid Application Development) – JAD (Joint Application Development)– Spiral Model– Synchronize-and-stabilize Model

• Agile model (timeboxes )– ASD (Adaptive Software Development) – FDD (Feature Driven Development, and DSDM)

( Vendor specific: HP-Mercury, IBM-RUP, Compuware - ASD, etc)


Section 2 – Auditing SDLC

Audit Process2.1 – Evaluate adequacy 2.2 – Validate effectiveness

Reminder - We have been following the standard audit process of:Obtaining an understanding of the control environmentEvaluating the adequacy of controlsAssessing by testing of controls Substantiating risk of controls objectives not being met

Source - Control Objectives for Information and Related Technology (CoBiT), IT Governance Institute.


Evaluate the Methodology (2.1.1)Waterfall Model

• Analysis Phase• Scope definitions• Requirements Analysis

• Design Phase• Functional Design• Technical Design• Business Process Design (Across all Phases)

• Development Phase• Build/Coding• Testing (unit, integration and system testing)• Performance, Regression and Security testing• QA testing (UAT)

• Delivery and Transition Phase• Data conversion and Deployment • Training and Support


Evaluate the Methodology (2.1.2)Waterfall Model

Recommended for: – Customization or implementation of ERP or

other business support systems– Replacement of a legacy system where you

have defined requirements– Outsource developments with stage gate

payment terms


Evaluate the Methodology (2.1.3)Iterative Model


Evaluate the Methodology (2.1.4)Iterative Model

Recommended for:– New product (application) development– Prototype/Business intelligent systems– Innovative projects/products– Increment functionalities within a website


Evaluate the Methodology (2.1.5)Agile Model

• Self-contained mini-project• Each lasting only a few weeks • Each iteration has it own self-contained

stages of: – analysis – design – development – testing – deployment and – documentation

(Agile aims to reduce risk by breaking projects into small, time-limited modules i.e. timeboxes)


Evaluate the Methodology (2.1.6)Agile Model

Recommended for:– Large projects to use as a powerful

method to manage deployments– Projects that require rapid and significant

change– Projects where even late changes in

requirements are needed


Evaluate Methodology (2.1.7)

• After all, you’ve probably noticed that the three major development processes share the same fundamental phases: design, implementation, integration, testing and deployment.

• Validating the processes are not different to one another.


Section 2.2 - Validation

• Validating key controls within common SDLC components

Reminder - We have been following the standard audit process of:Obtaining an understanding of the control environmentEvaluating the adequacy of controlsAssessing by testing of controls Substantiating risk of controls objectives not being met

Source - Control Objectives for Information and Related Technology (CoBiT), IT Governance Institute.


Validate SDLC Components (2.2.1)

Conceptual Access control requirements (SOD vs. Open) Conceptual Application Security ( HIPAA, PCI, GLBA, etc)Conceptual System Security (internal vs. www systems)

Security Requirements

Number of simultaneous users and transactions updatesScalability / Throughput / CapacityResource utilization (especially of shared resources)Response time for a transaction

Performance Requirements

Business Case/requirements prioritiesHigh level use cases and required activitiesDependencies and redundancies (Impacted systems)System inputs and outputs – data, interfaces, etcRe-prioritize requirements as needed

Functional Requirements/Use cases

ValidationAnalysis



Standard BPD template that includes:Process flows (systems and functions)Controls, reports and process owners Manual check points and test scenarios

- Revised throughout SDLC phases to accommodate functional changes

Business Process Designs

Standard TD template that includes:Reference to related FD and functionsCode, Error handling, systems and integration pointsData schema or reference to data tablesSecurity designs

Technical Designs

Standard FD template that includes:Complexity (High, Medium and Low) Transaction Volume, Constrains and Dependencies Risk, Controls, Security and Test scenarios

Functional Designs/Use cases

ValidationDesign



Development standard documentation that includes: Coding standards Nomenclatures, Comment lines and segments Programming with multi-threading Code reviews (peer reviews and performance reviews)Application security/Source code analysisInput, process and output controlsError handling standards Defects classifications (Showstoppers, Sev 1, etc.) Unit testing, Coding quality controlCode version management

Development/Coding

ValidationBuild



Integration approach should include:Inventory of FDs and TDs with priorities and dependencies Integrators, Adaptors and Middleware (MQ series) System architecture, data flow diagramsIntegration with vanilla codes or functionalitiesIterative vs. Incremental integration Integration Test approach Dependencies (systems and processes) Change and Version Control Error handling

System Integration

ValidationIntegration



System Test approach should include:Production like testing environment Acceptable defects rate (%)Entry and exit criteria for system test

Unit test completed and acceptable defects rate Code certified (if developed by a third party)

Functional test scenarios approved by stakeholdersPerformance testing includes:

Number of users, Volume, response time, etc.

Security testing includes:Application, Access and System security

Rework and retest standardsRegression testing

Functional Performance and Security Testing

ValidationTesting



System Quality Assurance approach should include: Requirements quality (functions, performance and security)Defects tracking and trend analysisIssue tracking and trend analysis system/toolsStage gate sign-off process Security settings and role base access controls Automated process workflows System alerts for transaction exceptions Regression testing Performance and stress testingApplication and system security testingUAT (user acceptance test) scenarios and testing High availability, failover/recovery and disaster recovery QA exit criteria – Meeting customer/business requirements

System/Software Quality Assurance

ValidationQA



Launch approach & customer impact assessmentDeployment timeframe and system down time (impact)Data conversion and validation processGo/No go decision pointsFailover/recovery during the migration process

Deployment

Post deployment support (30 days – 6 months) Expert teams knowledge transfer Documents repository Training support Defects clearingProblem resolution

Support

ValidationDelivery



Requirements Documentation (catalogue)Design and Development Approach Test and defects management Approach Quality Assurance Approach Deployment and Launch Approach Functional Designs /Use CasesTechnical Designs and Data Schemas Business Process DesignsTest scripts/scenarios, Issues log and defects log Deployment process with contingency rollbackSecurity settings (access, system and roles)System specification, data sheets and user guides

Adequate Documentation

ValidationDocumentation



Change management toolsQuality management tools (e.g. Quality Center) Issue tracking tools (e.g. PVCS) Code version manager (e.g. Subversion)Source code analysis tools (e.g. DevInspect) Application QA tools (e.g. QAInspect) Code migration tools/scriptsValidation checklists and standard templatesEnterprise target infrastructure (e.g. Tech Blueprint/BOB)Enterprise information security policies & standardsCapacity, performances and scalability testing tools(e.g. LoadRunner)

SDLC Tools

ValidationTools


Validate SDLC Components (2.2.10.1)

Architect (software, system and performance)Business Systems Analyst Developer, Code Reviewer, Tester Security ArchitectProduct Manager/Business/process owner Stakeholder Technical Writer Trainer

Development

QA ManagerQA Analyst Security Analyst Performance AnalystBusiness SMEs ( Subject Matter Expert)

Quality Assurance

ValidationRoles


Validate SDLC Components (2.2.10.2)


Project Management

Project management methodologyAdequate business engagement in the projectProject managers engaged with the stakeholdersIT leaders engaged with end usersScope, Schedule and Budget monitoringInterim Merit ReviewsFailsafe Approach

Project Management

Project risk management process Organizational alignment (business readiness)Adequate training and communicationDefined service levelsDefined project delivery processContingency plan and roll back approach

Project Risk Management

ValidationProject


Auditor’s Role

• Auditor Vs. Quality Assurance– Auditor is not playing the role of quality assurance

• Auditor Vs. Risk Management – Risk management is a project activity

• Auditor’s Role– Auditor is a SME (subject matter expert) for risks

and controls (What may go wrong on process and recommendation to mitigate such risks)


Q & A

Thank You

Email your questions to – [email protected]

References;1. IS Control Journal – The Auditor's Role in IT Development Projects – NT Hettigei2. CoBit ; Control Objectives for Information and Related Technology (CoBiT), IT Governance

Institute. URL http://www.itgi.org/3. IT Auditing Standards – Information Systems and Controls Association URL

http://www.isaca.org/Template.cfm?Section=Standards&Template=/TaggedPage/TaggedPageDisplay.cfm&TPLID=29&ContentID=8529

4. ITIL – The ITIL and ITSM Directory – URL http://www.itil-itsm-world.com/5. CMM – Capability Maturity Model – URL http://www.sei.cmu.edu/cmm/cmms/cmms.html6. Which Development Method Is Right for Your Project? By Adam Kolawa URL:

http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=ART&ObjectId=31527. Models for Managing Projects, IT Lecture Notes by Mark Kelly, McKinnon Secondary College

URL: http://www.mckinnonsc.vic.edu.au/vceit/models/index.htm#agile8. Internet Security System White Paper: Dynamic Threat Protection: URL

http://documents.iss.net/whitepapers/DynamicThreatProtection.pdf

Download the presentation from ISACA website – URL http://www.mnisaca.org/