GTAG-14 Auditing User-Developed Applications MANAGING RISK.IMPROVING PERFORMANCE. September 13, 2012Tim Fawcett, CISA, CISSP

GTAG-14 Auditing User-Developed Applications

M A N A G I N G R I S K . I M P R O V I N G P E R F O R M A N C E .

September 13, 2012Tim Fawcett, CISA, CISSP

2 MANAGING RISK. IMPROVING PERFORMANCE.© Stinnett & Associates LLC

Groupon – March 2012Groupon’s stock plunged 14% as its first-as-a-public-company 10-K filing with the Securities and Exchange Commission revealed that its auditor, Ernst & Young, found “a material weakness in its internal controls over its financial statement close process,” raising questions in some quarters as to why these weaknesses were not identified earlier.

The weaknesses identified in the 10-K included “a number of manual post-close adjustments” (that is, a lack of adequately automated financial reporting leading to a welter of difficult-to-consolidate spreadsheets), and a failure to maintain both “effective controls to provide reasonable assurance that accounts were complete and accurate” and measures to ensure that account reconciliations “were properly performed, reviewed and approved.”

UDAs in the News

Management Accounting | April 02, 2012 | CFO.com | US - David Rosenbaum


BiovailBiovail Corp. has restated its 2005 and 2006 earnings upward because of an understatement stemming from a data error in tracking discounts in purchases of a drug, according to the company. The revisions boosted earnings for those years by $10.2 million and $7.7 million, respectively.

As part of the restatement process, Biovail found that the data-input errors and the amortization calculation represented a material weakness. The company also concluded that the failure of later efforts of local management to find those errors in a timely way also represented a material weakness.

To address the material weaknesses, management is installing measures to fix the control deficiency where the amortization error happened, … The measures include strengthening internal controls around the development and usage of spreadsheets and the review and analysis of those spreadsheets by local management. They also include mulling the automation of the spreadsheet-based data within the company's enterprise-resource-planning system.

UDAs in the News

Stephen Taub - CFO.com | May 10, 2007


KodakIn reconciling the general ledger balance sheet account for severance as of September 30, 2005 relating to one of the Company’s plant closings in the United Kingdom under its ongoing restructuring program, the Company discovered that a spreadsheet error caused it to overstate a severance accrual as of and for the quarter ended June 30, 2005 by $11 million (net of tax). The Company performed a root cause analysis to understand the control deficiency, which revealed that the error was primarily the result of a failure in the operation of, not the design of, the existing preventive and detective controls surrounding the preparation and review of spreadsheets that include new or changed formulas. This deficiency resulted from a failure to follow established policies and procedures partially due to changes in personnel.

The Company has concluded that this deficiency constitutes a “material weakness” as defined by the Public Company Accounting Oversight Board’s Auditing Standard No. 2. This material weakness resulted in an adjustment that was included in the restatement of the Company’s consolidated financial statements as of and for the quarter ended June 30, 2005. Additionally, if the material weakness is not corrected, it could result in a material misstatement of other financial statement accounts that utilize spreadsheets that would result in a material misstatement to annual or interim financial statements that might not be prevented or detected.

UDAs in the News

Excerpt taken from the EK 10-Q filed Dec 12, 2005

http://www.wikinvest.com/stock/Eastman_Kodak_Company_(EK)/Filing/10-Q/2005/F3453657


A survey in 2006 of 685 senior financial executives from a broad range of companies, revealed that revenue recognition and reporting activities are not automated within Financial / ERP systems. As a result, 92% of public companies are forced to rely on spreadsheets to fill vital gaps in their revenue reporting processes—despite the fact that spreadsheets are prone to errors, lack audit capabilities, and resist internal controls.

UDAs in the NewsSpreadsheet-based revenue recognition

and reporting tasks.(Multiple responses accepted, n=685)

1. Creating accounting entries 52%

2. Creating revenue recognition schedules for future periods 47%

3. Reporting on future revenue streams 47%

4. Applying revenue allocation rules 43%

5. Performing revenue contribution analysis 42%

6. Redistributing revenue (e.g. SOP 97-2, EITF 00-21) 35%

7. Reviewing sales orders for deferred revenue 27%

8. Do not use spreadsheets for any of these activities 8%Source: www.RevenueRecognition.com and IDC (International Data Corporation) 2006

http://www.revenuerecognition.com/


What is a UDA? Benefits and Risks

Internal Audit’s Role

Scoping an Internal Audit of UDAs

Best Practices for Frameworks of Controls over UDAs

MS Office/ Excel Control Examples

Overview


For purposes of this GTAG, UDAs are any application that are not managed and developed in a traditional IT environment and under a formal development process. Spreadsheets used on an ad hoc basis — to provide lists of information or to quantitatively illustrate data available elsewhere — usually are not considered UDAs.

A UDA is key if at least one of the following criteria are met: The UDA is used to initiate, accumulate, record, report, or monitor material financial

reporting-related transactions and key operational management reports and/or meet regulatory compliance requirements.

The UDA’s use is inherent in performing key financial and/or operational control processes (e.g., account reconciliations and key performance indicator reports) so that if the spreadsheet or data was lost or corrupted, the loss would impact the control’s effectiveness.

Defining User-Developed Applications (UDA)


SpreadsheetsAccess DatabasesCrystal ReportsOther DatabasesScripts (SQL Scripts)

ACLWeb AppsAppsExecutableEasytrieve


Examples of UDAs


“If the normal operation of the manual portion of the control is sufficient to detect an error in the automated portion (e.g., the computer report), then the control can be considered entirely manual since no reliance is being placed on the computer application. For example, a bank reconciliation might use a report from the general ledger system of cash transactions; if the report was incorrect or incomplete, it would be detected by the bank reconciliation process.”

Sarbanes-Oxley Section 404: A Guide for Management by Internal Control Practitioners, Page 34



Almost every organization uses some form of UDAs because they are:

Quicker to develop and use.

Readily available tools at a lower cost.

Configurable and flexible.

Benefits of User-developed Applications


Risks Associated With User-developed Applications

Control breakdowns within UDAs are often traced to:

Lack of structured development processes and change management controls

Data download issues

Increasing complexity

Lack of developer experience

Lack of version controls

Lack of documentation

Lack of support

Limited input and output controls

Lack of formal testing

Hidden data columns or worksheets


Global Technology Audit Guide (GTAG®) 14 Auditing User-developed Applications

GTAG-14 provides direction on how to scope an internal audit of UDAs.

GTAG-14 also provides guidance for how the internal auditor’s role as a consultant can be leveraged to assist management with developing an effective UDA control framework, including:

Identifying the UDA population by using different discovery techniques.

Assessing and ranking the risks associated with each UDA based on the potential impact and likelihood of risk occurrence.



GTAG-14 Summary

Use of UDAs can contribute to or detract from an organization’s control environment. Professional judgment must be applied as to what constitutes key when auditing UDAs.

Ideally, the organization has established an enterprise definition that can be used. However if such a definition is absent, a systematic approach must be used to determine the extent of risk to the organization and, more importantly, the level of risk that the organization is willing to accept.



Scoping an Internal Audit or UDA Program

An internal auditor, whether auditing a UDA program or providing guidance to users on the development of a UDA program must:

1. Define what constitutes a key UDA and

2. Determine the population of UDAs for Audit, or include in the UDA Program by:

Defining Risk Factors

Risk Ranking


Define what constitutes a key UDA Any application that are not managed and developed in a traditional IT environment and under

a formal development process. A UDA is key if at least one of the following criteria are met: The UDA is used to initiate, accumulate, record, report, or monitor material financial

reporting-related transactions and key operational management reports and/or meet regulatory compliance requirements.

The UDA’s use is inherent in performing key financial and/or operational control processes (e.g., account reconciliations and key performance indicator reports) so that if the spreadsheet or data was lost or corrupted, the loss would impact the control’s effectiveness.

Spreadsheets used on an ad hoc basis — to provide lists of information or to quantitatively illustrate data available elsewhere — usually are not considered UDAs.



Determine the population of UDAs for AuditManagement may call for a review of specific, known UDAs (e.g., those that support journal entries) or it may require the identification of all steps and tools used to support business processes. In either case, if management does not maintain a consolidated list of UDA applications, the auditor may, in the role of consultant, guide management on how to identify and inventory UDAs by evaluating business process documentation such as business process flows and procedural narratives. Other techniques that management may consider for identifying the UDA population include:

The use of a search capability to identify spreadsheet and database file tags within all or specific file directories related to a business process.

Use of purchased software tools to detect UDA populations. (See section 4.1 for UDA discovery tool attributes and capabilities)

Review of reports identifying manual journal entries, which likely are supported by a UDA.



Defining Risk FactorsUsing spreadsheets or other UDAs for accumulating and calculating critical operational and material financial information can present significant risk to the organization, including:

Data integrity issues.

Errors made during input, processing, and output, including interfaces and reports.

Errors or intentional manipulation due to unsecured files or unmanaged change.



Risk Ranking

At a minimum, the risk factors for identifying the impact of a failure in a UDA should include:

Financial, operational, and regulatory compliance materiality of the UDA.

Expected life and frequency of use of the application.

Number of users of both the application and the results.

At a minimum, the risk factors for identifying the likelihood of a failure in a UDA should include:

Complexity of obtaining inputs and generating desired outputs.

Frequency of modification to the UDA.

















High Level Approach to Risk RankingAnother approach to consider evaluates risk at a much higher level. As with the previous approach, the UDA population is identified by business processes.

This approach identifies the risk, mitigating controls, and residual risk with recommended inclusion or exclusion from the population.





A. System Security and AccessB. Audit TrailsC. Inputs, Edits, and InterfacesD. Data Processing and Data IntegrityE. Reports and OutputF. RetentionG. Backup and RecoveryH. Change Management

Audit Areas


A. System Security and Access1. Identify in-scope UDAs and related data and determine the file names,

directories, datasets, and/or databases where the UDAs and data reside.2. Obtain the access rights to in-scope UDAs and related data and evaluate the

appropriateness of such access. 3. Verify that user authentication controls to the systems containing the4. UDAs and data appropriately restrict unauthorized access.5. Determine whether there are other ways to access the UDA or the data and

evaluate the controls over the access.6. Verify whether access is periodically reviewed.

Audit Areas


B. Audit Trails1. Identify whether audit trails exist and where they reside.2. Determine the appropriateness of the audit trail. 3. Verify that users with the ability to change or delete audit trail programs and

logs are not the users of the UDA and/or data.4. Verify that the audit trails are periodically reviewed and retained for an

appropriate period of time.

Audit Areas


C. Inputs, Edits, and Interfaces1. Identify the source and type of input data.2. Verify that controls over critical file inputs are appropriate. Consider:

• Data validation rules.• Edits are consistent regardless of source.• Record/item counts and balances ensure completeness.

3. Verify whether error notifications or reports are produced and correctiveactions have been taken. Consider:

• Control totals are reconciled to ensure completeness.• Erroneous input files can be backed out and rerun.

Audit Areas


D. Data Processing and Data Integrity1. Determine whether the system-produced records are overridden manually on a

routine basis to fix processing errors.

2. Determine whether data manipulation tools are used to correct processing errors.

3. Verify that detailed audit trails for manual overrides are maintained with the source request from the business.

4. Verify that processing errors are clearly described, promptly detected, and flagged for correction.

5. Determine whether a process exists to reverse transactions, correct errors, and re-process transactions with special manual handling.

6. Verify processing controls exists for spreadsheets.

Audit Areas


E. Reports and Output1. Verify that output control totals are compared with input control totals and

errors are resolved.

2. Verify that UDA application logic and critical formulas are periodically validated.

3. Determine whether mitigating business controls exist to detect output errors (e.g., downstream reconciliations and/or control processing).

Audit Areas


F. Retention1. Verify that data is appropriately retained.

2. Ensure that appropriate information or notations exist for documents/ reports retained past the period outlined in the data retention policy.

Audit Areas


G. Backup and Recovery

1. Verify that a list of critical UDAs is maintained.

2. Verify whether critical UDAs and related data are periodically backed up.

3. Determine whether backups are retained in a safe location.

4. Determine whether UDA recovery is periodically tested.

Audit Areas


H. Change Management1. Verify that appropriate application change management procedures are

followed.

2. Verify that a separate source copy is maintained.

3. Verify that the approved application version is moved into production.

Audit Areas


Spreadsheet Development and Maintenance Overview

Access Guidelines

Source Data Guideline

Source Output Guidelines

Testing Guidelines

Logic Guidelines

Version, Backup, and Archiving

Documentation Guidelines

Control Framework or Guidelines



Academic research indicates that spreadsheet development shares many characteristics with traditional software development*… the benefits gained from a sound development lifecycle… includes design, inspection, and maintenance.

* Panko, Raymond R. and Nicholas Ordway. “Sarbanes-Oxley: What about All the Spreadsheets?” University of Hawaii, 2005.

Define Requirements Design Implement Test &

Verify DeployMaintain & Document

Spreadsheet Development and Maintenance


Process RisksInaccuracies in end-user systems result in financial reporting misstatement.

Process ControlsAll spreadsheets and other end-user systems are protected from unauthorized access. Spreadsheets and other end-user systems are saved in secure directories on secure network file servers where access privileges are limited to appropriate people or business groups.

To ensure data is input correctly and completely, the input data is reviewed and verified for reasonableness by both the preparer and reviewer of the spreadsheet or other end-user system.

Changes to the logic or mechanics of the end-user system are reviewed and verified by both the preparer and the reviewers of the spreadsheet or other end-user system.



Access GuidelinesLimit access to spreadsheets and other end user systems stored on a network server on a need-to know basis according to job responsibilities.



Source Data Guidelines The data input area generally should not contain formulas. “When each cell

contains both key data and the complicated assumption-laden algorithms to be applied, confirming the results are appropriate or reasonable may be virtually impossible — even if calculated correctly. It is a better practice to separate the data from the algorithms and assumptions being applied to the data.”*

When possible, data input — manual or interfaced — should be in the same order as the source data to facilitate review and minimize input errors.

Lock formulas.


* Spreadsheet ‘Worst Practices,’” CFO.com


Source Output Guidelines Do not use the same worksheet and only change the assumptions and variables

while leaving no baseline or trail of what has been changed during the “what if” analysis.

“The best way to compare and review results from different combinations of variables are (a) to copy the original data sets and calculations into a separate spreadsheet tab, and (b) to build a comparison spreadsheet tab, which presents and contrasts the original.”*

Consider what the final presentation format needs to look like. Avoid the need to manually retype the output into other formats and tools, causing errors.*

Identify authorized users for each report that is output as well as data storage and retention guidelines.


* Spreadsheet ‘Worst Practices,’” CFO.com


Testing Guidelines Make sure that changes to highly complex or critical UDAs are formally requested,

documented, and tested.

Task someone other than the spreadsheet’s user or developer with testing complex or critical calculations and logic.

Use analysis and reasonableness reviews to detect errors in calculations and logic.



Logic Guidelines Place critical values in a separate cell and refer to this cell in the formula rather

than incorporating the number in a formula in one or more cells.

Incorporate batch totals and control totals.

Use formulas that foot and cross-foot data.

Ensure data integrity by locking or protecting cells to prevent inadvertent or intentional changes to static data or formulas.

Include expected results where possible to compare and monitor the reasonableness of UDA output.



Version, Backup, and Archiving Guidelines Use unique folder and file naming conventions that include the month, quarter,

and year to help ensure that only current and approved versions of UDAs are used. Consider using check-in and check-out software to manage version control.

Ensure data backup by storing spreadsheets and other UDAs on a network server that is backed up daily.

Store historical files and databases not in use in a segregated, read-only folder to avoid mistakenly using them.



Documentation Guidelines Document the purpose and use of each critical UDA and update accordingly.

The documentation should include the business objective, inputs, outputs, and sequence of execution for multistep processes.

Create a consistent layout for spreadsheets and other UDAs to simplify use and testing. The areas for data input, calculations, and output should be distinct and separate.

Use Consistent Cell Styles



Documentation Guidelines (continued) Label files, data sets, worksheets, key fields, rows, columns, and data for easy

identification.

Inventory all key spreadsheets and other UDAs impacting financial statement preparation.

Clearly document assumptions applied and leveraged to generate data or perform calculations.



Specific Controls and Methods for Controlling Excel UDAs

Preventing Unauthorized Access to Spreadsheets

Managing and Monitoring Changes with SharePoint

Retaining and Archiving Spreadsheets

Developing Robust Spreadsheet Models

MS Office/Excel Controls


Preventing Unauthorized Access to Spreadsheets

Office SharePoint Server Capabilities

Sharing Spreadsheets Using Excel Services

Information Rights Management

Workbook Encryption

MS Office/Excel Controls – Spreadsheet Access


Managing and Monitoring Spreadsheet Changes with SharePoint Versioning - SharePoint Server has a robust check-in/check-out and versioning

mechanism

Auditing - SharePoint Server allows administrators to audit key events within document libraries. While there is no built-in capability to audit changes within spreadsheets individually, the audit log records spreadsheet events such as Open, Modify, and Delete.

Workflow - With SharePoint Server management can build workflows that map to important business processes.

MS Office/Excel Controls – Spreadsheet Changes


Retaining and Archiving Spreadsheets The following Office SharePoint Server capabilities can help users fulfill records management requirements:

Vault Capabilities - The Records Repository has several features that help ensure the integrity of files stored in the repository.

Information Management Policies - Provide controls that consistently and uniformly enforce the labeling, auditing, and expiration of records.

Hold - The Records Repository allows users to apply one or more holds that suspend records management policies on specific items to prevent documents from being changed during litigation, audits, or other investigations.

MS Office/Excel Controls – Retaining and Archiving


MS Office/Excel Controls – Spreadsheet Models Developing Robust Spreadsheet Models Microsoft Excel can be used to create a robust spreadsheet model that meets compliance challenges and enhances productivity.

MS Excel capabilities can help an organization deploy spreadsheet models that make it easier to become, and stay, compliant.

1. Cell styles 2. Checksums3. Lock important cells 4. Using Excel Tables to reduce errors 5. Defined Names 6. Formula auditing tools 7. Data Sources and Input


MS Office/Excel Controls – Cell Styles

Cell styles help distinguish input cells from calculation cells


DataSafeXL August 2010 white paper “Excel Hell: How Simple Checksums Can Ease The Pain of Financial Modeling” provides a good primer on one approach to managing checksums.

Cell-based modeling is a root cause of some of the issues including:1. Simple errors in formula construction, returning error values such as: #VALUE! ; #REF! ;

#NAME? ; #N/A; etc.2. Errors in formulas dependent on other feeder cells that only become apparent later on,

usually in different tabs to the tab you are currently working on, but missed because you cannot see them or are not alerted to them.

3. Changing the spreadsheet structure which frequently creates errors containing the notation #REF! which ripples through financial statement rollups, thus making them unreadable.

MS Office/Excel Controls – Use of Checksums


Create a page purely for checksums.



For each sheet in your workbook select all cells with the arrow situated between the A and the 1.

Give this range a name similar to “INDEXSHT1.” This creates a named range which will detect any formula errors in the whole sheet, e.g. #VALUE! ; #REF! ; #NAME? ; #N/A; etc.



For cells B10 through B12 write the formula for the appropriate sheet =IF(ISERROR(SUM(INDEXSHT1)),FALSE,TRUE)

For cells B10:B12 name the range “SUMMARYCHECK” and Cell B6 name “SUMMARY”

Add the formula to Cell B6 =IF(COUNTIF(SUMMARYCHECK,FALSE),FALSE,TRUE)



Add some simple conditional formats to the checksum cells (green for TRUE, red for FALSE) to help make them more visibly identifiable.



Checksums shown are in their simplest format

Use checksums at a more advanced level by creating multiple checksums for a single sheet, perhaps referencing various important ranges rather than whole sheet ranges

Help to pinpoint errors much more quickly and effectively

You can include any kind of formula such as those to identify mistakes or to aid reconciliations, e.g. =IF(SUM(RANGE1)SUM(RANGE2),FALSE,TRUE)



MS Office/Excel Controls – Protect Worksheets

Shortcut = Ctl+1


MS Office/Excel Controls – Allow Users to Edit RangesAllow Users to Edit Ranges


MS Office/Excel Controls – TablesTables make common tasks easier to perform and more robust. As data is added to a table, any elements associated with the table automatically adjust. Formatting applies to new rows and formulas update to include new data. PivotChart views, PivotTable views, Conditional Formatting, and Data Validation will all update to fit the new data.


This type of referencing is called “Structured Referencing” and it increases the readability of formulas to make them easier to maintain and edit later.

MS Office/Excel Controls – Table Referencing

Formulas that reference data in a table do so by name (the name of the column, e.g. “Sales”) rather than by an undecipherable A1-style address (e.g., D1:D10).


Table formatting features behave intelligently. For example, if alternate-row formatting is enabled on a table, Excel will maintain the alternating format rule

MS Office/Excel Controls – Spreadsheet Models

through actions that would have traditionally disrupted this layout, such as filtering, hiding rows, or manual rearranging of rows and columns.


MS Office/Excel Controls – Use of Named Ranges

Create a Named Range in ExcelSelect the cell or range of cells to be named, such as B2 to B5Click in the Name box, to the left of the formula bar. Type a name for the list, e.g. Jan_sales Press the Enter key on the keyboardThe name will appear in the Name box

Named Range ExamplesA named range can be used when creating charts, and in formulas and functions such as:

= SUM( Jan_sales )= Jan_total + Feb_total + Mar_total

Since a named range doesn't change when a formula is copied to other cells, it provides an alternative to using absolute cell references in functions and formulas.


The Name Manager View important details such as the

name’s reference, value, and scope. Create and scope names. Rename existing names. Delete multiple names at once. Sort and filter the name list by

common criteria including scope, type, and if the name returns an error.

MS Office/Excel Controls – Spreadsheet Models


Trace Precedents using auditing arrows

Graphically display (or “trace”) the relationships between cells and formulas.

Trace a cell's precedents (the cells that provide information to that cell).

Trace a cell's dependents (the cells that receive information from that cell.)

Check for errors in a formula.

MS Office/Excel Controls – Trace Precedents


MS Office/Excel Controls – Importing From Data Sources


1. There are a variety of data sources that you can connect to: Analysis Services, SQL Server, Microsoft Access, other OLAP and relational databases, spreadsheets, and text files.

2. Many data sources have an associated ODBC driver or OLE DB provider.3. A connection file defines all the information that is needed to access and

retrieve data from a data source.4. Connection information is copied from a connection file into a workbook,

and the connection information can easily be edited.5. The data is copied into a workbook so that you can use it just as you use

data stored directly in the workbook.

MS Office/Excel Controls – Importing From Data Sources


What is a UDA? Benefits and Risks


Scoping an Internal Audit of UDAs

Best Practices for Frameworks of Controls over UDAs

MS Office/ Excel Control Examples

Overview


Tim FawcettManager, Stinnett & [email protected]

Questions?

Documents

GTAG-14 Auditing User-Developed Applications MANAGING RISK.IMPROVING PERFORMANCE. September 13, 2012Tim Fawcett, CISA, CISSP