PCAOB Inspection Themes - PhxSACphxsac.com/.../2013/06/PCAOB-Inspection-Themes.pdf · Payroll Files A B C D E F G H I J K L M N A systems interface diagram is a key source of

PCAOB Inspection Themes

What your external auditors are about to ask you 22 October 2013

Page 2

Introductions

Jessica Hatch – Manager, Advisory Services – IT Risk & Assurance. Seventh year in public accounting. Serves Fortune 100 public company based out of Arizona. Participated in PCAOB and internal quality inspections.

Diana Gomes – Manager, Assurance Services. Seventh year in public accounting. Serves multiple public companies based out of Arizona. Participated in PCAOB and internal quality inspections.

Special thanks to our friends at Deloitte and PwC for their input!


Page 3

Agenda

►  Overview of PCAOB, inspection process, and recent results

►  Recent IT-related PCAOB inspection themes ►  Better understanding flows of transactions, IT interfaces, and

considering all IT risks ►  Testing management’s controls over electronic audit evidence ►  Testing precision of review controls ►  Evaluating controls over service providers (SOC reports)

►  The future of external audit


Page 4

Overview of PCAOB, inspection process, and recent results

►  The Public Company Accounting Oversight Board (PCAOB) is a private-sector, nonprofit corporation created by the Sarbanes–Oxley Act of 2002 to oversee the audits of public companies and other issuers in order to protect the interests of investors and further the public interest in the preparation of informative, accurate and independent audit reports.

►  The PCAOB audits “Big 4” accounting firms in calendar Q2 and Q3 each year, and other public accounting firms in Q4. The inspection typically consists of review of audit documentation over internal controls and substantive audit testing over selected high risk/focus areas. The inspections typically require 1-2 weeks of on-site fieldwork. Comments can be verbal, written (does not appear in report) or audit deficiencies (appears in public report)

►  EY’s 2012 inspection report (which covered the results of reviews of 2011 audits) was released on 6/28/13. ►  The PCAOB inspection 51 audits of public companies during 2012 ►  25 issuers had audit deficiencies that appeared the report, 22 of which (43% of

inspections) had comments related to internal controls over financial reporting


Page 5

IT-related PCAOB inspection themes


Page 6

Flows of transactions, IT interfaces, and considering all IT risks


Page 7

PCAOB inspection theme

►  Not all risks of material misstatement were identified - resulting in an incomplete set of controls identified and tested

►  Missing risks of material misstatement often related to the following IT-related considerations: ►  Not obtaining a sufficient understanding of the systems and flow of transactions,

which is necessary to identify all risks of material misstatement ►  Not all IT risks, particularly interface risks, were properly considered ►  All points within processes where material misstatements could arise not identified ►  Lack of focus on walkthroughs leading to the insufficient understanding of the

systems and flow of transactions or consideration of all processing alternatives ►  During walkthrough procedures, we did not gain an understanding of whether

application controls were properly configured or contained the appropriate parameters.


Page 8

Common IT risks that need to be considered within significant financial processes

►  Unauthorized initiation/authorization of transactions ►  Lack of segregation of incompatible duties

►  Reliance on IT applications or programs that are inaccurately processing data

►  Potential for errors and fraud within IT applications

►  Inappropriate dependence on the results of computer processing

►  Lack of transaction trails or loss of data


Page 9

System interface diagrams


A system interface flow chart gives a pictorial representation of the systems that support significant business processes, including how data flows from system to system.

System Interface flow charts provide the reader with a quick understanding that can help us to:

►  assess the complexity of the IT environment

►  identify where application interface controls should exist (or where control gaps do exist)

►  understand the inputs/outputs from systems

►  understand the types of electronic audit evidence generated

►  understand applications and tools supporting significant processes

Page 10

Example system interface diagram


E2 Hyperion HFM

FRP

EMP

Accurate NXG

Financial Statements

Caesar

CASH

CDS

CIMS GEAC

Pep+

TMS

CDE

OCRA

Policy Administrative

Systems

Treasury Customer

Online Check

Requests

Cost allocation

files

Payroll Files

A

B

C D

E

F

G

H I

J

K

L

M

N

A systems interface diagram is a key source of information used to understand a complex and highly automated IT environment

Page 11

System interface inventory

Interface Description Data Description Interface Type Process Control Language

F CDS à E2 Check disbursement data Flat file data set within MF environments as

scheduled job

Cash Disbursements –

Checks

Daily CDS transactions are balanced to check stock used. Admin and online transactions are balanced to CDS output. CDS totals are balanced to the general ledger. Error reports are reviewed and corrections are processed.

G E2 à TMS Banking and cash information

Connect direct file transfers as a

scheduled job from MF to AIX directory

Bank Reconciliations

Weekly bank reconciliation performed by Accounting department.

H CASH à E2 Cash receipts data

Flat file transfer from Windows SQL to MF

throughout day. Nightly batch job

picks up flat file data to E2

Cash Receipts

Interface from CASH System to e-2 is automated. All general ledger entries are accomplished with this interface except for required correcting entries made subsequent to initial processing.

I CIMS à E2 Cost allocation data Scheduled job within MF GS02 from CIMS

to E2

EMP Cost Allocation/Acquisition

Variance analysis is completed each month. Expense Accounting, senior management (quarterly), and cost center personnel review the expenses. Significant variances are explained in the Quarterly review book.

J E2 à FRP Financial reporting data

Informatica is utilized to read the DB2 table and create an Oracle

table that is then loaded into FRP

FRP Data Load from E2

Reconciliation of E2 to FRP by legal entity (evidenced by zeroes legal entity in the reconciliation report)


Page 12

Testing management’s controls over electronic audit evidence


Page 13


►  Not identifying and testing Issuer controls (either ITGC or business process controls) to assess the completeness and accuracy of system-generated data and reports -- electronic audit evidence or “EAE” -- used in the performance of a control

►  Not testing completeness and accuracy of system generated data used to select control testing samples or to support our reliance for substantive tests

►  Not testing IT general controls over all applications that produce system-generated data or reports used in the performance or a control or in our substantive tests

►  Not testing appropriate controls over end-user computing solutions used in performance of controls


Page 14

Increased focus on issuer controls over EAE used in performance of controls

►  Auditor needs to better consider that the specific system-generated data or report is considered and tested within IT general control testing ►  Report changes need to be considered within change

management testing ►  Controls over access and changes to reporting tools (e.g.,

Hyperion HFM, Cognos, data warehouses) need to be considered

►  Auditor needs to better consider controls that issuer has in place over completeness and accuracy of underlying data

►  Auditor needs to better consider if system-generated data or reports used in performance of controls are subject to manual change, and if so the proper controls are in place


Page 15

Data and reports supporting the performance of internal controls

Example: Review of A/R aging report and allowance for doubtful accts calculation Section Poor example Better example

Does the control use EAE, and if so, what is the basis for our reliance?

Yes We tested the AR aging report by selecting a sample of 25 items and recalculating the aging buckets. We clerically tested the schedule We agree the total of the AR aging to the general ledger without exception. (Poor example because auditor performed direct testing of report)

Yes. IT General Controls for Great Plains have been tested and are effective. We have tested various controls over the completeness and accuracy of data in the accounts receivable, revenue and cash receipts SCOTs which are the primary inputs and outputs of data within the AR aging report (AR_1005) – see B10 and B11 walkthrough and TOC workpapers). Additionally, we have tested application control GP_20-The system ages all invoices based on invoice date) at workpaper Z_10 and determined this control over the aging is operating effectively. We confirmed through that testing that this control is not configurable. In performing the review of the calculation, in which the aging is exported from the system to Excel and manipulated to apply certain formulas to the aging buckets, the AR manager agrees key totals (aging bucket totals) to a screenshot of the AR aging at the time the report was run, which is attached as support to the journal entry recording the change in the AR aging (B10.4a). The parameters of the report as run in the system are captured on the Excel file in the report header and a screenshot of those parameters embedded in the Excel file by the credit manager (B10.4b). Additional procedures performed by the AR manager to validate the completeness and accuracy of the report (footing aging bucket totals as well as agreement of items greater than $10,000 that are specifically reserved to the system) are not documented explicitly in the supporting Excel file. We’ve inquired of the AR manager regarding the performance of these procedures on a consistent basis, and noted no exceptions and re-performed these procedures, noting no exceptions (B10.4b).


Page 16

Data and reports supporting the performance of internal controls Control: The allowance for doubtful accounts reserve calculation is reviewed

by the accounts receivable manager on a monthly basis.

Cash receipts

A/R subledger

Analysis prepared by

the credit manager

Sales and trade

receivables

Application

A/R aging report


Page 17



Cash receipts

A/R subledger


the credit manager

Sales and trade

receivables

Application

A/R aging report

Step #1: What data or reports are used in the performance of the control?


Page 18



Cash receipts

A/R subledger

Sales and trade

receivables

Application Great Plains

A/R aging report

Step #2: Is the data or report generated by an in-scope application?


the credit manager

Excel


Page 19



Cash receipts

A/R subledger


the credit manager

Excel - NO

Sales and trade

receivables

Application Great Plains - YES

A/R aging report

Step #3: Are ITGCs over the application or end user computing solution that generated the data or report effective?


Page 20



Cash receipts

YES

A/R subledger

Sales and trade

receivables YES


A/R aging report

Step #4: Have we tested specific controls over the completeness and accuracy of the underlying data? Are the controls effective?


the credit manager

Excel


Page 21



Cash receipts

A/R subledger

Sales and trade

receivables


A/R aging report - NO

Step # 5: Is data or report subject to manual change?


the credit manager

Excel - YES


Page 22

Data and reports supporting the performance of internal controls

►  Extent of identification and testing of controls over key

data and reports depends on: ►  Importance of the data or report to the functioning of

the control ►  Complexity of the calculations in a spreadsheet or

manipulation of the data in the preparation of the report ►  Generally, the “further away” from the application with

effective ITGCs, the greater the importance of controls over the data and reports used by management

►  Focus on the data and reports with greater importance to the functioning of the controls, particularly review controls, and higher complexity of calculations not performed by the application with effective ITGCs


Page 23

Example of controls over review of A/R aging report and preparation of bad debt allowance

EAE = A/R Aging Report

Quantities shipped are reconciled to quantities billed (Initiation)

The invoice amount is posted automatically into the customer’s account upon generation of the invoice (Recording)

The system ages invoices based on the invoice data (Processing)

On a monthly basis, the sub-ledger is posted automatically to the GL (Processing)

An AR reconciliation is performed by the senior accountant and reviewed for completeness and accuracy by the accounting manager (Processing)

The controller reviews the bad debt allowance calculation and approves the adjusting journal entry on a quarterly basis (Processing)


Page 24

End-user computing solutions

►  End-user computing solutions likely are not subject to IT-general controls ►  Excel files ►  Access databases ►  Dynamic data warehouse reporting tools ►  System-generated data in slide decks

►  Need to better consider issuer controls over end-user computing solutions ►  Input control – the company reconciles data back to source documents ► Access control – Access is restricted to authorized personnel and is

password protected ► Version control – Standard naming conventions are in place so only

current and approved versions are used


Page 25

Testing precision of management review controls


Page 26


►  Beyond verifying that the control occurred (e.g., evidence of signature) there was no evaluation of the review control’s effectiveness and level of precision

►  Cannot rely on absence of exceptions from substantive review as evidence controls are operating effectively (controls need to be tested directly)

►  Our evaluation of review controls should consider all evidence of their precision, sensitivity and ability to detect significant errors/misstatements

►  Verifying existence of management’s signature, by itself, does not test operating effectiveness

►  Our evaluation of review controls should consider how management identified errors/issues in the review and how the ensure that those errors/issues are resolved

►  Often related to financial controls (e.g., non-routine transactions like business combinations), but can impact IT general controls as well


Page 27

Example – periodic access review

►  Test of control – bad example: ►  Obtained evidence of review, saw review was signed off and some

updates were noted in the review listing

►  Test of control – good example: ►  Inquired with individual(s) performing review to understand how they

review and identify errors/exceptions ►  Obtained understanding of how access reports were generated and how

reviewer knows listings are complete ►  Observe the performance of the review ►  For each review tested, confirm the review was signed off ►  For each review tested, traced a sample of updates requested through to

updated system access ►  For each review tested, considered significant instances of inappropriate

access identified and their impact on the overall control environment


Page 28

Evaluation of controls at service providers (SOC reports)


Page 29

PCAOB inspection themes

►  Reliance on service organizations was either not identified or not appropriately documented to determine whether the service auditor’s report provided sufficient audit evidence about the effectiveness of relevant controls

►  Sub-service organizations that were scoped out of the report were not addressed (i.e., SOC 1 report was not obtained and there was no documentation of considerations and conclusion if such sub-servicers were deemed insignificant or not relevant)

►  Complementary user entity controls were either not sufficiently tested, or were not properly linked to engagement team testing of user controls that would address the relevant considerations

►  Update procedures were not properly performed or documented when the service auditor’s report did not sufficiently cover the entire audit period

►  Control exceptions identified by the service auditor were not evaluated to determine whether sufficient audit procedures to support our combined risk assessments were still appropriate to prevent or detect potential misstatements


Page 30

Why do we review SOC reports?

►  Many entities outsource aspects of their business to service organizations that provide services ranging from performing a specific task under the direction of the entity to replacing an entity’s entire business unit or function. These services are relevant to the audit when these services, and the controls over them, are part of the entity’s information system relevant to financial reporting (e.g., if the client uses electronic audit evidence from a third-party provider as part of a control activity).

►  If we plan to place reliance on controls at the service organization, we ordinarily obtain and review a service auditor’s report (SOC 1) covering a sufficient portion of the audit year (this includes sub-service providers of those organizations).

►  We review the SOC 1 report and document our evaluation of the service provider and their impact on the audit.


Page 31

Sub-service organizations

►  Service providers relevant to our audit may outsource part of their processes/controls to another third party, called a sub-service provider ►  Can be part of transaction processing (e.g., claims processing) ►  Can be part of IT environment (e.g., data center hosting)

►  The service organization will identify sub-service providers in their assertion, and the service auditor will identify sub-service providers in their opinion (these should be the same)

►  We must evaluate the audit impact of all identified sub-service providers (including IT sub-service providers) in our documentation


Page 32

Complementary User Entity Controls (CUECs)

►  Controls at the service provider alone do not ensure the accuracy of our client’s financial statements, and the SOC 1 report will outline control considerations for user (our client) of the service

►  For each CUEC, we should evaluate if the CUEC is relevant (e.g., does the CUEC directly impact financial reporting risk(s) that we have identified that the service providers’ controls help mitigate?)

►  For IT-related CUECs, IT specialists should be used and consider the client’s responsibilities in things like user access administration (e.g., who has access to transmit data to the service provider for processing) and testing/approving program changes from provider

►  For each CUEC deemed relevant to the financial reporting risk(s) that were identified, we must demonstrate that the client has the appropriate controls in place and we have tested the operating effectiveness of those controls (e.g., these controls should be defined as key SOX controls)


Page 33

Evaluating time period of the report and gap between year-end ►  Generally, to rely on a SOC 1 report, the report must cover at least six months

of our audit period. If the report covers less than six months and a second report is not available, we must consider/document how we are comfortable relying on the report with a smaller coverage period (and expect to be challenged on this). ►  At minimum, consider what controls are in place at the user entity that gives us

comfort that the client’s internal controls would detect a material misstatement made by the service provider if there is a large gap between the report end date and our client’s year-end date. The client’s controls must be sufficiently precise.

►  If there is a gap larger than three months between the report end date and our client’s year-end date, we again must document our considerations of how we are comfortable relying on the report with a large time period gap (and expect to be challenged on this). ►  At minimum, bridge letters should be obtained; but we should challenge if a bridge

letter alone is sufficient and how else the client gets comfortable over the service providers’ control environment (e.g., client controls over the reports/data).


Page 34

Evaluating control exceptions

►  The service auditor’s section of the report will summarize the test of controls performed and results of controls testing. Exceptions (often called deviations) will be noted in this section.

►  Auditor should evaluate all relevant exceptions noted in review documentation ►  All exceptions relevant to control objectives that mitigate identified financial

reporting risks should be evaluated ►  Exceptions related to ITGCs supporting relevant applications that mitigate identified

financial reporting risks should be evaluated

►  The exceptions should show an appropriate amount of evaluation of the risk of the exception. A blanket “This exception has no impact on our audit approach” is generally not sufficient and could lead to increased scrutiny during a quality inspection.


Page 35

Evaluating SOC reports – other considerations

►  Management should review/evaluate SOC reports as part of their testing of controls for management’s opinion on their internal controls over financial reporting

►  PCAOB appears to have a list of “problem reports”, and will challenge how teams addressed these “problem reports” when used in the audit of an issuer

►  Some chatter on PCAOB auditing service auditors who issue SOC reports in the near future


Page 36

The future of audit


Page 37

Audit transformation activities

►  Intelligent data - a broader use of data analysis techniques and tools to support risk assessments and substantive analytical procedures

►  Rather than a random sample, using data analysis to look across a population of transactions in its entirety to identify anomalies and unusual items and highlight important trends

►  Development of a new audit tool and supporting tools/ enablers to increase efficiencies in our audit and respond to the inspection themes discussed


Page 38

Summary


Page 39

Key areas of IT / Internal Audit involvement

►  Process and data flow diagrams ►  Assisting with development

►  Providing interface details

►  Testing interface / reconciliation controls

►  System-generated data and reports ►  Identifying key reports and data

►  Documenting the sufficiency of supporting controls

►  ITGC testing over reporting systems and report / query changes

►  Identifying / testing controls over end user computing


Page 40

Key areas of IT / Internal Audit involvement

►  Review controls ►  Documenting additional detail in tests of review controls

►  SOC reports ►  Mapping SOC control objectives to significant processes and risks

►  Evaluating identified sub-service organizations

►  Mapping CUECs to controls and testing relevant controls

►  Evaluating control exceptions identified


Page 41

Questions?


Documents

PCAOB Inspection Themes - PhxSACphxsac.com/.../2013/06/PCAOB-Inspection-Themes.pdf · Payroll Files A B C D E F G H I J K L M N A systems interface diagram is a key source of