Energy Future Holdings (EFH)Inclusion of Data Analytics into the Internal Audit Lifecycle

June 3, 2015

Starting Place – Baseline Questions Pertaining to the utilization of data analytics in the internal audit

lifecycle, where are we currently?

What is our desired end state?− Where do we want to use it: Planning? Execution? Reporting?− Which Business Units?− Which systems will we use?

How soon do we want to get there?

How do we get there?− Existing Resources, Dedicated Resources, Additional Resources

How do we measure success?− Existing Data Analytics− Likely Easy Wins

1

The 2013 PwC State of the Internal Audit Profession Survey indicated that the most commonly underperforming IA attributes are…

2

Promoting quality improvement and innovation

Obtaining, training and/or sourcing the right talent to match the organization’s risk profile

Leveraging technology such as automation, data and advanced analytics

So … What is Data Analytics?

3

A process of inspecting, cleaning, transforming, and modeling data with the

goal of discovering useful information, suggesting conclusions, and supporting

decision making.

CAE’s Expectations for Data Analytics1

4

59% of CAEs plan to add data analytics skills in the next 18 months

“Among those who intend to expand the use of analytics, 71% lack a well-developed plan to do so.”

- Biggest barriers:

• Inefficient data collection processes and tools

• Finding capabilities and resources to support vision

1 Source: 2013 PwC State of the Internal Audit Profession Survey

CAE’s Thoughts on the Use of Data Analytics1

5

1 Source: 2013 PwC State of the Internal Audit Profession Survey

85%81%74%71%31%

Data analytics are used regularly

Plan to expand use of data

analytics but do not have a well-developed plan

Data analytics are important to

gaining a better understanding of

risks

Data analytics are important to

improving the quantification of

issues

Data analytics are important to

strengthening audit coverage

Percent of CAEs who agree

Common Benefits of Data Analytics1

6

Increased efficiency (often after year 1)

Productivity and cost savings (broaden assurance scope without increasing staff size)

Reduced audit risk (stratifying the population, thereby honing risk assessment)

100% testing (providing greater assurance)

Standardization (less reliance on individual “super auditors”)

Enhanced understanding (data profiling/trending)

Streamlined data correlation (automatic match)

Reduced travel (data accessed from anywhere)

1 Source: MIS Training Institute internal survey

Common Roles Associated with Data Analytics1

7

Data Analyst- Access, prepare and make data available- Perform advanced analysis and create analytics and results for sharing

Staff Auditors- Review and interpret results- Perform simple analysis independently

Audit Directors- Oversight of analytic-enabled audit activities

1 Source: Global Technology Audit Guide 16

Data Analytics Across the Audit Process1

8

1 Source: Global Technology Audit Guide 16

Planning & Preparation

Fieldwork & Testing Reporting

Identify data-driven risk indicators- Current- Emerging

Use to refine risk-oriented audit plan

Prepare automated test routines in advance of fieldwork

Make the above available to audit team so able to hit the ground running

Provide trending and profiling Provides consistency because

same factors are applied across all audit areas

More accurately quantify findings Reduce subjectivity and

interpretive judgment Ability to dig through data already

acquired to mine for other issues and trends

Increased assurance through 100% testing

When sampling is required, ability to more intelligently sample

More easily re-create and automate tests in subsequent periods

Increase auditor understanding of process(es)

Data Analytics Capability Levels1

9

1 Source: Global Technology Audit Guide 16

Level 1: Basic

Level 2: Applied

Level 3: Managed

Level 4: Automated

Level 5: Continuous

Supports specific audit objectives

Ad-hoc, often unplanned use

Limited audit staff

Centralized and structured knowledge management:- Data- Audit tests- Results- Supporting docs

Collaboration Controlled, secure

access Well-documented

procedures

Fully integrated into target audit processes

Includes both audit planning and fieldwork

Structured approach Tests re-used and

added in subsequent audits

Established data access protocols

Suites of tests available to audit team

Concurrent ongoing auditing of multiple areas

Structured issue reporting

Management monitoring of own process

IA assesses monitoring activities

Common Data Analysis Tools

10

Microsoft Excel- Quick, ad-hoc analysis of small to moderate file size- Easy formatting and graphing of results; easy distribution to others- Basic automation through macros

Microsoft Access- Relates/combines information from multiple tables- Handles larger file sizes than Excel- Complex analysis possibilities

ACL- Desktop and Exchange packages- Heavily used in audit community- Handles very large data volumes- Good for automated analysis for continuous/repetitive use

Idea / Spotfire- Handles data from multiple systems- Handles very large data volumes- Visual and graphical tools

Common Data Analysis Tools (cont’d)

11

IBM Cognos- Handles data from multiple systems- Handles very large data volumes- Visual and graphical tools

SQL- A standard computer language for relational database management and

data manipulation- Used to query, insert, update and modify data from various systems

Keys to Success1

12

Align strategy with audit plan, risk, goals and objectives

Manage like a program

Develop uniform practices and procedures

Assign responsibility

Document the intent of analysis

Understand the process

Understand the data

Document the content of analysis

Ensure results are accurate and appropriate

Establish a peer or supervisory review process

Standardize procedures and tests in a central repository

Safeguard source data from modification or corruption

Develop strategies to address potential impact on production processing

Treat training as a continuous process

Aim for constant improvement

1 Some data analytics are performed within processes at EFH

EFH Data Analytics – Vision & Objectives

13

Vision:Integrate data analytics as a standard audit practice and support the expanded utilization of continuous auditing and monitoring practices across EFH. Included will be:

Objectives: Embed data analytics into audit processes and procedures to enable:

1) Integration of expanded data analytics into the annual risk assessment and audit planning processes,

2) Assessment of data analytics applicability during the planning and testing phases for designated audits, and

3) Leveraging of data visualization to enhance audit reporting. Establish a technical data analytics environment with a targeted business

data model to support an ongoing data analytics program. Identify opportunities to expand continuous auditing in high risk and

recurring audit areas to increase audit coverage.

14

Implementation Project – Overview

EFH IA decided to develop and implement an information management and data analytics capability to support EFH’s entire audit spectrum (fieldwork, reporting, planning, & risk assessment). The following tasks and deliverables were completed from a functional and technical perspective for this effort:

Current State Assessment Gap Analysis Vision and Roadmap Proof of Concept Develop Processes & Procedures

Definition Information Gathering Analysis and Planning

Technical

Project Management

Project Planning &

Kick-Off

Duration Estimates TBD (e.g., 6-8 weeks)

Business

Conduct Audit

Leadership Interviews

Document Existing

Capabilities & Landscape

Analyze and Document

Current State Architecture

and Technology

Determine Audit &

Information Processes

Current State Assessment

Perform Business

Need & Value Assessment

Perform Current /

Future State Gap Analysis

Need & Value

Summary

Gap Analysis Summary

Vision & Roadmap Program

Planning

Master Plan

Executive Summary

Planning & Support

Documents

Execution

POC

3* Months

Develops vision for the organization based on key business drivers Current state assessments identify areas for improvement and effectiveness from a cost and productivity perspective Roadmap is built upon business need and value while also addressing possible technology gaps Roadmap defines an iterative / phased approach to implementation Incorporates proof of concept to prove technology and analytic benefit Develops multi year plan for future releases based on business need

Pilot Phase 1

4-6 Weeks2 Weeks

Strategy, Roadmap, and Plan Processes and Deliverables

Audit Plan

Completed

Process

Project Management

Current

Next

15

Defined• Capabilities developed and adopted

• Capabilities used to drive audits • Defined goals and standardized processes and tools

Initial Developing• Limited capabilities

• Ad hoc activities resulting in unpredictable and inefficient performance

• Success based on individual competence

Advance• Capabilities are well developed and practiced with appropriate governance

• Data sources are readily available • Activities begin to become repeatable and CM metrics are developed

Prog

ram

Mat

urity

Industry Leader• Scale is achieved for department specific teams • improvement methodologies are implemented • monitoring occurring for metrics and controls

Predictive

Mitigated

Managed

Reactive

As Needed

High

Low

Industry Maturity LevelWhere is EFH today and where we want to be …

EFH

EFH Future State 2016 - 2017

Current

Return on Investment, Effeciency, Insight

16

Recommended Phases & TimelinesFEB APRJAN JUN JUL AUGMAY OCT NOV DECSEPMAR

2015Fo

unda

tion

Audi

t Spe

ctru

m

17

Technology

Measurement

Process/ Practice

Phase 1

EFH - LUMP

IA Road Map

Current StateFuture StateRoad MapPOC - AP

Contract Compliance

AP (Non PO – PO)

Phase 2Phase 3

Functional Data

Functional Data

Functional Data Technology

Measurement

Process/ Practice

LUMP - EFH

Plant Operations

Mine Operations

Audit Policies/Procedures

Systems – FIM / MAX. / MyExpense

Incentive Plans

Reporting Data Quality

Systems – FIM / Max. / Workday

Systems – Epsilon / CATs / Lodestar

IT – LUMP-TXUE

Environment and Safety

Technology

Con

tinue

Nex

t Pag

e

**Technology includes Data Model, Data Capture and Interfaces. **Measurement includes Data Quality, Dashboards and Data Analytics. ** Process/Practice include Data Integrity, Data Standardization and Master Data Management.

Planning, Testing and Reporting associated with each phase

T&E

Current Data Analytics at EFH

18

Name Description Business Area

1 T-Card Late Payment Notification

Review late payments of Travel Card transactions to reinforce policy. Maintain a log of late notices from 2011 to allow historical review.

Travel and Expense

2 Improper T-Card Use Review late payments of Procurement Card transactions to identify unusual usage transactions. Follow-up as appropraite based on fraud potential.

Travel and Expense

3 Unassigned T-Card Transactions

Review Travel Card transactions which have not been submitted on an expense report for manager review. Follow up with manager and employee as appropriate.

Travel and Expense

4 Deal Confirmation Validation

Match Deal Confirmation Spreadsheet with the deal tickets to test that deals are approved by counter-party.

Trading

5 Zainet Deal Update Review deal changes in Zainet to ensure that any changes were made by a trader are consistent with the books he/she operates in.

Trading

6 Deal Change Monitoring Analyze deal key attribute changes (after 3-day entry period) to identify any patterns which could represent an inappropriate change. Any identified anomalies are reviewed with the business owner.

Trading

7 Inter-Desk Deal Review Review transactions between the various deal books to ensure that they are in balance and that they have not been improperly changed between "Accrual" and "Mark to Market" to manipulate earnings.

Power Sales

8 Market Power Price Review Correlate wholesale electricity prices at 4 load centers closest to our plants and investigate any correlation between prices spikes and Luminant plant outages.

Power Sales

9 Controls Monitoring Analyze and report against the Compliance System controls by area and type to provide management with a view of the overall control coverage and compare to historical coverage to identify changes to the controls environment during the period.

Compliance

10 Duplicate Invoice Compare Invoice (Vendor, Invoice Number, and Invoice Date to the voucher before and after to see if it is a duplicate. Set flag to 1 if it is a duplicate.

Account Payable

19

Name Description Business Area

11 Invoice Number Order Compare Invoice Number to the voucher before to see if it is ouot of order (INVOICE_ID < previous INVOICE_ID. Set flag to 1 if it is out of order.

Account Payable

12 Invoice Entry Day of Week Set Flag to 1 (True) if voucher was an Online Entry and entered on a Saturday or Sunday. Account Payable

13 Invoice ID Pattern Mismatch Set Flag to 1 (True) if the voucher number pattern is inconsistent with the voucher before or after.

Account Payable

14 Invoice Amount Check Set Flag to 1 (True) if the amount is more that one Standard Deviation form the average for that vendor.

Account Payable

15 Invoice Amount Round Numb Set Flag to 1 (True) if the invoice amount is an even dollar amount and not a prepay. Account Payable

16 Invoice Amount Benford TestSet Flag to 1 (True) if the first 2 digits are inconsistent with the expected Benford occurrence percentage by a factor > 5.

Account Payable

17 Invoice Date Receipt Period Set Flag to 1 (True) if the Invoice Date occurrence timing is more than one standard deviation less than normal date occurrence time .

Account Payable

18 Invoice Number Spread Set Flag to 1 (True) if the average spread between invoice numbers is < 3. Account Payable

19 Late Payment Set Flag to 1 (True) if the voucher Payment Date is greater than the invoice due date (and not a canceled voucher).

Account Payable

Current Data Analytics at EFH

POC - Executive Dashboard

20

POC - Duplicate Payments

21

POC - Early Payment

22

EFH Internal Audit & SOX ComplianceData Analytics Request Execution Process

AuditProject

Investigation

Management Request

Audit Team---------------------------------

Data Requirements Identification

Data Analytics Team

---------------------------------------Data Analytics Assessment &

Evaluation

New Analytics Required

ExecuteData Analytics

ScheduleData Analytics

Assess / Tune

UtilizeData Analytics

Assess Data Analytic Future Use

One Time Use – No Future Plans

Implement as aContinuous Audit Stream

Support Client’sContinuous Monitoring Usage

Install Data Analytics

System PackageConfigureOptimize

Document

Document / Archive Data

Analytics Objects

Review Analytics with Client

Transition to Client

OR

OR

Analytics Already Exists

Data Analytics not applicable

End

OR

OR

DevelopNew

Analytic

TuneExistingAnalytic

Origination Assessment Execution

CaptureFinalize

GenerateData Reports,

Visuals, and/orExceptions

Next Steps for EFH

24

Modify audit procedures to include data analytics considerations throughout the audit lifecycle (i.e., during familiarization & planning, risk assessment, fieldwork and reporting)

Revise TeamMate audit programs to reflect new processes associated with data analytics

Continue to develop analytics that can be used for continuous auditing (not associated with audits in the annual plan)

Work with the businesses to develop analytics that can be used for continuous monitoring