View
1
Download
0
Category
Preview:
Citation preview
1 Insight. Oversight. Foresight. SMMichigan Texas Florida North Carolina
Data Analytics:Implementation
Presented by:Robin D. Hoag, CPA, CGMA, CMCShareholder, Financial Institutions Group
Region 3 MeetingSeptember 18 – 20, 2019Lansing, Michigan
2
Financial Institutions GroupToday’s Objectives
• Introduction to data analytics• Why it’s important to Internal Auditors• Overview of the key elements, attributes, challenges• Steps in the data analytics process• Data analytic tools• Roles and responsibilities• Applications for Internal Audit• Resources
3
Financial Institutions GroupWhat is Data Analytics? Definition
• The process of inspecting, cleansing, transforming, and modeling data with the objective of highlighting meaningful information, suggesting conclusions, and supporting decision-making.
4
Financial Institutions GroupData Analytics: More!
• Problem-solving process• Extracts insights• Historical, real-time, or predictive• Data analytics (DA) can be:
• Risk-focused (i.e., controls effectiveness, fraud, waste, policy/regulatory non-compliance)
• Performance-focused (i.e., increased sales, decreased costs, improved profitability.)
5
Financial Institutions GroupFOCUS on Relationships
• Identify and interpret relationships among variables to facilitate decision-making using the Five W’s:• Who• What• Why• Where• When
6
Financial Institutions Group
Why Is Data Analytics Important To Internal Audit?
Strategic Area EnhancementCredit Union Expectations Audit coverage, quality, business
impact, on a finite audit budgetRegulatory Expectations Stronger assurance and quantifiable
resultsCompetitive Landscape Strengthen capabilities
Seek new talentIncreased Value Deeper discussion on issues
Develop/strengthen relationships
Talent Development Strengthen business skills Appeal to other staff members Boost recruiting
7
Financial Institutions Group
Why Is Data Analytics Important To Internal Audit?
• Internal audit departments leverage data analytics in order to:• Identify additional risk• Increase scope and coverage - assurance• Better understand existing risk through data• Identify holes in control systems• Augment limited resources• Provide insights to management
8
Financial Institutions Group
Why Is Data Analytics Important To Internal Audit?• Some areas that benefit from data analytics:
• Loan operations• Accounting and general ledger transactions• Accounts payable• Payroll• Deposit transactions• Compliance• Legal and regulatory compliance
9 Insight. Oversight. Foresight. SM
Challenges To Using Data Analytics
10
Financial Institutions GroupPeople
• Limited resources (financial and human) to execute on a sustained basis
• Appetite for investment in time and training needed to develop an effective DA / AI process
• Someone needs to create, run, and maintain queries• Proficiency using analytic software• Proficiency in performing analysis
The top barrier for implementation of big data analytics is “inadequate staffing or skills for big data analytics.”
(Source: The Data Warehousing Institute (TDWI))
11
Financial Institutions GroupPeople: Senior Analysts
12 Insight. Oversight. Foresight. SM
Past, Present, and Not So Distant Future
13
Financial Institutions Group
“Computers will disrupt work habits and replace old jobs with ones that are radically different.”
Isaac Asimov predicts 2019 (in 1983)
14
Financial Institutions Group
We are in the Midst of a Data-Enabled Shift that is Transforming Our Stakeholders and How We Serve Them
We have transformed from a “Digital Driven Economy” to where the “Economy is Digital”
15
Financial Institutions GroupImpact of Analytics
16
Financial Institutions Group
Artificial Intelligence: Introducing the Technology
“I think the simplest definition of AI is that it’s a machine or a computer exhibiting the characteristics that we would normally associate with human intelligence”
Robin Grosset, CTO MindBridge AI
17
Financial Institutions GroupLet’s Get a Bit Technical
Artificial Intelligence
1956+
Early artificial
intelligence stirs
excitement
Machine Learning1980+
Machine learning begins
to flourish
Deep Learning2010+
Deep learning breakthroughs drive AI
boom (Google Cat)
18
Financial Institutions Group
Comparing Ensemble AI with Traditional Rules
Rules alone: Ranked risk score
Machine learning: Ranked risk score
Recent real-world example:In 1.6M transactions, flagging those of interest • Machine learning identified the fraud as 34th riskiest transaction• Traditional rules ranked the fraud ~31,000th in priority to examine
900X improvement
19
Financial Institutions Group
What Can Machines Do Better Than Humans?
20
Financial Institutions GroupTrust Within AI-Enabled Analytics
21
Financial Institutions GroupTraditional CAATs
Traditional CAAT tools are rules-based, with limited coverage:
22
Financial Institutions GroupUsing AI
23
Financial Institutions GroupThe Art of Evaluating AI
Its potential for good and progress will only fully emerge when algorithms become explainable in simple language
24
Financial Institutions GroupTake-Aways
• The pace of technological disruption is accelerating and will continue to impact us
• Big data and improved processing power, drive the development of AI applications
• Big data is underutilized in the internal audit space and AI can help harvest those opportunities
• AI can help with narrow tasks (data processing, risk scoring, search) with the human focusing on cognitive tasks (communicating and advising)
• Adopting AI is a journey and not a silver bullet. Educate yourself and start small.
25 Insight. Oversight. Foresight. SM
Data Analytics Tools
26
Financial Institutions GroupData Analytics Tools
• The right data analytic software will:• Handle large data sets efficiently• Integrate well with big-data• Include wide-array of analytical and statistical functions and procedures• Be relatively easy to program• Log procedures performed on data• Allow users to easily re-run analysis with minor changes• Be scalable with regards to the platform• Ensure the vendor’s vision is inline with the organization’s vision• Include training and support
27
Financial Institutions GroupData Analytics Tools
• Basic• Microsoft Excel• Microsoft Access
• Integrated query tools• PeopleSoft• SAP• Oracle• JDE
• Specialized DA visualization software• Tableau• Qlikview/Qlik Sense
• Data analytics• Mindbridge• Treasure data• NICE• Periscope• Zoho• DXC• Sisense Data analytics• Inflow• AWS
• Specialized auditing software• ACE, IDEA, Arbytus, SAS
28 Insight. Oversight. Foresight. SM
Data Analytic Team’sRoles and Responsibilities
29
Financial Institutions Group
Key Roles and Responsibilities: Internal Audit
Splitting the analytics roles - essential ingredients…
1. Audit management & staff Provides comprehensive understanding of the audit objectives Identifies opportunities to introduce data analytics into the audit
process Drives demand through personal insights and relationships Keeps focus on solving audit related issues
30
Financial Institutions Group
Key Roles and Responsibilities: Internal Audit2. Data analytics SME
Proficient in use of DA tools and is able to design queries and manipulate data easily Experienced auditor with a knack for analysis May have knowledge of advanced statistical topics and modeling Excellent problem solving skills
31
Financial Institutions Group
Key Roles and Responsibilities: Internal Audit3. Data specialist
Strong programming and coding proficiency Has been a database administrator or systems analyst Has spent time as developer and has built applications Expertise in core IT related functions in querying, data extraction,
cleansing, and manipulation
32 Insight. Oversight. Foresight. SM
Data Analytic Applications for Internal Audit
33 Insight. Oversight. Foresight. SM
Practical Examples
34
Financial Institutions Group
Data Analytics Applied to Accounts Payable
• AP tests can be designed to address risks, cost savings and/or recoveries
• Data analytic tests can be designed to identify any of the following:• Improper disbursements• Duplicate payments• Unapproved purchases• Payments for items not received• Payments in excess of approval levels• Missed discounts or credits
35
Financial Institutions GroupAP Analysis: A Few Ideas
• Improper payments or questionable disbursements• Detect duplicate payments using dates, payees, vendor invoice numbers and
amounts*• Identify invoices or payments to vendors without a valid purchase order*• Look for invoices from vendors not in approved vendor file• Find invoices for more than one purchase order authorization*• Identify multiple invoices with the same item description*• Extract vendors with duplicate invoice numbers*• Look for multiple invoices for the same amount on the same date*• Find invoice payments issued on non-business days (Saturdays and Sundays)• Identify multiple invoices at or just under approval cut-off levels• Identify credits issued by or outstanding with vendors• Identify goods invoiced and paid, but not shown as being received• Look for payments to vendors not on contract.
* signifies potential for recoveries
36
Financial Institutions GroupAP Analysis: A Few More Ideas
• Look for multiple payments to the same vendor on the same date or for the same amount (excluding recurring charges, such as rent)*
• Stratify vendor balances, check amounts, invoice amounts, PO amounts, etc., for unusual trends or exceptions*
• Calculate and validate annualized unit price changes in PO/payments for the same product over time*
• Review sequence of check numbers for gaps• Identify payments where no discount was taken*• Review changes to the vendor master file
* signifies potential for recoveries
37
Financial Institutions GroupAP Analysis: A Few More Ideas
• Look for multiple payments to the same vendor on the same date or for the same amount (excluding recurring charges, such as rent)*
• Stratify vendor balances, check amounts, invoice amounts, PO amounts, etc., for unusual trends or exceptions*
• Calculate and validate annualized unit price changes in PO/payments for the same product over time*
• Review sequence of check numbers for gaps• Identify payments where no discount was taken*• Review changes to the vendor master file
* signifies potential for recoveries
38
Financial Institutions GroupAccounts Payable Schemes
• Phantom vendors• Match names, addresses, phone numbers, bank accounts
and taxpayer identification numbers between vendor source documents.
• Verify existence of vendors using a PO Box for an address
39
Financial Institutions GroupAccounts Payable Schemes
• Kickback or conflict-of-interest• Vendor prices greater than standard• Price increases greater than acceptable percentages• Continued purchases in spite of high rates of returns, rejects,
or credits• High volume purchases from one vendor• Frequent change orders• Identify payments to vendors with same names, addresses,
phone numbers, etc., as employees
40
Financial Institutions GroupAccounts Payable Schemes
• Bidding and contracting• Patterns of rotation among vendors• Bids that are exceptionally lower than those of other vendors• Low winning bids followed by numerous change orders• Excessive use of one contractor in a competitive field• Patterns in awards to vendors• Identical bids• Multilateral drops in bid prices (accompanied by the entry of new
competitor)• Competitors with the same addresses, principals, sales agents, phone
numbers, etc.• Vendors with same names, addresses, phone numbers, etc., as
employees.
41
Financial Institutions GroupOther Applications for Data Analytics
• Accounts Receivable• Valid sales orders• Accurate product pricing• Authorized shipments• Proper invoicing• Valid cash receipts• Timely collections & write-offs• Sales contract compliance• Other adjustments
• Payroll• Accurate & authorized payments• Timely & accurate hires &
terminations• Reasonable OT & commissions• Proper timekeeping & attendance• Search for non-existent
employees and other payroll schemes
• Comparison of periods for unusual trends
42
Financial Institutions GroupOther Applications for Data Analytics
• General ledger• Journal entries• Closing activities• Adjustments
• Master files• Members• Loans
• Travel and entertainment• Purchasing cards
• Data quality• Reasonable• Within expected range• Validity• Complete
• Compliance• BSA/AML• Money service business
43
Financial Institutions Group
Rules Based View of Audit Procedures (Legacy CAATs)
A legacy but still common practice encourages the use of audit testing tools where each test is done one-by-one. Each test is performed in isolation and then examined by the auditor to look for issues.
This focuses the auditor on specific issues and helps to verify the presence of controls and good accounting practices.
These techniques increase the odds of finding anomalies. But this can be improved
44
Financial Institutions GroupRules Based Tests
Typical audit test pattern: Tests performed one by one and human auditor inspects results
45
Financial Institutions GroupHuman Hypothesis Generation
What can I say about these transactions? What else is interesting?
A Human Centric View of
Analysis
46
Financial Institutions GroupRules Based Testing: Combinations
By considering all tests together and viewing a combined result score, you can see higher risks floating to the top which fail more tests.
More Risk Factors Fewer Risk Factors
Ranked Interest Score
Conclusion: Combining the test outcomes and understanding the intersect produces better results when trying to understand transaction risk.
47
Financial Institutions GroupA Real World Scenario: Rules Combined
Rules-based testing can create wide (large) buckets of transactions
48
Financial Institutions GroupA Real World Scenario: Machine Learning
Machine learning techniques create much better differentiation of transaction risk factors.
Ranked Risk Factor Scores Histogram of Risk Factor Scores
Good distribution and separation enables appropriate focus and use of time
Smooth range of risk scores, no large buckets
49
Financial Institutions Group
Quick Comparison: Machine Learning vs. Rules
Conclusion: Machine learning is proving a better tool to understand risk factors and it produces better outcomes than rules based approaches alone.
In an example scenario “Unusual Cash Disbursements”: • Rules: flagged a transaction as normal putting it in the 30th percentile of risk • Machine Learning: flagged the same transaction in the 3rd percentile of risk
50
Financial Institutions GroupMachine Learning: Rare Flows
Characteristics of potentially inappropriate journal entries• Made to unrelated, unusual (e.g., unusual combinations of
debits and credits), or seldom-used accounts
Audit Rationale
51
Financial Institutions GroupMachine Learning: Rare Flows
Algorithm Finds: Unusual Journal Entries based on frequency of account interactions
For each and every transaction (100%), measure how often they occur and flag rare interactions
Algorithm Class: Unsupervised Machine Learning
52
Questions?
53
Financial Institutions GroupReferences
ACFE (2018). Report to the Nations. Retrieved 4/30/2019: https://www.acfe.com/report-to-the-nations/2018/
AICPA (2018). Beyond robotics: How AI can help improve audit process. Retrieved: https://blog.aicpa.org/2018/08/beyond-robotics-how-ai-can-help-improve-the-audit-process.html#sthash.3Lzmii1i.uAOqhSbF.dpbs
AICPA (2019). A CPA’s Introduction to AI: From Algorithms to Deep Learning, What you Need to Know. Retrieved:
ARRIA (2018). Why the Finance Industry is Ripe for AI Disruption. Retrieved: http://blog.arria.com/why-the-finance-industry-is-ripe-for-ai-disruption
https://www.aicpa.org/content/dam/aicpa/interestareas/frc/assuranceadvisoryservices/downloadabledocuments/cpas-introduction-to-ai-from-algorithms.pdf
BBC (2018). Carillon: The audit industry’s existential question. Retrieved: https://www.bbc.com/news/business-44201251
Change.org(2019). Change Oxford English Dictionary’s Archaic Definition Of The Word ‘Accountant'. Retrieved: https://www.change.org/p/oxford-english-dictionary-change-oxford-english-dictionary-s-archaic-definition-of-an-accountant?utm_source=twitter&utm_medium=social&utm_campaign=change_oed
CNBC (2019). 40% of A.I. start-ups in Europe have almost nothing to do with A.I., research finds. Retrieved: https://www.cnbc.com/2019/03/06/40-percent-of-ai-start-ups-in-europe-not-related-to-ai-mmc-report.html
54
Financial Institutions GroupReferences
EY (2018). How artificial intelligence will transform the audit. Retrieved: https://www.ey.com/en_gl/assurance/how-artificial-intelligence-will-transform-the-audit
Financial Times (2018). PwC’s failure to spot Colonial fraud spells trouble for auditors. Retrieved: https://www.ft.com/content/c2cc45d6-f1f6-11e7-b220-857e26d1aca4
Forbes (2016). How Artificial Intelligence Will Transform The Delivery Of Legal Services. Retrieved: https://www.forbes.com/sites/markcohen1/2016/09/06/artificial-intelligence-and-legal-delivery/#20c428cf22cd
Forbes (2018). How Much Data Do We Create Every Day? The Mind – Blowing Stats Everyone Should Read. Retrieved: https://www.forbes.com/sites/insights-kpmg/2018/10/19/artificial-intelligence-real-breakthroughs-the-practice-and-promise-of-ai-in-auditing/#1aee86126c10https://www.forbes.com/sites/bernardmarr/2018/05/21/how-much-data-do-we-create-every-day-the-mind-blowing-stats-everyone-should-read/#7ef6f8cf60ba
Forbes (2018b). Artificial Intelligence, Real Breakthroughs: The Practice And Promise Of AI In Auditing. Retrieved:
Huffpost (2015). Disrupting Today’s Healthcare System. Retrieved: https://www.huffpost.com/entry/disrupting-todays-healthc_b_8512200
55
Financial Institutions GroupReferences
• Journal of Accountancy (2011). Special Focus Report: Going Paperless. Retrieved: https://www.journalofaccountancy.com/news/2011/jul/july2011goingpaperless.html
• Markets and Markets (2019). Artificial Intelligence Market by Offering (Hardware, Software, Services), Technology (Machine Learning, Natural Language Processing, Context-Aware Computing, Computer Vison), End-User Industry, and Geography –Global Forecast to 2025. Retrieved: https://www.marketsandmarkets.com/Market-Reports/artificial-intelligence-market-74851580.html
• Riedl,Mark (2017). Human-Centered Artificial Intelligence. Retrieved: https://medium.com/@mark_riedl/human-centered-artificial-intelligence-70b019f956d1
• Spangler, William et al. (pending publication). Educating Business Students for the Age of Intelligent Machines: A Framework for On-Line AI-enabled Learning.
• TechTerms (no date). Data. Retrieved 4/30/2019: https://techterms.com/definition/data• The CPA Journal (2017). Big Data in Business Analytics: Implications for the Audit Profession. Retrieved:
https://www.cpajournal.com/2017/06/26/big-data-business-analytics-implications-audit-profession/• The Globe and Mail (2017). Supreme Court says Livent auditors liable but sets conditions. Retrieved:
https://www.theglobeandmail.com/report-on-business/supreme-court-says-livent-auditors-liable-but-sets-conditions/article37393018/
• The Guardian (2017). BT loses almost £8bn in value as Italy accounting scandal deepens. Retrieved: https://www.theguardian.com/business/2017/jan/24/bt-loses-7bn-in-value-as-italian-accounting-scandal-deepens
• The Guardian (2018). KPMG to fine staff £100 for late time sheet. Retrieved: https://www.theguardian.com/business/2018/dec/20/kpmg-to-fine-staff-for-late-time-sheets
• Wikipedia (no date). Optical character recognition: Retrieved: https://en.wikipedia.org/wiki/Optical_character_recognition• Wikipdedia (no date). Cloud computing. Retrieved: https://en.wikipedia.org/wiki/Cloud_computing• Wikipedia (no date). Blockchain. Retrieved: https://en.wikipedia.org/wiki/Blockchain• Wikipedia (no date). Big data. Retrieved: https://en.wikipedia.org/wiki/Big_data• Wikipedia (no date). Machine Learning. Retrieved: https://en.wikipedia.org/wiki/Machine_learning• World Economic Forum (2018). The Future of Jobs Report 2018. Retrieved: https://www.weforum.org/reports/the-future-of-jobs-
report-2018
56 Insight. Oversight. Foresight. SM
Thank You!
Michigan Texas Florida North Carolina
Robin D. Hoag, CPA, CGMA, CMCShareholder, Financial Institutions GroupOffice: (248) 244-3242Cell: (248) 709-1270hoag@doeren.com
Recommended