Using Metrics to Inform, Enforce and

PrepareJulie Cole, Duke University

Todd Leovic, Duke UniversityPanda Powell, UNCW Sarah Smith, UNCGA

Learning Objectives• Present examples of formalized performance and

monitoring programs implemented at the:• System-level• Campus-level• Department-level

• Present example of how performance and monitoring results can be used to determine and train to needed competencies

Setting the Stage• The UNC System and Duke have developed programs for monitoring

performance using data – both have evolved. Commonalities include:• Management support• Campus/unit-level ownership• Regular data collection and management review

• Individual campuses and departments are utilizing this information to build more compliant structures

• The confluence of operational data and informed training helps the institutions become even more prepared, informed and compliant

Informing Compliance: UNC- GA and Duke University

Sarah M. Smith UNC-GAJulie Cole Duke

UNC FIT History

• Presidential and Board of Governors Priority• Operational Risk Assessment• Strengthen the control environment• Improve operational efficiencies

• General Accounting, Financial Aid, Contracts and Grants, Student Accounts, Capital Assets

Inform

UNC FIT Organization/Design• UNC Project Management Officer• Executive Steering Committee/BOG Audit Committee• Campus-based Project Managers• Process Leads and Advisory Teams• Baseline Standards• Performance Indicators• Monitoring

Inform

C&G Standards• Standards ≠ Policy• Dynamic and reviewed regularly• Identify critical elements of the grant life cycle and best practices • Document Operating Standards

• Account set-up• Time and effort• Grant Management• Billings/drawdowns• Reporting• Closeout

Inform

Key Performance Indicators

• Indicators for Each Operating Standard• Establish Performance Targets• Report Indicators Monthly• Management Review (campus and BOG)• Below Target/Negative Trending• Remediation plan

Inform

Key Performance Indicators• Number of days to fund set up• Number/percent of T/E reports outstanding• Number/percent of late salary transfers (>90 days old)• Number of A-133 certification letters outstanding• Dollar/percent of accounts receivable >60, >90, >120 days• Number/percent of funds with end date >120 days old and

fund is still active

Inform

Monitoring• Compliance Checklist/Assessment of Internal Controls• On-site Monitoring Visit• Risk-based, Informed by KPI Results• Document Review• Report to Management

Inform

UNC FITPerformance Funding Model

• Adopted FY13• 10 Measures to Determine Eligibility for Funds• Performance linked to enrollment growth

projections• UNC FIT Composite Score• Audit, Internal Control Assessment, Compliance

with Standards, KPIs and MonitoringInform

12

Financial Assessment Management

ManagementCenters

(ManagementOversight)

Department(Mitigation)

Senior LeadershipAudit Committee for

Board of Trustees(Institutional Risk)

Inform

13

RCC Metrics:1. Very similar to UNC – GA

2. Focus on Financial Assessment Management (FAM)

3. Provide useful reporting to better understand risk at all levels

4. Utilize sufficiently detailed analysis to fully understand underlying root causes in specific units/schools/colleges• Example: 5 projects in Period 7 were responsible for 93% of the $ in untimely NSCT moved on to federal

awards• Example: 76% of all untimely transfers aggregated to 2 PIs• Example: trend analysis indicates 80% of all untimely transfers occurred in 6 departments over an annual

period

5. Provides insight into policy and process adjustments• Example: Adjusting metrics if necessary (other examples: adjusting definitions of cost transfers to exclude

parent/sub, etc.)

4. Provides a consistent reporting and assessment platform for all levels of the universityInform

Enforcing the Metric

Panda Powell, UNCW

UNCW – Campus Perspective

• Compilation of data for reporting•Systems used•RAMSeS•Banner

•Departments involved•Sponsored Programs•Controller’s office/billing

Enforce

• New fund establishment– Average number of days • Cause of “yellow” or “red”

– Risk of consistent “red” section

• Time & Effort Certifications– Timeliness of certifications– Banner T&E system • Training needs?• Any system issues?

Enforce

• Salary Transfer Requests– >90 days – why?– Transfers affect Time & Effort – Banner system– Risk mitigation

• A-133 Audit Certifications– Critical for Sub-recipient monitoring– Review of sub-award entities• Vendor versus true Sub-award

Enforce

• Accounts Receivable– Age of receivables– Reasons for slow or non-payment• Final reports not submitted• Submitted invoice lost or misplaced

– Note number/type of invoicing required– Sponsored Programs and Billing work together

Enforce

•Management Review•At a glance” areas of concern•GREEN – everything OK•Yellow – Which areas? What causing issue?•Red – closer review

•Why red? Is there reasonable explanation?

•Look for Trends

Enforce

Duke Department Use of Internal Controls

Todd LeovicImmunology Duke University

Enforce

Internal Control TypesOperational• Promotes operational effectiveness and efficiency as well as adherence to policies

and procedures.

Financial• Designed to safeguard assets and ensure completeness, accuracy and reliability of

financial records.

Compliance• Ensures compliance with applicable laws and regulations.

Enforce

Financial Assessment Management, FAM

-Establishes guidelines for Cost Object activity oversight and fiscal responsibility. Including assessment, verification and substantiation for account level transactions.-FAM considers the Internal Control structure of a business unit and the preventative controls gained with the implementation of new technologies and allows flexibility based on funding source and risk level. - FAM is a process to detect corrections necessary for the accuracy, integrity and reliability of reported financial reports.

Enforce

Internal Control Environment ( ICE )

Scope

The Internal Control Environment (ICE) Assessment document focuses on the financial analysis and oversight, risk assessment, internal controls, proper training and documentation for all aspects of the department’s revenue, expenditures, and balance sheet accounts. Each ICE covers all object codes owned or managed by a unit.

Enforce

Internal Control Environment ( ICE )

Roles and Responsibilities

The individual completing the ICE document must ensure compliance with Duke Policy, GAP, and other external reporting agencies. Individuals performing FAM reviews must have appropriate knowledge of the transactions, delegated responsibility for fund oversight and proper training. Management Centers are responsible for approving the ICE document.

Enforce

Internal Control Environment ( ICE )

ProceduresRisk Assessment:

Each transaction category documents the Institutional risk level. The Business Unit is expected to document the internal controls in their unit that differ from the institutional overview. The Business Unit risk assessment is used for the basis for transaction sampling.

Enforce

Internal Control Environment ( ICE )

ProceduresSampling:

Preventative controls enable responsible financial persons to use sampling quality assurance reviews. In general, higher risk will result in a larger sample size, whereas, lower risk may produce smaller sample sizes. Besides risk, key deciding factors are funding source, internal controls and historical performance.

Enforce

Internal Control Environment ( ICE )

Procedures

Transaction Review and Assessment:

A transaction review and assessment for selected sample transactions should consider reasonableness and accuracy of funding for the transaction and general ledger account. Sponsored funds should also assess the allowability and allocability of the goods, services and wages/effort charged to the fund.

Enforce

Internal Control Environment ( ICE )

Procedures

Analytical Assessment: Compares actual performance to budgeted expectations and /or to projected performance, in total , by cost and general ledger line item. Analysis should identify variances requiring explanation and discussion, including overdraft status. Types of assessment include: Supporting documentation, Noting errors and corrections, Documenting Completed assessment.

Enforce

Accounts Payable Check Request (APCR) including: Petty cash, check requests, Experimental Subject Payments and Short-Term Research Funds

Institutional Risk Level: Medium ( reliance on preventative and detective controls)

Institutional Internal Control Environment Overview:

Accounts Payable Check Requests (APCRs) receive at least one approval signature prior to processing for payment. Accounts Payable manually inputs payment information from the hard-copy form provided. Posting errors may occur for cost object, general ledger account and payment amount fields.

Recommended FAM Review Procedures: On a sample basis, compare transactions to supporting documentation. The FAM reviewer should perform analytical procedures to assess reasonableness.

Example of ICE transaction category

Accounts Payable Check Request (APCR) including: Petty cash, check requests, Experimental Subject Payments and Short-Term Research Funds

Institutional Risk Level: Medium ( reliance on preventative and detective controls)

Institutional Internal Control Environment Overview:

Accounts Payable Check Requests (APCRs) receive at least one approval signature prior to processing for payment. Accounts Payable manually inputs payment information from the hard-copy form provided. Posting errors may occur for cost object, general ledger account and payment amount fields.

Recommended FAM Review Procedures: On a sample basis, compare transactions to supporting documentation. The FAM reviewer should perform analytical procedures to assess reasonableness.PCR) including: Petty cash, check requests, Experimental Subject Payments and Short-Term Research Funds

Enforce

© 2007 Office Of Internal Audits

Yes No N/ARoles and Responsibilities Designated individuals prepare APCRs ☐ ☐ ☐

The Click here to select a position/role. designates individuals authorized to approve APCRs

If “Other”, specify person or title: Click here to enter text.

☐ ☐ ☐

All transaction initiators and approvers receive training appropriate for the assigned role ☐ ☐ ☐

Approval Threshold The business unit designates approval thresholds

Level 1: $Click here to enter amount.

Level 2: $ Click here to enter amount.

Other (explain below):$ Click here to enter amount.

Click here to enter text.

☐ ☐ ☐

The transaction approver verifies the transaction will be charged to the appropriate cost object and general ledger account

☐ ☐ ☐

(*) Segregation of Duties The initiator and the approver are two different people ☐ ☐ ☐

(*) The approver and the FAM reviewer are two different people ☐ ☐ ☐

Enforce

• Business Unit FAM Review Procedures:

© 2007 Office Of Internal Audits

Restricted Funds

Business Unit Risk Level

Sample Size

Sample Methodology

Sample Determination

☐ Low – Business unit has adequate preventive controls

15% FAM Sampling Tool

FAM Sampling Tool

☐ Medium – Business unit relies jointly on preventive and detective controls

30% FAM Sampling Tool

FAM Sampling Tool

☐ High – business unit relies primarily on detective controls

100% Not Applicable Not Applicable

Business Unit Internal Control Environment Overview:

Document the business unit internal control environment overview if different from the Institutional overview (i.e., additional internal controls or segregation of duty issues that may result in a business unit risk level that differs from the institutional risk level above).Business unit risk level is inherently the Institutional risk level of Medium unless the business unit Internal Control Environment Assessment above indicates additional internal controls that mitigate risks further or recommended internal controls are not in place in the business unit. Click here to enter text.

Enforce

Technical Advances-Role assignments and tracking -Procurement tool, (Buy@Duke)-Travel reimbursement-Effort Certification -Cost Transfer workflow and approval-Sampling tool-Re-budgeting workflow-Reporting capabilities© 2007 Office Of Internal Audits Enforce

Outcomes-Improved communication and targeted training-Cost savings and efficient ordering-Faster reimbursement-Improved tracking of effort and cost sharing-Less ticking and tying-Overall better internal controls

Enforce

A Lesson Learned-Adoption of promising new technology, Buy@Duke-Extensive training on use of system-Focus on front-end controls-Objective to reduce cost transfers, ensure allowability and improve order efficiency.-Road Block at Institutional level – General guidance needed.-Discovered that A/A/R assessment was being assigned to individuals who lacked training/understanding of AAR.-Grant managers not able to make allocability decisions as well as those closer to the science.-Uncovered a fundamental issue which needed to be addressed for improving decision making, (A/A/R) in a decentralized environment.-Enabled Duke to implement directed training, and enforce with internal controls and metrics

Enforce

Preparing the Grant Management Workforce

Julie Cole, Duke University

Preparing for Compliance

Monitorin

g

Training

Prepare

FAM

New RCC Training in Support of FAM

• Historic RCC Training Program Changed, based on several factors:– Functional analysis of individual’s pre and post-award duties– FAM/ICE identification of training needs– New technologies: Teaching the “Why” and not just the “How”– Reclassification of grant managers from multiple job descriptions and

partial grant responsibilities (1200 individuals) to consolidated groups, with REQUIRED TRAINING CERTIFICATION

– University LMS manages enrollment (Saba)– Content informed by RCC Monitoring team (Risk Assessment Metrics,

Department ICE feedback, etc.)

Prepare

Philosophy Change

– Moving from “the sage on the stage” to the “guide on the side”• Hands on, practical training and experiences

– Providing supervisors and practitioners with meaningful evidence of learning/knowledge/practical skills – Test results and Metrics

– Supported by Multi-Media resources

Prepare

Prepare

Prepare

Function and Metrics Driven: FAM

Core Competency Driven: HR

• Summary …..