Florida UCM As-Is Audit

7/27/2019 Florida UCM As-Is Audit

1/46

State of Florida

Agency for Workforce Innovation

Unemployment Compensation ProjectPhase 2

As-Is Business Process - Audit


2/46

Page i

Document Control

Revision History

Date Version Description Author

9/30/09 0.01 Initial draft Pete Peterson/Tad

McDonald

10/20/09 0.07 Incorporate QA feedback Pete Peterson/Tad

McDonald

10/23/2009 0.10 Incorporate Workshop feedback Pete Peterson/Tad

McDonald

11/4/2009 0.11 Incorporate Initial Deliverable Review Comments Pete Peterson/Tad

McDonald

11/5/2009 1.00 Final Version Pete Peterson/Tad

McDonald

Quality Review History

Date Reviewer Comments

10/19/2009 Lori Nolen Comments provided on printed copy

11/3/2009 Audit Staff Comments provided on printed copy

11/5/2009 Denise Birch Deliverable Approved


3/46

Page ii

Table of Contents

0 PURPOSE AND OBJECTIVES OF THE AUDIT CORE PROCESS ............................. 1

0.1 BEGINNING AND ENDING POINTS ..................................................................................................................... 20.2 AUDIT DIAGRAM .............................................................................................................................................. 30.3 AUDIT ACTIVITIES ........................................................................................................................................... 4

1 PURPOSE AND OBJECTIVES OF THE AUDITBUSINESS ACCURACYMEASUREMENT (BAM) SUB-PROCESS ............................................................................... 6

1.1 BEGINNING AND ENDING POINTS ..................................................................................................................... 61.2 AUDITBUSINESS ACCURACY MEASUREMENT (BAM)DIAGRAM ................................................................. 71.3 AUDIT -BAMACTIVITIES ............................................................................................................................. 10

1.4 AUDITBAMPROCESS ATTRIBUTES ............................................................................................................ 121.4.1 Elapsed Time ....................................................................................................................................... 121.4.2 Paper Documentation ........... ........... .......... ........... .......... ........... .......... .......... ........... .......... ........... ....... 121.4.3 Business System Data Entry & Data Transfer .......... .......... ........... .......... ........... .......... .......... ........... .. 151.4.4 Communication .................................................................................................................................... 161.4.5 Opportunities ....................................................................................................................................... 18

2 PURPOSE AND OBJECTIVES OF THE AUDIT - BTQ QUARTERLY REPORTING

SUB-PROCESS ........................................................................................................................... 20

2.1 BEGINNING AND ENDING POINTS ................................................................................................................... 202.2 AUDIT -BENEFIT TIMELINESS QUALITY (BTQ)QUARTERLY REPORTING DIAGRAM .................................. 212.3 AUDITBTQQUARTERLY REPORTING ACTIVITIES ................................................................................... 23

2.4 AUDITBTQQUARTERLY REPORTING PROCESS ATTRIBUTES .................................................................. 242.4.1 Elapsed Time ....................................................................................................................................... 242.4.2 Paper Documentation ........... ........... .......... ........... .......... ........... .......... .......... ........... .......... ........... ....... 252.4.3 Business System Data Entry & Data Transfer .......... .......... ........... .......... ........... .......... .......... ........... .. 262.4.4 Communication .................................................................................................................................... 272.4.5 Opportunities ....................................................................................................................................... 27

3 PURPOSE AND OBJECTIVES OF THE AUDITBTQ NON-MONETARY

REVIEWS SUB-PROCESS ....................................................................................................... 29

3.1 BEGINNING AND ENDING POINTS ................................................................................................................... 293.2 AUDITBTQNON-MONETARY REVIEWS DIAGRAM .................................................................................. 303.3 AUDITBTQNON-MONETARY REVIEWS ACTIVITIES ............................................................................... 31

3.4 AUDITBTQNON-MONETARY REVIEWS PROCESS ATTRIBUTES .............................................................. 323.4.1 Elapsed Time ....................................................................................................................................... 323.4.2 Paper Documentation ........... ........... .......... ........... .......... ........... .......... .......... ........... .......... ........... ....... 323.4.3 Business System Data Entry & Data Transfer .......... .......... ........... .......... ........... .......... .......... ........... .. 333.4.4 Communication .................................................................................................................................... 343.4.5 Opportunities ....................................................................................................................................... 35

4 PURPOSE AND OBJECTIVES OF THE AUDITBTQDATA VALIDATION

SUB-PROCESS ........................................................................................................................... 36


4/46

Page iii

4.1 BEGINNING AND ENDING POINTS ................................................................................................................... 364.2 AUDITBTQDATA VALIDATION DIAGRAM .............................................................................................. 374.3 AUDITBTQDATA VALIDATION ACTIVITIES ............................................................................................ 384.4 AUDITBTQDATA VALIDATION PROCESS ATTRIBUTES ........................................................................... 39

4.4.1 Elapsed Time ....................................................................................................................................... 394.4.2 Paper Documentation ........... ........... .......... ........... .......... ........... .......... .......... ........... .......... ........... ....... 404.4.3 Business System Data Entry & Data Transfer .......... .......... ........... .......... ........... .......... .......... ........... .. 404.4.4 Communication .................................................................................................................................... 414.4.5 Opportunities ....................................................................................................................................... 41


5/46

Page 1

0 PURPOSE AND OBJECTIVES OF THE AUDIT CORE

PROCESS

The Audit core process is comprised of four distinct sub-processes: Benefit Accuracy Measurement

(BAM), Benefit Timeliness and Quality (BTQ)Quarterly Reporting, BTQNon-Monetary Reviews,and BTQData Validation.

Benefit Accuracy Measurement (BAM)

BAM is a federally mandated program providing the basis for assessing the accuracy of Unemployment

Insurance payments made to claimants and the accuracy of benefits denied to claimants. BAM is also adiagnostic tool for Federal and State workforce agency staff members to assist in error identification andcausal analysis. BAM supports the Agencys desire to implement corrective actions and to track theeffectiveness of those remedial actions.

The objectives of the Benefit Accuracy Measurement program are to:

Assess the accuracy of Unemployment Insurance payments

Assess improvements in program accuracy and integrity

Encourage more efficient administration of the Unemployment Insurance program

A weekly, representative sample of UI payments and disqualifying ineligibility determinations are drawn(from the U.S. Department of Labor UI State Menu system), investigated, and audited intensively todetermine whether they were properly administered to claimants and whether the claimants were properly

paid or appropriately denied. Up to ten cases each for paid and denial determinations are randomlyselected for audit. The normal annual caseload is 480 paid claims and 450 denied determinations. Thespecific number of required audits may be changed by the U.S. Department of Labor depending upon

prevailing conditions (i.e., during periods of high unemployment, the Federal government may reduce the

number of audits to allow an agency to re-assign human resources to other tasks such as claimadjudication).

Benefit Timeliness Quality (BTQ)Quarterly Reporting

The determination of a claimants eligibility for unemployment insurance (UI) benefits is a critical UIprogram function. When issues arise that may affect a claimants past, present, or future benefits, the

adjudicator is responsible for determining the claimants eligibility for those benefits. Suchdeterminations may also affect an employers liability for benefit charges, depending on the type of issuesadjudicated. The adjudicators work impacts the rights of both claimants and employers. The BTQ unit

is responsible to report quarterly to the U.S. Department of Labor (DOL) for nonmonetary claims.

Benefit Timeliness Quality (BTQ)Non-Monetary Reviews

UC has instituted an internal quality review process under the auspices of the BTQ unit to performperiodic assessments of adjudicators in each Adjudication Hub. The information gathered through the

reviews provides insight to existing or potential issues that may be causing difficulties in particularadjudication areas (e.g., separation issues). Managers can use that information to take corrective actionwhich may include arranging for remedial training, modifying policies and procedures, and/or suggestingchanges to the automated system(s) to address less than satisfactory performance.

Benefit Timeliness Quality (BTQ)Data Validation

The comprehensive Federal data validation program is necessary to ensure the accuracy of

Unemployment Insurance Required Reports (UIRR) data. UIRR data are used for economic statistics, toallocate UI administrative funding based on state workload, to measure state claimant eligibility criteria


6/46

Page 2

and performance in providing benefits, and to account for fund utilization. It is therefore essential thatstates report UIRR data accurately and uniformly. States report to DOL on a monthly and quarterly basis

under the Unemployment Insurance Required Reports system.

States are required to validate reported data every third year, except for data elements used to calculate

Government Performance and Results Act (GPRA) measures. GPRA data are validated annually. The

validation year will coincide with the State Quality Service Plan (SQSP) performance year. Validationof any reporting period during the twelve months beginning April 1 and ending March 31 will be

considered part of the validation year. The SQSP is the vehicle through which states submit plans toimplement validation or to revalidate failed items. GPRA and SQSP reporting is covered in a separate

process, Federal Reporting.

Note: Many units within AWI perform audit functions to monitor their performance. The processesdescribed here address specifically BAM and BTQ and are not meant to encompass other audit functionswithin AWI.

The following sections describe the activities associated with the Audit functions in UC.

0.1

BEGINNING AND ENDING POINTS

BAM

Beginning Point:

A list of completed claims is downloaded from the U.S. Department of Labor (DOL) State Menu

system.

Ending Point:

The BAM investigators case file is sent to imaging or a storage facility.

BTQQuarterly Reporting

Beginning Point:

The UC Benefits system generates a random list of claims for review on the first workday of each

quarter.Ending Point:

The BTQ investigators case file is copied to a local, shared network drive.

BTQNon-Monetary Reviews

Beginning Point:

An Adjudication Hub is selected for review.

Ending Point:

The results of the claims reviews are recorded in an internal reporting document.

BTQData Validations

Beginning Point:

A list of necessary populations to be validated is received from the U.S. Department of Labor(DOL) State Menu system.

Ending Point:

The results of the validations are submitted via the U.S. Department of Labor (DOL) State Menusystem.


7/46

Page 3

0.2 AUDIT DIAGRAM

BAM Unit

Benefit Accuracy Measurement

BTQ Unit

Benefit Timeliness & Quality

Process Start

+

11.01

BAM Weekly

Reviews

Process Ends

Process Start Process Start Process Start

(Data Validation)

+

11.02

Quarterly

Reports

Process Ends

+

11.03

Non-Monetary

Reviews

Process Ends

+

11.04

Data Validation

Process Ends

Exhibit 1Audit Process Flow


8/46

Page 4

0.3 AUDIT ACTIVITIES

Exhibit 2Audit Activities lists the activities that make up the Audit Core Process. The following is a

description and definition of each of the columns in this table.

Index A number for use in referencing the activity

Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram.Activity Description The full-text description of that activity

Actors The job or role title of the individuals having a primary role for that activity

Index Activity Label Activity Description Actors

11.01 BAM Weekly

Reviews

Each week (Monday mornings), a supervisor in the BAM unit logs

on to the State Menu system (U.S. DOL application) and

downloads cases for investigation/audit.

Investigators within the BAM Unit audit their assigned cases for

both Paid and Denied claims (they have a maximum of 120 days to

complete their investigations). The investigator enters the results in

the U.S. DOL State Menu system.

Once an investigation/audit is complete, the results are returned to

the BAM unit supervisor for review. The supervisor reviews the

entire case file and the entries in the State Menu system, approves

(or coordinates corrective action with investigators), and closes the

case(s).

Benefit Accuracy

Measurement Unit

11.02 Quarterly Reports A U.S. DOL algorithm is coded within the UC Benefits system to

randomly select, on the first day of each quarter, non-monetary

cases (50 separation, 50 non-separation) for BTQ reporting. The

audit reports are due by the 20 thof the following month.

(Note: The Quarterly Report has not been run for quite a while due

to the re-assignment of BTQ personnel to support the increased UCworkload.)

Reviewers in the BTQ unit examine the cases assigned to them by a

BTQ supervisor, score each according to established guidelines,

and return the results to the unit supervisor for additional

processing.

The results of the BTQ audits are recorded in the U.S. DOL State

Menu system.

Benefit Timeliness &

Quality Unit

11.03 Non-Monetary

Reviews

The BTQ unit conducts periodic audits of the Adjudication Hubs.

A selection of claims (10 cases per adjudicator) is randomly

extracted from the UC Benefits system for a given date range and

assigned to BTQ examiners for review.

Examiners review cases assigned to them and score each according

to established guidelines.

A BTQ unit supervisor compiles the results of the reviews and

communicates the results to the Adjudication Hub supervisors for

further action.


Quality Unit


9/46

Page 5


11.04 Data Validation A BTQ supervisor accesses the U.S. DOL State Menu system and

downloads validation data requirements established by the U.S.

Department of Labor. Validation and reporting requirements are

divided into populations by DOL and sent to all states in June or

July. The agencys previous performance (Pass/Fail criteria)

influences the validation requirements for content (which

populations to validate) and frequency (usually, a passing score

relieves the agency of reporting for a given population for three

years while a failing score necessitates corrective action, re-

validation, and reporting on a more frequent basis).

The results of the BTQ audits are recorded in the U.S. DOL State

Menu system.


Quality Unit

Exhibit 2Audit Activities


10/46

Page 6

1 PURPOSE AND OBJECTIVES OF THE AUDITBUSINESS

ACCURACY MEASUREMENT (BAM) SUB-PROCESS


(BAM), Benefit Timeliness and Quality (BTQ) - Quarterly Reporting, BTQNon-Monetary Reviews,and BTQData Validation.

Benefit Accuracy Measurement (BAM)

BAM is a federally mandated program providing the basis for assessing the accuracy of Unemployment

Insurance payments made to claimants and the accuracy of benefits denied to claimants. BAM is also adiagnostic tool for Federal and State workforce agency staff members to assist in error identification andcausal analysis. BAM supports the Agencys desire to implement corrective actions and to track theeffectiveness of those remedial actions.

The objectives of the Benefit Accuracy Measurement program are to:

Assess the accuracy of Unemployment Insurance payments

Assess improvements in program accuracy and integrity

Encourage more efficient administration of the Unemployment Insurance program

A weekly, representative sample of UI payments and disqualifying ineligibility determinations are drawn(from the U.S. Department of Labor UI State Menu system), investigated, and audited intensively todetermine whether they were properly administered to claimants and whether the claimants were properly

paid or appropriately denied. Up to ten cases each for paid and denial determinations are randomlyselected for audit. The normal annual caseload is 480 paid claims and 450 denied determinations. Thespecific number of required audits may be changed by the U.S. Department of Labor depending upon

prevailing conditions (i.e., during periods of high unemployment, the Federal government may reduce the

number of audits to allow an agency to re-assign human resources to other tasks such as claimadjudication).

Note: Many units within AWI perform audit functions to monitor their performance. The processes

described here address specifically BAM and are not meant to encompass other audit functions withinAWI.

The following sections describe the activities associated with the Audit - BAM functions in UC.

1.1


BAM

Beginning Point:

A list of completed claims is downloaded from the U.S. Department of Labor (DOL) State Menu

system.

Ending Point:

The BAM investigators case file is sent to imaging or a storage facility.


11/46

Page 7

1.2 AUDITBUSINESS ACCURACY MEASUREMENT (BAM)DIAGRAM

Supervisor

11.01.01Download Cases

InvestigatorUS Department of

Labor

State Menu System

11.01.02 Assign

Investigator

State Menu System

11.01.04 Generate

Code Sheets

Winscape

11.01.06 Update

Investigator OpenCase Document

Word

11.01.07 Save

Open Case

Document to

Investigator folder

Local S: Drive

11.01.05 Move

Case files to

Investigator folder

Local S: Drive

11.01.08 Request

Weekly

Certifications

UC

11.01.03 Run

Script

UI Sun (Putty)

Print Job Request

Process Start

Continued on

Following Page


12/46

Page 8

EmployerInvestigatorSupervisor

11.01.09 Retrieve

Open Case

Assignments

Local S: Drive

11.01.11 Retrieve

Pre-Investigation

Documents

(Claimant) UC, Imaging,

SunTax,

Employ Florida

11.01.13

Assemble

Investigation Case

Folder

11.01.15

Investigate Case

Word

11.01.10 Print

Code Sheets

11.01.14 Send

Employer Wage

Verification Form

& Claimant

Questionnaire

Claimant

Mail/Email/Fax

Wage Verification Form

Mail/Fax/Email

Questionnaire

Phone/Mail /Fax/Emai l Phone/Mai l/Fax/Emai l

Other AWI Units

If Determination Error

Request

Redetermination

If Possible Fraud

Notify

Benefit Payment Control

Notify

Appeals Clerks Office

If Under Appeal11.01.16 Inform

Other Areas

BD15V Weekly Certifications

11.01.12 Retrieve

Pre-InvestigationDocuments

(Employer)

SunTax

State Menu System

Continued from

Previous Page

Continued on

Following Page


13/46

Page 9

Supervisor Investigator

11.01.21 Review

Case

11.01.22 Make

Corrections

11.01.17 Complete

Code Sheet

11.01.18 Update

Code Sheet Data

11.01.20 Return

Case Folder

State Menu System

11.01.19 Close

Case

State Menu System

11.01.23 Close

Case

State Menu System

11.01.25 Store File

(3 Years)

If Paid Case

11.01.26 Send File

to Imaging

If Denied Case

Process Ends

Incorrect

Continued from

Previous Page

Investigator

If Improper Payment11.01.24 Review

Case

Interoffice

Memo

Interoffice

Memo

Exhibit 3Audit - BAM Process Flow


14/46

Page 10

1.3 AUDIT -BAMACTIVITIES

Exhibit 4Audit - BAM Activities lists the activities that make up the AuditBAM sub-process. The

following is a description and definition of each of the columns in this table.

Index A number for use in referencing the activity

Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram.Activity Description The full-text description of that activity



11.01.01 Download Cases The U.S. Department of Labor (DOL) provides a random selection

of Paid (PCA) and Denied (DCA) claims for state audit. Each

week (Monday mornings), a supervisor in the BAM unit logs on to

the State Menu system (U.S. DOL application) and downloads

cases for investigation/audit. (Note: the cases are automatically

converted.)

US Department of

Labor (DOL)

Supervisor

11.01.02 Assign Investigator The supervisor selects Case Assignment from the State Menu

selection screen and assigns each case to a specific investigator

(investigators are identified with a unique number). The process is

completed for Paid (PCA) cases and repeated for Denied (DCA)

cases.

Supervisor

11.01.03 Run Script The supervisor logs onto the uisfl2 (UI Sun) system and runs a

preloaded script to prepare the coding sheets to be used for the

case reviews.

Supervisor

11.01.04 Generate Code

Sheets

Each investigation has its own code sheet to record the results of

the examination. The BAM supervisor generates the code sheets

using the Winscape application.

Supervisor

11.01.05 Move Case files to

Investigator folder

Each investigator is assigned a folder on the network S: drive. The

coding sheets for the cases assigned to each investigator are placed

in their S: drive folder.

Supervisor

11.01.06 Update InvestigatorOpen Case

Document

The supervisor maintains a Word document containing the caseassignments for each investigator.

Supervisor

11.01.07 Save Open Case

Document to

Investigator Folder

The Word document created by the Supervisor is placed in each

investigators S: drive folder.

Supervisor

11.01.08 Request Weekly

Certifications

For Paid (PCA) cases, the Supervisor runs a print request to create

Weekly Certifications from the UC Benefits system.

Supervisor

11.01.09 Retrieve Open Case

Assignments

The investigator accesses the network S: drive folder to locate the

cases assigned (Word document placed there by the Supervisor).

Investigator

11.01.10 Print Code Sheets The code sheets are printed from the State Menu system. Investigator

11.01.11 Retrieve Pre-

Investigation

Documents

(Claimant)

Using a variety of systems at the investigators disposal (UC,

Imaging, SunTax, Employ Florida), the investigator gathers

information to support the case examination. In addition, the

printed output from the BD15V transaction is returned to the

investigator (the print job runs overnight).

Investigator


Investigation

Documents

(Employer)

The investigator retrieves employer-related information from the

SunTax system.

Investigator


15/46

Page 11


11.01.13 Assemble

Investigation Case

Folder

The investigator follows a common case folder assembly process

to standardize the content and order of documents in the case

investigation folder (one folder is prepared for each case).

Investigator

11.01.14 Send Employer

Wage Verification

Form & Claimant

Questionnaire

The investigator sends a questionnaire to claimants, asking them to

supply information about their particular claim. At the same time,

employers are asked to verify the information about wages using

the Wage Verification form.

(Note: Claimants are informed the failure of responding to the

questionnaire may result in the suspension of benefit payments (if

any are being currently paid). If the Claimant does not respond,

further payments, if any, are suspended. Once a response is

provided, payments, if any, are resumed. No such leverage is

available regarding employers.

Investigators

11.01.15 Investigate Case Investigators review all the information at their disposal to

investigate the cases assigned to them. They may have to

communicate with claimants and employers on an ad-hoc basis to

clarify existing information or to gather additional information.

Investigator

11.01.16 Inform Other Areas If, during the course of an investigation, it is found that a

determination error may have been made, investigators may

request redetermination from the Wage Determination unit. If the

case being investigated is under appeal, the Clerks Office in

Appeals is notified. If the possibility of fraud exists, the Benefit

Payment Control (BPC) unit is informed. The results of the

redetermination and/or fraud investigation are returned to the

Investigator.

Investigator

11.01.17 Complete Code

Sheet

For each assigned case, the investigator completes a separate code

sheet that represents the results of the investigation.

Investigator

11.01.18 Update Code Sheet

Data

The investigator transfers the information from the paper code

sheet to the State Menu system.

Investigator

11.01.19 Close Case After the code sheet data is input to the State Menu system, the

investigator closes the case and prepares the case folder to be

given back to the BAM supervisor.

Investigator

11.01.20 Return Case Folder After review for completeness and accuracy, the investigator

returns the case folder(s) to the BAM supervisor.

Investigator

11.01.21 Review Case The Supervisor reviews the case information and, in an iterative

process, directs the investigator to make necessary corrections to

the case.

Supervisor

11.01.22 Make Corrections When notified by the BAM supervisor, the investigator makes

modifications to the case.

Investigator

11.01.23 Close Case Once the case investigation is complete to the satisfaction of the

BAM supervisor, the case is closed in the State Menu system. If an

Improper Payment has been made, the Supervisor will prepare amemorandum for the Administrators review.

Supervisor

11.01.24 Review Case The Administrator will review the case. If the improper payment

was made in violation of law or policy, a memorandum will be

prepared summarizing the results of the investigation and sent to

the Adjudication Hub Supervisor and/or AWI Management for

appropriate action.

Administrator

11.01.25 Store File (3 Years) Completed cases are archived. Paid cases are retained for three

years in hard copy.

Supervisor


16/46

Page 12


11.01.26 Send File to

Imaging

Denied payment cases are sent to imaging for capture and

retention.

Supervisor

Exhibit 4Audit - BAM Activities

1.4

AUDIT

BAMPROCESS ATTRIBUTES

The following sections define and describe sub-process specifics for elapsed time, paper documentation,business system use, communications, and process improvements. The information is arranged in thisway to support analysis of these activities, both individually and collectively.

1.4.1

Elapsed Time

Exhibit 5Audit - BAM Activity Elapsed Time lists the time required to complete each of the activities

that make up the AuditBAM sub-process. The times are estimated except where noted. The followingis a description and definition of each of the columns in this table.

Index A number for use in referencing the activity.

Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram.Process Time The hands-on time it takes to complete the activity (does not include mail, transport, or time

spent waiting for another activity to complete).

Cycle Time The total elapsed time from the beginning of the activity until it is complete, inclusive of all delays

from transport, mail, other processes, and work load.

Time Goals / Limits Any time limits or goals, from any sourceinternal or externalpertaining to this activity.

Limit/ Goal Source The source of the limit or goal listed above (Florida Statutes, AWI policy, etc.)

Speed Bumps Specific, recurring difficulties that slow the progress of this activity.

Index Activity

Label

Process

Time

Cycle

Time

Limit or Goal Time Limit or

Goal

Source

Speed Bumps

11.01 BAM 4-6 hours

per file

120 Days All investigations must be

completed within 120 days.

ETA 395

Handbook

published by

Department

of Labor.

Gathering

documentation from

multiple systems.

Obtaining responses

from Claimant &

Employer.


Investigation

Documents

Overnight Overnight Official reproductions of

certain documents must be

requested and printed during

an overnight process.

AWI Staff The overnight print

process, while

generally

cumbersome, does

not create a

significant delay in

the BAM process.

Exhibit 5

Audit - BAM Activity Elapsed Time

1.4.2 Paper Documentation

Exhibit 6Audit - BAM Paper Documentation lists the paper documents required to complete each of

the activities that make up the AuditBAM sub-process. The reference column refers to the originatorand title of policy, procedure, or instruction that contains the form or specifies the content of thedocument. More than one document may be required to carry out a single activity, so each is listed. The

following is a description and definition of each of the columns in this table.


17/46

Page 13


Activity Label A short name for that activity that is used to represent it in the detailed workflow diagram.

Document Title Short title, form number or other designation for the document.

Document

Description

General description of the document contents and purpose.

Reference or Source Originator and title of the source instruction or policy.

Index Activity Label Document Title Document Description Reference

or Source

11.01.06 Update

Investigator Open

Case Document

Investigator Open

Case Document

Word Document used to track all open cases for

each Investigator

BAM

Supervisor


Certifications

BD15V Weekly Claim Certification Report (for Key Week).

Printed from the UC System.

AWI

Investigator

11.01.10 Print Code Sheets Code Sheet Printout from the State Menu System. Completed by

hand. (4 VersionsPaid Claims, Separation Denial,

Monetary Denial, Non-Separation Denial)

AWI

Investigator


InvestigationDocuments

(Claimant)

BI08 UC/SunTax Wage Crossmatch. Printed from the UC

System.

AWI

Investigator


Investigation

Documents

(Claimant)

BD11 Wage Transcript & Determination. Printed from the

UC System.

AWI

Investigator


Investigation

Documents

(Claimant)

BH01 Claim History. Printed from the UC System. AWI

Investigator


Investigation

Documents

(Claimant)

BF01 Overpayment History. Printed from the UC System. AWI

Investigator


Investigation

Documents

(Claimant)

BQ01 Determination History. Printed from the UC

System.

AWI

Investigator


Investigation

Documents

(Claimant)

Claim Form Printed from the EIS (Imaging System) AWI

Investigator


Investigation

Documents(Claimant)

Wage Items Screen

Print

Printed from the Department of Revenue SunTax

System.

AWI

Investigator


Investigation

Documents

(Claimant)

SIDI Out of State Wages Summary Report. Printed from

the UC System.

AWI

Investigator


18/46

Page 14


or Source


Investigation

Documents

(Claimant)

IBIQ Out of State Wages Detail Report. Printed from the

UC System.

AWI

Investigator


Investigation

Documents

(Claimant)

Job Registration

History

Printed from the EmployFlorida System. AWI

Investigator


Investigation

Documents

(Employer)

Employer Report Retrieved from the Department of Revenue SunTax

System. Screen print of the Detail 1 and Detail 2

tabs.

AWI

Investigator

11.01.13 Assemble

Investigation Case

Folder

Investigation Activity

Log

Blank template (Word). Completed by hand. AWI

Investigator

11.01.13 Assemble

Investigation CaseFolder

Case Checklist Blank template (Word). Completed by hand. AWI

Investigator

11.01.14 Send Wage

Verification Form

& Claimant

Questionnaires

Claimant

Questionnaire

Document used to gather information from the

Claimant to be used during the Audit Investigation

(4 Versions - Paid Claims, Separation Denial,

Monetary Denial, Non-Separation Denial). Attached

to a Cover Letter generated from a Word Template.

AWI

Investigator

11.01.14 Send Wage

Verification Form

& Claimant

Questionnaires

Information for the

Claimant on the

Unemployment

Compensation Benefit

Accuracy

Measurement

Program (LES UCR

Bulletin 1 (Rev. 6/00)

Pamphlet inserted with the Claimant Questionnaire

containing information regarding the BAM

program.

AWI

Investigator

11.01.14 Send Wage

Verification Form

& Claimant

Questionnaires

Wage Verification

Form

Document used to gather information from the

Employer to be used during the Audit Investigation.

Consists of two parts - Claimant Wage Verification,

Quarterly Earnings Amounts. Attached to a Cover

Letter generated from a Word Template.

AWI

Investigator

11.01.15 Investigate Case School Verification

Form

Document sent to the School that Claimant was

attending during key week. Attached to a Cover


AWI

Investigator

11.01.15 Investigate Case (Private) Employment

Agency Verification

Document sent to a private employment agency to

validate Claimant's job search history. Attached to a

Cover Letter generated from a Word Template.

AWI

Investigator

11.01.15 Investigate Case Work Search

VerificationRegular

Employer

Document sent to a potential employer to validate

Claimant's job search history. Attached to a Cover


AWI

Investigator

11.01.15 Investigate Case Work Search

Verification - Labor

Organization

Document sent to a union or labor organization to

validate Claimant's job search history. Attached to a

Cover Letter generated from a Word Template.

AWI

Investigator


19/46

Page 15


or Source

11.01.16 Inform Other

Areas

MemoRequest for

Monetary

Redetermination

Word Template. Completed to request a

redetermination based on the results of the

investigation. (Note: The results of the

redetermination are returned to the BAM

Investigator)

AWI

Investigator

11.01.17 Complete Code

Sheet

Summary of

Investigation

Document used to present summary investigation

information to BAM Supervisor. Investigation (4

Versions - Paid Claims, Separation Denial,

Monetary Denial, Non-Monetary Non-Separation

Denial).

AWI

Investigator

11.01.18 Update Code Sheet

Data

Data Collection

Instrument (DCI)

Report

Printout from the State Menu System after data

entry is completed. (4 VersionsPaid Claims,

Separation Denial, Monetary Denial, Non-

Separation Denial)

AWI

Investigator

Exhibit 6Audit - BAM Paper Documentation

1.4.3

Business System Data Entry & Data TransferExhibit 7Audit - BAM Data Entry & Data Transfer lists the business systems that are employeddirectly (touched) or indirectly (data transfer) in the course of the AuditBAM sub-process. Thefollowing is a description and definition of each of the columns in this table.



Business System The name or designation of the system touched (if any).

Purpose or Intent The business objective accomplished by using the system or systems.

Mode Choose one; entry is made by a person, transfers are from system to system.

Index Activity Label Business System Program or Intent Mode

11.01.03 Run Script UI Sun System (Putty) Command Line Program - Convertsdownloaded cases to a format to be used for

Winscape

Transfer

11.01.04 Generate Code

Sheets

Winscape Command Line Program - Creates Code Sheets

for all selected cases for review. Output is a

series of Word documents

Transfer


Certifications

UC BD15V - By SSN/Date

Weekly Claim Certification Report (for Key

Work)

Inquiry


Investigation

Documents

(Claimant)

UC Retrieve by SSN:

(BI08) UC/SunTax Wage Crossmatch

(BD11) Wage Transcript & Determination

(BH01) Claim History

(BF01) Overpayment History

(BQ01) Determination History

Inquiry


Investigation

Documents

(Claimant)

EIS (Imaging) Input SSN on the Search Screen. Results are

displayed and the Original Claim Form is

printed.

Inquiry


20/46

Page 16



Investigation

Documents

(Claimant)

SunTax Input SSN on the Search Screen.

Select and print the Wage Items Screen (By

Employer for Base Period & Following)

Inquiry


Investigation

Documents

(Claimant)

UC Retrieve by SSN

(SIDI) Out of State Wages (Search)

If results are found, input SSN and State to

retrieve:

(IBIQ) Out of State Wages (Detail)

Inquiry


Investigation

Documents

(Claimant)

EmployFlorida Input SSN on the Assist Screen. Print the Job

Registration History

Inquiry


Investigation

Documents

(Employer)

SunTax Input the Employer # on the Search Screen.

Print the:

Detail 1 Tab (Basic Demographics)

Detail 2 Tab (Tax Rate/NAICS Code)

Inquiry

Exhibit 7

Audit - BAM Data Entry & Data Transfer

1.4.4 Communication

Exhibit 8Audit - BAM lists the messages, such as the US mail, emails, phone calls and any othercorrespondence, regularly used in the normal courseof the AuditBAM sub-process. Messages are sentto convey required documentation, inform recipients about the status of an activity, to trigger the start orend of other activities, to grant or withdraw authorization, etc. This section only concerns messages sentduring normal (as intended) operations. There may be more than one message associated with any

activity in this process. The following is a description and definition of each of the columns in this table.



Message Type US mail, email, phone call, etc.Message Sender The organization or role of the message sender.

Message Purpose The intended purpose for sending the message.

Message Recipient The organization or role of the intended recipient.

Occasion Event or action that triggers the requirement to send the message

Index Activity Label Message

Type

Sender Message

Purpose

Message

Recipient

Occasion

11.01.14 Send Wage

Verification

Form & Claimant

Questionnaires

Mail Investigator Claimant

Questionnaire

Claimant During the

Investigation into a

determination.

11.01.14 Send Wage

Verification

Form & Claimant

Questionnaires

Mail/Fax Investigator Wage

Verification

Form

Employer During the


determination.

11.01.15 Investigate Case Phone/Mail/Fa

x/Email

Investigator To gather

additional

information as

necessary.

Claimant/Emplo

yer

During the


determination.


21/46

Page 17


Type

Sender Message

Purpose

Message

Recipient

Occasion

11.01.15 Investigate Case Mail/Fax Investigator To gather

additional

information as

necessary.

School During the


determination.


additional

information as

necessary.

Private Job

Search Firm

During the


determination.


additional

information as

necessary.

Potential

Employer

During the


determination.


additional

information as

necessary.

Labor

Organization

During the


determination.

11.01.16 Inform OtherAreas (as

necessary)

Email/Phone/Interoffice

Investigator RequestRedetermination

(Note: The

results of the

redetermination

are returned to

the BAM

Investigator)

AdjudicationHub

If the file appears tocontain a

determination error.


Areas (as

necessary)

Email/Phone/

Interoffice

Investigator Notify Appeals

of a

Redetermination

Appeals Unit If the file appears to

contain a

determination error.


Areas (as

necessary)

Email/Phone/

Interoffice

Investigator Notify BPC of

possible fraud

for further

investigation.

(Note: The

results of the

investigation are

returned to the

BAM

Investigator)

BPC Unit If the file appears to

evidence of

fraudulent activity.

11.01.23 Close Case Interoffice Supervisor Notify the

Administrator of

a possible

improper

payment made

in violation of

law or policy.

Administrator When a possible

improper payment is

detected.


22/46

Page 18


Type

Sender Message

Purpose

Message

Recipient

Occasion

11.01.24 Review Case Interoffice Administrator Notify the

Adjudication

Hub Supervisor

and/or AWI

Management of

the improper

payment for

appropriate

corrective

action.

Adjudication

Hub Supervisor/

AWI

Management

When an improper

payment is detected

in violation of law or

policy.

Exhibit 8Audit - BAM Communication

1.4.5 Opportunities

The purpose of this section is to capture improvement suggestions that may have a positive effect ontechnical, functional, or physical components of the Audit - BAM activities.

Over the course of our interviews and observations, we identified a number of possible processimprovement opportunities (long and short term). The following tables contain descriptions (ProblemStatement) of what may be negatively affecting current operations and corresponding potential solutions(Recommendation) to address the issues. In some cases, additional analysis is required tounderstand

fully the impact of the problems and/or recommendations.

The following is a description and definition of each of the columns in this table.

Origin The area of the improvement such as: Central Office, Hub, etc.

Problem

Statement

Description of the problem or opportunity identified.

Recommendation Description of the proposed recommendation, including expected result.

Value Added Description of the expected value received if the recommendation to the problem statement is

identified.

Short Term Opportunities for Improvement

Origin Problem Statement Recommendation Value Added

11.01.11

11.01.12

There are a large number of reports that

are gathered as part of the BAM

process. For the most part, each report

must be requested individually.

Provide a single transaction

for investigators to

automatically generate the UC

reports required to support

audit activities.

Significant time savings.

11.01.14 There appears to be a problem getting

employers to respond to inquiries in a

timely fashion which results in delays

for the investigation process.

Identifying better/easier

methods (e.g., online) for

employer feedback may help

to simplify the process for

employers.

Needs further investigation.

Quicker employer responses will

decrease the time necessary to

complete audit activities.


23/46

Page 19

11.01.25 Paid case investigation results are not

imaged like the Denied cases.

Paid cases are a lot larger (many more

pages of documents because Paid cases

involve claimants and employers;

denied cases usually only involve

claimants) and there arent time or

resources available to prepare the case

folders for imaging.

We believe further

investigation is required to

develop workable solutions

leading to the capture of Paid

cases in the imaging system.

Capturing images of the Paid cases

would lead to relieve the need to

maintain paper archives and could

provide a faster, easier method for

retrieval.

Exhibit 9Short Term Opportunities for Improvement

Long Term Opportunities for Improvement


Exhibit 10Long Term Opportunities for Improvement


24/46

Page 20

2 PURPOSE AND OBJECTIVES OF THE AUDIT - BTQ

QUARTERLY REPORTING SUB-PROCESS


(BAM), Benefit Timeliness and Quality (BTQ)Quarterly Reporting, BTQNon-Monetary Reviews,and BTQData Validation. The following section describes the BTQQuarterly Reporting related

activities.

Benefit Timeliness Quality (BTQ)Quarterly Reporting

The determination of a claimants eligibility for unemployment insurance (UI) benefits is a critical UIprogram function. When issues arise that may affect a claimants past, present, or future benefits, theadjudicator is responsible for determining the claimants eligibility for those benefits. Suchdeterminations may also affect an employers liability for benefit charges, depending on the type of issuesadjudicated. The adjudicators work impacts the rights of both claimants and employers. The BTQ unit

is responsible to report quarterly to the U.S. Department of Labor (DOL) for nonmonetary claims.

Note: Many units within AWI perform audit functions to monitor their performance. The processes

described here address specifically BAM and BTQ and are not meant to encompass other audit functionswithin AWI.

The following sections describe the activities associated with the Audit - BTQ functions in UC.

2.1


BTQQuarterly Reporting

Beginning Point:

The UC Benefits system generates a random list of claims for review on the first workday of each

quarter.

Ending Point:The BTQ investigators case file is copied to a local, shared network drive.


25/46

Page 21

2.2 AUDIT -BENEFIT TIMELINESS QUALITY (BTQ)QUARTERLY REPORTING

DIAGRAM

Supervisor Reviewer AWI Print Shop

11.02.05 InsertData

Collection

Instrument

If First Review

11.02.06 Review &

Score File

11.02.01 Select

SSNs for Review

11.02.02 Assign

Reviewer

11.02.03 Setup

Folder & Label

Word

Interoffice

11.02.04 Retrieve

Pre-Review

Documents

UC, Imaging,

FLUID, AENR

If Second Review

UC

Adjudication Hub

If Determination Error

Request

Redetermination

Process Start

Continued on

Following Page


26/46

Page 22

SupervisorReviewerAdjudication Hub

11.02.11 Complete

Internal Report

11.02.09 Complete

ETA 9056

Reviewers

Agree?

State Menu System

No

11.02.07 Insert

Data

Collection

Instrument

11.02.08 Review &Score File

Yes

If Failing Score

Interoffice to

Hub Supervisor

11.02.10 Archive

File

Excel

Process Ends

Continued from

Previous Page

Exhibit 11Audit - BTQ - Quarterly Reporting Process Flow


27/46

Page 23

2.3 AUDITBTQQUARTERLY REPORTING ACTIVITIES

Exhibit 12AuditBTQQuarterly Reporting Activities lists the activities that make up the Audit

BTQQuarterly Reporting sub-process. The following is a description and definition of each of thecolumns in this table.

Index A number for use in referencing the activityActivity Label A short name for that activity that is used to represent it in the detailed workflow diagram.

Activity Description The full-text description of that activity



11.02.01 Select SSNs for

Review

The UC Benefits system has coded within it a U.S. Department of

Labor (DOL) algorithm which automatically selects random claims

for review (50 Separations, 50 Non-Separations). For quarterly

reporting, the process is executed on the first workday of each

quarter (January, April, July, and October) and creates printouts

for each claim.

UCB45Wage Determination(s)

BH01History

The output is routed to the BTQ Supervisor via interoffice mail

and the Quarterly Reports are due by the 20 thday of the month

following the quarter (e.g., Quarterly Report review reviews for

Jan-Mar are run on April 1 and the results are due back to DOL by

May 20).

AWI Print Shop

11.02.02 Assign Reviewer The BTQ supervisor assigns a reviewer for each case. The cases

are separated by type (50 Separations and 50 Non-Separations).

The supervisor gives 50 of one type to one reviewer and 50 of the

other type to a different reviewer. Later, in this process, the

reviewers exchange cases and re-score each others work.

Supervisor

11.02.03 Setup Folder &

Label

The reviewer sets up a separate folder for each assigned case and

labels the folder.

Reviewer


Review Documents

The reviewer queries multiple systems to retrieve the information

necessary to complete the assessment.

Reviewer

11.02.05 Insert Data

Collection

Instrument

A blank data collection instrument (scoring sheet) is inserted into

the case folder by each reviewer.

Reviewer

11.02.06 Review & Score

File

The results of each review are scored in accordance with the ET

Handbook 301 (the scoring is indicated on the data collection

instrument for each scoring category).

Once all 50 assigned cases of one type are reviewed by one

examiner, the reviewers exchange the folders and score the cases

again. If there are any disagreements in the scoring, the cases are

submitted to a supervisor to be scored a third time.

If it appears a determination error was committed and is in

violation of law and policy, the case is referred to the Adjudication

Hub for redetermination.

Reviewer


28/46

Page 24



Collection

Instrument

If the initial reviewers do not agree about the score for a given

case, the case is scored by the BTQ supervisor using the same

process.

A blank data collection instrument (scoring sheet) is inserted into

the case folder.

Supervisor


File

The case is reviewed and scored in accordance with the ET

Handbook 301 as indicated on the data collection instrument.

Supervisor

11.02.09 Complete ETA

9056

Once all cases are scored, the results are input to the State Menu

system by completing the ETA 9056.

If a case receives a failing score, a memo and a copy of the case is

sent to the Adjudication Hub supervisor to whom the initial case

adjudicator reports.

Reviewer

11.02.10 Archive File The review files are stored in local archives for two years. Reviewer

11.02.11 Complete Internal

Report

An internal report (created in Excel), containing the results of the

reviews, is stored on the network Program Evaluation drive.

Supervisor

Exhibit 12AuditBTQQuarterly Reporting Activities

2.4

AUDITBTQQUARTERLY REPORTING PROCESS ATTRIBUTES

The following sections define and describe sub-process specifics for elapsed time, paper documentation,

business system use, communications, and process improvements. The information is arranged in thisway to support analysis of these activities, both individually and collectively.

2.4.1 Elapsed Time

Exhibit 13AuditBTQQuarterly Reporting Elapsed Time lists the time required to complete each of

the activities that make up the AuditBTQQuarterly Reporting sub-process. The times are estimatedexcept where noted. The following is a description and definition of each of the columns in this table.



Process Time The hands-on time it takes to complete the activity (does not include mail, transport, or time








29/46

Page 25

Index Activity

Label

Process

Time

Cycle

Time


Goal

Source

Speed Bumps

11.02 Quarterly

Reports

1.5-2 hrs

per file.

50 days Process Begins on the first

working day of Quarter.

Results due on the 20thof the

month following.

ETA 301

Handbook

published by

Department

of Labor.

Gathering

information from

multiple systems.

Information

occasionally not

present in system

requiring manual

intervention.

Exhibit 13AuditBTQQuarterly Reporting Elapsed Time


Exhibit 14AuditBTQ Paper Documentation lists the paper documents required to complete each ofthe activities that make up the AuditBTQQuarterly Reporting sub-process. The reference column

refers to the originator and title of policy, procedure, or instruction that contains the form or specifies the

content of the document. More than one document may be required to carry out a single activity, so eachis listed. The following is a description and definition of each of the columns in this table.




Document

Description




or Source


Review

UBMSEPBSA20 Report produced from the UC system identifying

the 50 Separation Issues scheduled for review.Includes official reproductions of the UCB-45 for

each issue.

AWI

Reviewer


Review

UBMSEP2390 Report produced from the UC system containing the

BH01 (Claims History) for each issue identified in

the report above.

AWI

Reviewer


Review

UBMNSEPBSA20 Report produced from the UC system identifying

the 50 Non-Separation Issues scheduled for review.

Includes official reproductions of the UCB-45 for

each issue.

AWI

Reviewer


Review

UBMNSEP2390 Report produced from the UC system containing the

BH01 (Claims History) for each issue identified in

the report above.

AWI

Reviewer

11.02.03 Setup Folder &

Label

Label Word Template Label used to identify review file. AWI

Reviewer


Review

Documents


System.

AWI

Reviewer


Review

Documents

Automated Non-

Monetary Worksheet

Adjudication Worksheet retrieved from the EIS

(Imaging) System. Accompanied by any other

documents imaged during the Adjudication Process.

AWI

Reviewer


30/46

Page 26


or Source


Collection

Instrument

Blank Word Template Document used to record information about file and

to record Reviewers comments and score.

Completed by hand.

AWI

Reviewer


Collection

Instrument


to record Reviewers comments and score.

Completed by hand.

AWI

Reviewer


9056

ETA 9056

Nonmonetary

Determinations

Quality Report (OMB

1205-0359)

Printout from the DOL State Menu System showing

the results of the review process. Attached to the

case file.

AWI

Reviewer

11.02.11 Complete Internal

Report

Summary Report Summary report of all 100 issues reviewed as part

of the cycle. Entered into Excel and stored on a

central share drive.

AWI

Reviewer

Exhibit 14AuditBTQ Paper Documentation

2.4.3

Business System Data Entry & Data TransferExhibit 15AuditBTQQuarterly Reporting Data Entry & Data Transfer lists the business systemsthat are employed directly (touched) or indirectly (data transfer) in the course of the Audit BTQQuarterly Reporting sub-process. The following is a description and definition of each of the columns in

this table.






Index Activity Label Business System Program or Intent Mode11.02.01 Select SSNs for

Review

UC Selects 100 issues (50 Separation, 50 Non-

Separation Claims) for review. Also triggers

the printout of the UBMSEPBSA20,

UBMSEP1430, UBMNSEPBSA20, and

UBMNSEP1430 reports.

Inquiry


Review

Documents

UC Retrieve By SSN


Inquiry


Review

Documents

EIS (Imaging) Input SSN. Select and print the

Adjudication Fact Finding File. If the Fact

Finding File does not contain the UCB-412, the

UCB-412 should be printed as well.

Inquiry


Review

Documents

FLUID Input SSN on the Search Screen. Results are

displayed and the appropriate Claim is selected

and printed. Used only if the Fact Finding File

does not already contain a copy.

Inquiry


31/46

Page 27



Review

Documents

AENR Input SSN on the Search Screen. Results are

displayed and the appropriate Employer

Verification is selected and printed. Used only

if the Fact Finding File does not already

contain a copy.

Inquiry

Exhibit 15

Audit

BTQ

Quarterly Reporting Data Entry & Data Transfer

2.4.4 Communication

Exhibit 16AuditBTQQuarterly Reporting Communication lists the messages, such as the US mail,emails, phone calls and any other correspondence, regularly used in the normal courseof the Audit

BTQQuarterly Reporting sub-process. Messages are sent to convey required documentation, informrecipients about the status of an activity, to trigger the start or end of other activities, to grant or withdrawauthorization, etc. This section only concerns messages sent during normal (as intended) operations.There may be more than one message associated with any activity in this process. The following is a

description and definition of each of the columns in this table.



Message Type US mail, email, phone call, etc.

Message Sender The organization or role of the message sender.





Type

Sender Message

Purpose

Message

Recipient

Occasion


Review

Interoffice AWI Print

Shop

Provide copies of

UBMSEPBSA20,

UBMSEP1430,

UBMNSEPBSA20, and

UBMNSEP1430

reports.

Supervisor First working day

of each Quarter.

Starts the review

process.


File

Interoffice Reviewer Request

redetermination

of a particular

claim based on

facts uncovered

during the

review.

Adjudication

Hub

When errors are

found in the initial

claim adjudication.


9056

Interoffice Reviewer Inform

Adjudicator

Supervisor of a

failing score

(Training

opportunity)

Adjudication

Hub

If a case file has a

failing score.

Exhibit 16AuditBTQQuarterly Reporting Communication

2.4.5 Opportunities

The purpose of this section is to capture improvement suggestions that may have a positive effect on


32/46

Page 28

technical, functional, or physical components of the AuditBTQ Quarterly reporting activities.

Over the course of our interviews and observations, we identified a number of possible process

improvement opportunities (long and short term). The following tables contain descriptions (ProblemStatement) of what may be negatively affecting current operations and corresponding potential solutions

(Recommendation) to address the issues. In some cases, additional analysis is required to understand

fully the impact of the problems and/or recommendations.

The following is a description and definition of each of the columns in this table.

Origin The area of the improvement such as: Central Office, Hub, etc.

Problem

Statement

Description of the problem or opportunity identified.

Recommendation Description of the proposed recommendation, including expected result.

Value Added Description of the expected value received if the recommendation to the problem statement is

identified.

Short Term Opportunities for Improvement

Origin Problem Statement Recommendation Value Added11.02.01 Quarterly audits are not being run at this

time (due to workload constraints) but

the mainframe reports are still being

created (1,000+ pages).

Ideally, an option should be

provided to turn off the

reports unless they are needed

once reporting commences in

the future.

Paper savings as well as a

reduction in overall workload

(printing, dissemination,

shredding).

Exhibit 17Short Term Opportunities for Improvement

Long Term Opportunities for Improvement


The UC Benefits system has coded

within it a U.S. Department of Labor

(DOL) algorithm which automatically

selects random claims for review (50

Separations, 50 Non-Separations).

The algorithm in the future

system should be

administrator controlled to

obviate the need to modify

code within the system.

Ideally, the interface between

the DOLs State Menu system

and the new UC system will

be automated and preclude the

need for UC staff to perform

the manual download/upload

processes.

No need for IT resources to

implement code changes in

response to USDOL audit

requirements.

Exhibit 18Long Term Opportunities for Improvement


33/46

Page 29

3 PURPOSE AND OBJECTIVES OF THE AUDITBTQ NON-

MONETARY REVIEWS SUB-PROCESS


(BAM), Benefit Timeliness and Quality (BTQ)Quarterly Reporting, BTQNon-Monetary Reviews,and BTQData Validation. The following sections describe the BTQ Non-Monetary related activities.

Benefit Timeliness Quality (BTQ)Non-Monetary Reviews

UC has instituted an internal quality review process under the auspices of the BTQ unit to perform

periodic assessments of adjudicators in each Adjudication Hub. The information gathered through thereviews provides insight to existing or potential issues that may be causing difficulties in particularadjudication areas (e.g., separation issues). Managers can use that information to take corrective actionwhich may include arranging for remedial training, modifying policies and procedures, and/or suggestingchanges to the automated system(s) to address less than satisfactory performance.

Note: Many units within AWI perform audit functions to monitor their performance. The processesdescribed here address specifically BAM and BTQ and are not meant to encompass other audit functions

within AWI.

The following sections describe the activities associated with the Audit - BTQ Non-Monetary Reviewsfunctions in UC.

3.1


BTQ

Beginning Point:

An Adjudication Hub is selected for review.

Ending Point:

The results of the claims reviews are recorded in an internal reporting document.


34/46

Page 30

3.2 AUDITBTQNON-MONETARY REVIEWS DIAGRAM

Supervisor Reviewer Adjudicator Supervisor

11.03.05 Insert

Data

Collection

Instrument

11.03.06 Review &

Score File

11.03.01 Run

Process

for Selected

Time Period

11.03.02 Assign

Reviewer 11.03.04 Retrieve

Pre-Review

Documents

UC, Imaging,FLUID, AENR

UC

Process Start

11.03.07 Complete

Summary Scoring

Report for

Adjudicator

Excel

11.03.03 Select 10

SSNs for Review

For Each SSN

For Each Adjudicator

11.03.08 Complete

Summary Scoring

Report for Hub

Process Ends

Word/Excel

Interoffice

Hub Score

Report

Exhibit 193.2 Audit BTQNon-Monetary Reviews Process Flow


35/46

Page 31

3.3 AUDITBTQNON-MONETARY REVIEWS ACTIVITIES

Exhibit 20AuditBTQNon-Monetary Reviews Activities lists the activities that make up the Audit

BTQNon-Monetary Reviews sub-process. The following is a description and definition of each ofthe columns in this table.

Index A number for use in referencing the activityActivity Label A short name for that activity that is used to represent it in the detailed workflow diagram.

Activity Description The full-text description of that activity



11.03.01 Run Process for

Selected Time

Period

The BTQ unit conducts periodic audits of the Adjudication Hubs.

The supervisor begins the process by requesting a SAR Report of

all closed cases for a specified time period (usually a week) for a

particular adjudicator. This process is repeated for each

adjudicator to be audited.

Note: This process is very similar to the Quarterly Review outlined

above and is used to support internal quality reviews.

Supervisor

11.03.02 Assign Reviewer The BTQ supervisor assigns reviewers to complete the audits. Supervisor

11.03.03 Select 10 SSNs for

Review

The reviewer selects 10 social security numbers (determinations

for particular claimants) from the SAR Report and goes through an

iterative process to review all 10 cases.

Note: The reviewer attempts to select more Separation cases than

Non-Separation cases (Separation cases, in general, have a higher

error rate).

Reviewer


Review Documents

The reviewer queries numerous systems to gather the information

necessary to complete the examination.

Reviewer


CollectionInstrument

The information is written into the data collection instrument. Reviewer


File

The collection instrument is reviewed and scored. At the end of

the process, a copy of the scoring sheet, for each adjudicator, will

be provided to the Adjudication Hub supervisor.

Reviewer

11.03.07 Complete Summary

Scoring Report for

Adjudicator

The Reviewer creates a summary report (Excel file) for each

adjudicator and stores the results on the Program Eval Drive (a

designated network drive). At the end of the process, the reports

will also be provided to the Adjudicators supervisor.

Reviewer

11.03.08 Complete Summary

Scoring Report for

Hub

The supervisor prepares a summary report for the entire

Adjudication Hub (summary scores for each adjudicator and

overall statistics for the Hub) and a cover letter (memorandum)

and mails the package to the Adjudication Hub supervisor along

with the individual scoring sheets and summary scoring reports for

each adjudicator.

The supervisor also updates the state-wide adjudication hub review

results.

All reports are stored on the Program Eval drive.

Supervisor

Exhibit 20AuditBTQNon-Monetary Reviews Activities


36/46

Page 32

3.4 AUDITBTQNON-MONETARY REVIEWS PROCESS ATTRIBUTES

The following sections define and describe sub-process specifics for elapsed time, paper documentation,

business system use, communications, and process improvements. The information is arranged in thisway to support analysis of these activities, both individually and collectively.

3.4.1

Elapsed Time

Exhibit 21AuditBTQNon-Monetary Reviews Activity Elapsed Time lists the time required tocomplete each of the activities that make up the AuditBTQNon-Monetary Reviews sub-process. Thetimes are estimated except where noted. The following is a description and definition of each of thecolumns in this table.



Process Time The hands-on time it takes to complete the activity (does not include mail, transport, or time







Index Activity

Label

Process

Time

Cycle

Time


Goal

Source

Speed Bumps

11.03 Non-Monetary

Reviews

11.5 hrs

per file.

Varies

based on

size of

Hub.

Policy is for one to two Hubs

to be evaluated each Quarter.

Internal

Policy

Gathering

information from

multiple systems.

Information

occasionally not

present in systemrequiring manual

intervention.

Exhibit 21AuditBTQNon-Monetary Reviews Activity Elapsed Time


Exhibit 22AuditBTQNon-Monetary Reviews Paper Documentation lists the paper documentsrequired to complete each of the activities that make up AuditBTQNon-Monetary Reviews sub-

process. The reference column refers to the originator and title of policy, procedure, or instruction thatcontains the form or specifies the content of the document. More than one document may be required to

carry out a single activity, so each is listed. The following is a description and definition of each of thecolumns in this table.




Document

Description




37/46

Page 33


or Source


Selected Time

Period

UBDNONMW1420 List of closed adjudications within a selected time

period for a given adjudicator.

BTQ

Supervisor


Review

BH01 Claims History. Printed from the UC System. AWI

Reviewer


Review

UCB45 Claim Determination. Printed from the UC System. AWI

Reviewer


Review

Documents


System.

AWI

Reviewer


Review

Documents

Automated Non-

Monetary Worksheet

Adjudication Worksheet retrieved from the EIS

(Imaging) System. Accompanied by any other

documents imaged during the Adjudication Process.

AWI

Reviewer


Collection

Instrument


to record Reviewer's comments and score.

Completed by hand.

AWI

Reviewer

11.03.07 CompleteSummary Scoring

Report for

Adjudicator

Excel Template Report of all cases reviewed for a particularadjudicator with score and comments.

AWIReviewer

11.03.08 Complete

Summary Scoring

Report for Hub

(incl. Memo) &

State

Excel Chart Template Report of all cases reviewed for a Hub. BTQ

Supervisor

11.03.08 Complete

Summary Scoring

Report for Hub

(incl. Memo) &

State

Word Template Memo used as a cover sheet for the Adjudication

Hub Scoring Report

BTQ

Supervisor

11.03.08 Complete

Summary Scoring

Report for Hub

(incl. Memo) &

State

Excel Chart Template Report of all cases reviewed for the State BTQ

Supervisor

Exhibit 22AuditBTQNon-Monetary Reviews Paper Documentation

3.4.3

Business System Data Entry & Data Transfer

Exhibit 23AuditBTQNon-Monetary Reviews Data Entry & Data Transfer lists the businesssystems that are employed directly (touched) or indirectly (data transfer) in the course of the AuditBTQ

Non-Monetary Reviews sub-process. The following is a description and definition of each of the

columns in this table.







38/46

Page 34



Selected Time

Period

UC Input an Adjudicator Name and a Time period:

(UBDNONMW1420) List of closed

adjudications within the input time period for

the adjudicator.

BTQ

Supervisor


Review

Documents

UC Retrieve by SSN:


(UCB45) Wage Determinations

(BH01) Claim History

Inquiry


Review

Documents

EIS (Imaging) Input SSN. Select and print the

Adjudication Fact Finding File. If the Fact

Finding File does not contain the UCB-412,

the UCB-412 should be printed as well.

Inquiry


Review

Documents

FLUID Input SSN on the Search Screen. Results are

displayed and the appropriate Claim is selected

and printed. Used only if the Fact Finding File

does not already contain a copy.

Inquiry


ReviewDocuments

AENR Input SSN on the Search Screen. Results are

displayed and the appropriate EmployerVerification is selected and printed. Used only

if the Fact Finding File does not already

contain a copy.

Inquiry

Exhibit 23AuditBTQNon-Monetary Reviews Data Entry & Data Transfer

3.4.4 Communication

Exhibit 24AuditBTQNon-Monetary Reviews Communication lists the messages, such as the USmail, emails, phone calls and any other correspondence, regularly used in the normal courseof the Audit

BTQNon-Monetary Reviews sub-process. Messages are sent to convey required documentation,inform recipients about the status of an activity, to trigger the start or end of other activities, to grant orwithdraw authorization, etc. This section only concerns messages sent during normal (as intended)

operations. There may be more than one message associated with any activity in this process. Thefollowing is a description and definition of each of the columns in this table.



Message Type US mail, email, phone call, etc.

Message Sender The organization or role of the message sender.





Type

Sender Message

Purpose

Message

Recipient

Occasion

11.03.08 Complete

Summary

Scoring Report

for Hub (incl.

Memo) & State

Interoffice BTQ

Supervisor

Provide

Adjudicator

Supervisor with

all scoring

documentation.

Adjudication

Supervisor

After any Hub

review has been

completed.

Exhibit 24AuditBTQNon-Monetary Reviews Communication


39/46

Page 35

3.4.5 Opportunities

The purpose of this section is to capture improvement suggestions that may have a positive effect ontechnical, functional, or physical components of the AuditBTQ Non-Monetary Review activities.

Over the course of our interviews and observations, we identified a number of possible process

improvement opportunities (long and short term). The following tables contain descriptions (ProblemStatement) of what may be negatively affecting current operations and corresponding potential solutions(Recommendation) to address the issues. In some cases,