ICD-10 Testing January 27-30, 2015 Puerto Rico ICD-10 Implementation Assistance Site Visit ICD-10 implementation segments to assist the Puerto Rico with

ICD-10 Overview

Program Integrity

Code Structure & Definition

GEMS, Translation & Dual Processing

Claims Management

Analytics & IT Infrastructure

ICD-10 Testing

Post Implementation Imp. & Opportunities

Mapping & Policy Remediation

ICD-10 for Provider Administrative Staff

ICD-10 for Clinicians

ICD-10 Testing

January 27-30, 2015

Puerto Rico ICD-10 Implementation Assistance Site Visit ICD-10 implementation segments to assist the Puerto Rico with the transition

Agenda

Definition of testing types, components and artifacts The testing life-cycle Test plans, testing strategy and testing governance Scenario based testing Testing templates Managing recursive testing and remediation External testing

2

3

I don’t always test my systems, but when I do my friend,

I do it in production!

The most interesting man in the world!

Not the smartest, but interesting…

4Health Data Consulting © 2010

Components

<Graphic>

Testing ComponentsDefinitions

Test Plan– The test plan is the overall document that describes how

testing will be done for the enterprise relative to a change in business or system implementation.

Test Strategy– The test strategy, also referred to as the test approach,

speaks to the strategic goals of testing and includes a reference to what, how and why various testing components will be accomplished based on the stated goals and rationale for the approach.

5Source: Health Data Consulting 2013HResourcesc


Test Cases– Test cases are scenarios that represent aspects of the

business or intended system functions where failure may occur. These are generally prioritized based on financial impact, likelihood, critical path business functions and any other parameters that are important to the business enterprise.

Test Data– Test data is specific data that is created in a variety of

formats to be uploaded or entered into the system based on defined test cases.



Test Environment– The test environment traditionally refers to the system

platforms or configurations technically that have been set up outside of the production work flow to allow applications of test cases and evaluate results in the new system design.

Testing Governance– A structure defines the charter that sets parameters for

decisions on prioritization of testing and remediation and resolution of issues.

7


Testing Documentation– All aspects of the testing process will need to be

documented including test plans, test cases, governance charters, results reporting, project plans and all other aspects of testing that are important to the business.

Error Management– Tracking the identification, prioritization and resolution of

errors and issues discovered during the testing process

Testing Remediation– Testing remediation refers to the process of solving system

problems or “fixing bugs”.



Regression Testing– Regression testing refers to the ongoing process of system

fixes followed by retesting.

Internal Testing– Testing of internal systems and processes.

External Testing– Testing transactions with trading partners outside of the

internal enterprise.



Testing Phases– For the State Medicaid Agency testing check list, NGS

(National Government Services) has defined 3 levels of testing as part of the published testing check list.Level 1: Internal testing readinessLevel 2: testing with external trading partners is readyLevel 3: Testing with external trading partners has been

demonstrated to be “production ready”



Known Issues– Issues that have been documented and remain after final

remediation and release of the system version.

Unit Testing– Testing of isolated units, modules or sub-systems

independent of overall system function.

Integration Testing– Testing of the entire system after all remediated modules

have been tested



User Acceptance Testing– Testing by end users to make sure that the system meets

functional needs.

End to End Testing– Testing from a defined point of entry into the system to the

defined ending point on the system. Most commonly this means testing “round trip” internally and externally.

Business Testing– Testing to assure that the business will operate properly

after change even though the system is functioning as specified.


Testing ComponentsSystem Testing Types

Functional Testing– User Interface Testing– System Interface Testing– Report Testing– User Access Testing– Data Validation Testing– Import/Export Testing


Testing ComponentsSystem Testing Types

Non-functional Testing– Load/Stress Testing– Performance Testing– Accessibility Testing– Compatibility Testing– Recovery Testing– Scalability Testing– Security Testing– Monitor testing


15Health Data Consulting © 2010

Testing Lifecycle

Planning

Implementation

Resolution


The Testing LifecycleOverview

Focused on internal testing Prerequisites– Defining the testing strategy and goals– Establishing the testing environment– Executive levels understanding and support– Testing is a project; you need a project plan and

project management

Understanding the difference between business testing and system testing



The Testing Lifecycle1. Define change


Defining the change– Change in business requirements– Change in the system to support the same business

requirements

Defining the risk of change– Likelihood of failure– Impact of failure

MoneyTime Relationships

– What it takes to fix failure


The Testing Lifecycle2. Establish Governance


Strategy and Goals– What is the purpose of the change in the business or

system– What are the expected outcomes at a strategic level– What are the key strategic priorities

Ownership and accountability– Who sets the agenda (including scope)– Who monitors the agenda– Who has the authority to enforces the agenda

Acceptable levels of risk or failure


The Testing Lifecycle3. Create test plan


Defining the test plan– Test environment– Tactical definition of the scope– Tactical definition of the priorities– Defining and managing the project plan– Allocating resources– Assigning authority and accountability– Setting expectations (acceptable risk and limitations)– Contingencies

Test Plan Template


Predicting the future is difficult if we have no historical

reference for what’s coming.

Reference Implementation Model

Virtual worlds are safer that real ones

The Testing Lifecycle4. Create Scenarios


Scenarios simulate the business environment They are based on what we know today– High dollar– High volume– High complexity– Important to other business functions

Quality metricsPopulation riskEffectivenessOther initiatives

Scenario Development Example


The Testing Lifecycle5. Define test cases


Based on scenarios and system response to those scenarios

Different scenario based test cases may be needed for different functional and non-functional requirements– Example user Interface test case– Example report test case– And other functional non functional requirements

Test cases should be developed from business requirements not from system development


The Testing Lifecycle6. Create Test Data


Test data is directly linked to the test cases for each business requirement

Test data should be based on existing data with known system response and then altering parameters of that data to demonstrate business and system changes as defined in the test cast

Example of test case development


The Testing Lifecycle7. Execute Test Cases


Once test cases are defined they need to be executed and the results assessed against the expected result.

Key questions to consider– Is the environment ready to execute the test cases– Are there blocking issues– What is the turnaround time for remediation in order

to re-test the scenario– Does executing the test cases reveal the need for new

requirements of new scenarios and test cases


The Testing Lifecycle8. Document Findings


Documenting test findings is important:– Creates an ongoing work list– Identifies changes in scope and resource requirement for

the testing project– Provides the basis for adjusting scope, resources or time to

completion– Establishes decision points for governance– Helps to set expectations for end users and executive

management

– Provides a basis for requirements that may be pushed to the next scheduled system release.


The Testing Lifecycle9. Manage Errors


Testing will result in a list of errors– Tools will be needed to record errors,– Assign source– Attach to known requirements– Assign accountable resource for remediations– Assign priorities for remediations– Track progress– Share with development that may be impacted in other

modules


The Testing LifecycleBug Tracking Software Example


The Testing LifecycleBug Tracking Software Example

The Testing Lifecycle10. Remediate Errors




Once identified errors will need to be remediated– Identifying causes– Scheduling fixes– Finding resources– Who’s responsible?– Relationships – where and who may be impacted– Reporting – progress and challenges– Accepting – fix is good enough?



The Testing Lifecycle11. Re-test


A number of iterations will be needed to make sure the fix worked and continued to work– The fix may not have worked– The fix may have worked from a system perspective but still

not meet the business requirement – The fixed may have impacted other aspects of the module

and created other errors– The module may function properly but fail on system

integration testing– The level of regression test acceptable will need to be

established through the governance structure.



The Testing Lifecycle12. Document Final State


Summary of testing– Summary of initial strategy and Goals– Summary of key plan components– Summary of successful requirements test– Summary of requirements discovered during the testing

process– Itemization of known issues– Plan for remediation of known issues

IgnoreWorkaroundScheduled for future release



The Testing Lifecycle13. Define Next Release Cycle


Identify all of the planned features to be added to the next system release

Identify if the feature is an known issue remaining after the testing process

Identify if the planned feature is based on a discovery of the need for the new feature noted during the testing cycle but not defined in pre-testing requirements

Defining Trading Partners

Who are the data trading partners that will be impacted by the transition?

Which trading partners are at risk and how can risk be determined?

What’s the level of awareness? Are they friendly? What’s the scope of interaction? What’s the risk to your organization? Where are the priorities?

48

Establishing the Testing Rationale

SMAs want to know how providers are going to code. Providers want to know how the SMAs are going to

pay. Only cross external testing will address both

questions Identify those trading partners that represent a

significant part of the business on both sides or a significant risk for both

49

The External Testing Environment

Define relevant scenarios for testing Provide a portal for testing Provide ongoing support during the testing process Make sure that internal system remediation is ready

to support external testing Provider feed back on testing progress Provide a forum to discuss options for addressing

problems in a way that benefits both sides

50

External Testing What is external testing? Who benefits? Universal or selective testing? Platforms and portals for testing? Managing the testing process Certification for production? Sharing results with all stakeholders

51

Post Implementation Testing will not be done October 1, 2015 Many errors will be discovered that demand a

solution Solutions will need to be tested to make sure that

the problem is resolved and the solution did not create another problem

Assuming a problem free implementation, there will be many changes after the ICD-10 transition date that will require new testing of new system functions

52

Critical success factorsCMS define 5 critical success factors

53

1. Accept electronic claims

2. Adjudicate claims

3. Payment

4. Coordination of benefits

5. MSIS/T-MSIS data submission

Critical success factors1. Accept electronic claims

54

Process question:– Will the SMA be able to accept electronic claims with ICD-9 and ICD-10

coding based on the dates of service and dates of discharge using Version 5010 transactions on October 1, 2015?

Confirm:– Testing EDI gateways with of a variety or common test claim scenarios

from key clearinghouse and direct provider submitters to assure that ICD-9 and ICD-10 claims can be accepted into the MMIS system.

– Testing to assure that claim rejections associated with invalid codes provide the correct response transactions to the submitter.

– Assure that processes are in place to provide feedback and training support for submitters who are having challenges with test transactions.

Critical success factors2. Adjudicate claims

55

Process question:– Will the SMA be able to adjudicate diagnosis dependent claims?

Confirm:– Create and implement test cases with claim scenarios that will be

impacted by ICD-10 based policies, coverage logic, and other ICD-10 related edits; and rules function according to the intent of the policy, rule, or edit.

– Test claims in ICD-9 and ICD-10 given the same claims scenario to assure that processing results in the intended outcomes.

– Assure there is a process for ongoing remediation of the SMA, MCO, and provider side to address issues with coding as well as rule remediation as early as possible.

Critical success factors3.a Payment (Provider claim payment)

56

Process question:– Will the SMA be able to make provider payments to include

professional and institutional providers for ICD-9/ICD-10 codes on October 1, 2015?

Confirm:– Create and implement test claim scenarios with emphasis on high

dollar and high volume cases to assure that payment will occur as anticipated.

– Create test claims in ICD-9 and ICD-10 given the same claims scenario to assure that claim payment will be comparable before and after transition.

– Assure there is a process for ongoing remediation of the SMA, MCO, and provider side to address issues with payment as early as possible to assure revenue neutrality.

Critical success factors3.b Payment (MCOs)

57

Process question:– Will SMA be able to pay MCOs for ICD-9/ICD-10 codes on

October 1, 2015?

Confirm:– Test to evaluate MCO payments including risk adjustments,

‘kicker’ payments, or other condition related carve out models based on the use of ICD data operate as anticipated in an ICD-10 environment.

– Assure revenue neutrality for MCOs payments modeled on ICD-10 assumptions.

– Assure that all claim/encounter data is submitted from MCOs and that data content and quality is acceptable to support.

Critical success factors4. Coordination of benefits

58

Process question:– Will the SMA be able to complete coordination of benefit

processes and exchange claims with partners including Medicare and others on October 1, 2015?

Confirm:– Create test case scenarios where coordination of benefits

may occur and test with key trading partners to assure transaction acceptance inbound and outbound.

– Use test cases to assure that claims scenarios that should trigger coordination of benefits do so.

Critical success factors5. T-MSIS and MSIS reports

59

Process question:– Will the SMA be able to create and send MSIS and/or

TMSIS reports (data exchanged from MMIS to CMS MSIS) for ICD-10 claims on October 1, 2015?

Confirm:– Test to assure that all required data elements

associated with ICD-10 code and code type submission are populated according to specifications.

– Work with the CMS to assure that submitted MSIS and T-MSIS files meet required validation for content and format (especially related to ICD-10 codes and code type data elements).

Questions

60

Documents

ICD-10 Testing January 27-30, 2015 Puerto Rico ICD-10 Implementation Assistance Site Visit ICD-10 implementation segments to assist the Puerto Rico with