Embed Size (px)
Citation preview
Medicare and Medicaid Audits Using
Statistical Sampling and Extrapolation:
Challenging Methods and Results
Today’s faculty features:
1pm Eastern | 12pm Central | 11am Mountain | 10am Pacific
The audio portion of the conference may be accessed via the telephone or by using your computer's
speakers. Please refer to the instructions emailed to registrants for additional information. If you
have any questions, please contact Customer Service at 1-800-926-7926 ext. 1.
THURSDAY, JUNE 14, 2018
Presenting a live 90-minute webinar with interactive Q&A
Anna M. Grizzle, Member, Bass Berry & Sims, Nashville, Tenn.
Dr. Patricia L. Maykuth, Ph.D, President, Research Design Associates, Decatur, Ga.
Tips for Optimal Quality
Sound Quality
If you are listening via your computer speakers, please note that the quality
of your sound will vary depending on the speed and quality of your internet
connection.
If the sound quality is not satisfactory, you may listen via the phone: dial
1-866-927-5568 and enter your PIN when prompted. Otherwise, please
send us a chat or e-mail [email protected] immediately so we can address
the problem.
If you dialed in and have any difficulties during the call, press *0 for assistance.
Viewing Quality
To maximize your screen, press the F11 key on your keyboard. To exit full screen,
press the F11 key again.
FOR LIVE EVENT ONLY
Continuing Education Credits
In order for us to process your continuing education credit, you must confirm your
participation in this webinar by completing and submitting the Attendance
Affirmation/Evaluation after the webinar.
A link to the Attendance Affirmation/Evaluation will be in the thank you email
that you will receive immediately following the program.
For additional information about continuing education, call us at 1-800-926-7926
ext. 2.
FOR LIVE EVENT ONLY
Program Materials
If you have not printed the conference materials for this program, please
complete the following steps:
• Click on the ^ symbol next to “Conference Materials” in the middle of the left-
hand column on your screen.
• Click on the tab labeled “Handouts” that appears, and there you will see a
PDF of the slides for today's program.
• Double click on the PDF and a separate page will open.
• Print the slides by clicking on the printer icon.
FOR LIVE EVENT ONLY
Medicare and Medicaid Audits Using Statistical Sampling and Extrapolation
Anna Grizzle - Bass, Berry & Sims, PLCPat Maykuth, Ph.D. - Research Design Associates, Inc.
Use of Statistical Sampling for Overpayment Estimation
• Medicare or Medicaid audit
– MAC audits following TPE reviews
– ZPIC or UPIC audits
– OIG audits
– Medicaid agency audits
– MIC audits
• OIG self-disclosure protocol
• Internal compliance audit
• Calculation of damages in FCA case?
6
Legal Basis for Statistical Sampling for Overpayment Estimation
“The use of statistical sampling to project anoverpayment. . . does not deny a provider orsupplier due process. Neither the statute norregulations require that a case-by-case reviewbe conducted in order to determine that aprovider or supplier has been overpaid and todetermine the amount of overpayment.”
HCFA Ruling 86-1
7
Legal Basis for Statistical Sampling for Overpayment Estimation
Statistical sampling does not violate dueprocess “so long as extrapolation is madefrom a representative sample and isstatistically significant.”
Chaves County Home Health Service, Inc. v.Sullivan, 931 F.2d 914 (D.C. Cir. 1991), cert.denied, 402 U.S. 1091 (1992).
8
Legal Basis for Medicare Statistical Sampling and Extrapolation
A Medicare contractor may not use extrapolation to determine overpayment amounts . . . unless . . .
– There is a sustained or high level of payment error; or
– Documented educational intervention has failed to correct the payment error
42 U.S.C. §1395ddd(f)(3)
9
Legal Basis for Medicare Statistical Sampling and Extrapolation
• The PIM provides basic concepts rather than a checklist to complete a valid statistical sampling.
– When applied correctly, the PIM’s concepts can lead to a proper methodology to use as a basis for extrapolation.
• The PIM’s concepts often are not applied correctly in developing the statistical sampling methodology used as the basis of extrapolation in Medicare audits.
Source: Chapter 8 – Benefit Integrity; Medicare Program Integrity Manual; available at:
http://www.cms.gov/manuals/downloads/pim83c08.pdf10
Legal Basis for Medicaid Statistical Sampling and Extrapolation
• Dictated by state law
• If no explicit authority, look to due process requirements
11
Performance of Statistical Sampling and Extrapolated Overpayment
• Major Steps
– Selecting the provider or supplier• Prior history of overpayment error
• TPE Review by MACs
– Selecting the period to be reviewed
– Defining the universe, the sampling unit, and the sampling frame• Define the provider, issue to be reviewed, time
period, and methodology for measuring overpayment
Source: Chapter 8 – Benefit Integrity; Medicare Program Integrity Manual; available at:
http://www.cms.gov/manuals/downloads/pim83c08.pdf
12
Performance of Statistical Sampling and Extrapolated Overpayment
• Major Steps (cont.)
– Designing the sampling plan and selecting the sample
– Reviewing each of the sampling units and determining if there was an overpayment or under payment
– Estimating the overpayment
Source: Chapter 8 – Benefit Integrity; Medicare Program Integrity Manual; available at:
http://www.cms.gov/manuals/downloads/pim83c08.pdf
13
Documentation Requirements
• The following items must be documented:
– Universe
– Sampling frame (sorted)
– Random sampling process (seed, program, inputs and printout)
– Sample size determination calculations
– Extrapolation formulae, inputs and printouts
– Extrapolation recalculation where appropriate
14
Valid Methodologies
• MPIM specifically lists:
– Simple random
– Stratified random
– Cluster
• MPIM specifically requires proper execution of chosen methodology
15
Valid Statistics
• Statistically Valid Random Sample (SVRS)
• Probability Sample
• Use correct formulae
• Follow statistical requirements of chosen statistics
• Point Estimates and Confidence Intervals are valid statistics – if properly executed
• Minimum Sum Method and Penny sampling not been validated in the statistical literature
16
Probability Statistics • To know the number of samples of the chosen size
that can be created from the frame
• Known likelihood of selection of each sampling unit
• Proper randomization
• Proper execution of sample methodology
• Use correct formulae
• Accurate measurement of the variable of interest (overpayment)
Source: Chapter 8 –§ 8.4.2 Benefit Integrity; Medicare Program Integrity Manual;
available at: http://www.cms.gov/manuals/downloads/pim83c08.pdf17
Frame
x
Sample
(1, 2, 3, 4, 5) (2, 3, 4, 5, 6) (1, 11, 21, 31, 41) (1, 12, 23, 34, 45) (8, 23, 46, 73, 90)
(1, 2, 3, 4, X)
Outside of frame
18
Probability Sample
1. To be able to identify the number of samples of a given size that can possibly be selected from a frame of a frame of a given size claims. How many samples of 5 can be selected from a frame of 100?
2. In a simple sample each claim must have a known and equal probability of selection. Strata take simple samples from strata frames.
19
Probability Yardstick• Single sample chosen for the audit is only one sample (of the chosen
size) out of a large number of possible samples
• Sampling distribution of all possible means provides mathematical model of what is likely to occur if all possible samples analyzed
• If a large number of samples were drawn from the frame:
– A mean can be calculated for each sample
– Each sample mean would not be exactly the same value as others
– Means would be different from one another but would cluster around the frame’s central value (or mean)
– Differences in the means of different samples are basis of the error that occurs inferring from a sample to the frame rather than measuring all of the claims in the frame
– If repeated random samples (moving toward infinity) were made, means would be expected to fall into a normal distribution
20
Possible Samples That Can Be Randomly Drawn From the Frame
21
What’s the Big Deal About the Normal Distribution?
• In probability theory, the normal (or bell-shaped) distribution is continuous probability distribution (a function that tells the probability of a number falling between any two real numbers). The normal distribution is symmetric around the mean. The mean, median and mode are the same number.
• The normal distribution is immensely useful because of the Central Limit Theorem which states that the mean of many random variables independently drawn from the same distribution is distributed approximately normally, irrespective of the form of the original distribution. That is, the overpayment means will be randomly distributed if the sample is large ... moving toward infinity.
22
“Normal” Distribution
23
Confidence Levels on Normal Distribution
Lower Confidence Limit
Point Estimate
Upper Confidence Limit
24
25
Mode
Mean
Median
0 12,072.62566.09 1,274.35
Where is the Confidence Level?One-sided or Two
Illustration not based on actual data
26
Distribution of Frame
Paid Dollars
27
Non Normal Mean and Standard Deviation
Sample mean +/- 1 standard deviation
Mean = 430.92 + sd = 525.51 = 956.43
430.92 - 25.51 = -94.18
28
Confidence Levels Non Normal Data
Point Estimate +/- ½ Confidence Interval
Confidence Interval = 16,778 + 16,778 = 33,556
Lower 27,147 + 16,778 = $43, 925
Upper 27,147 - 16,778 = $ 10,368
Lower confidence Level ???Unknowable
29
Properly Executed Random Sample
• Prior known error rate or probe
• Appropriate for use with the audit data (dependent or independent)
• Of sufficient size to support the statistic used
• Use a replicatable random process
• Tested for randomness
• Is representative (without mathematical bias) and tested
30
31
Sample Size Determination Based on Chosen Precision and
Confidence
32
RAT-STATS Results
80% 90% 95% 99%
1% 75 77 78 79
2% 63 68 71 75
5% 29* 39 46 56
10% 10* 15* 20* 29*
15% 5* 8* 10* 16*
Confidence Level
Precision
Level
* Caution sample sizes less than 30
33
Representative Samples
• A small yet complete and accurate picture of the data in the frame.
• A subset of a statistical universe that accurately reflects the numerical membership of the entire universe and its distribution.
• A representative sample is an unbiased indication of what the frame is like.
• Representativeness is tested mathematically. When a sample is not representative, the result is known as a sampling error.
34
Representative Sample
Frame
sample
35
Sobering How Poor a Sample Can Be
36
RAT-STATS Overpayment Estimation
• Formulae:
• Confidence Level:
37
Precision vs Accuracy
• Precision measures how often the measurement tool produces a similar result every time it is used.
• Accuracy measures how close the sample value comes to the true value. How close is the sample overpayment mean to the true mean of overpayments in the frame.
• Sample results can be accurate but not precise or precise but not accurate
• Extrapolations require that the results meet a predetermined level of both.
38
Precision Calculation
Point Estimate +/- Precision amount = Confidence Level
Precision amount/Point estimate = Precision %
It is absurd to say the lower confidence level benefits the provider. The accuracy of the estimate is only the top and bottom of the confidence level around the point estimate. The lower confidence level is as close to accurate overpayment that can be obtained.
39
Sample of 5
If:
• A random sample of 5 selects the top most expensive claims (5540, 5550, 5555, 5560, 5570) mean = 5555
• Frame of 100 ranges from 500 to 5570 has a mean = 2827• The precision will very small and the confidence levels
will be very tight but the point estimate would be nearly twice as large as the frame mean
• The precision would be good but the accuracy would not• Point estimates can exceed the amount actually paid• It is a bad estimate.
40
41
Sample of 5
If:• A random sample of 5 selects the top 2 claims and bottom 3
(500, 525 540, 5560, 5570) mean = 2539• Frame of 100 ranges from 500 to 5570 has a mean = 2827• The precision will very large and the confidence levels will be
very large but the point estimate would be close to the frame mean
• The precision would be unacceptable but the accuracy would be OK
• The lower confidence level can be a negative number indicating an underpayment
• It is a bad estimate
42
Poor Audit Design & Execution Produce Only “Invalid Results”
Statistics in the hands of an inept auditor are like a
lamppost to a drunk: they are used more for support than
illumination.
43
Defending Against Extrapolation Results
• Medicare Appeals Process
– Redetermination
– Reconsideration
– Administrative Law Judge Hearing
– Medicare Appeals Council
– Federal District Court
• Medicaid Appeals Process
– Appeal rights under state law
44
Defending Against Extrapolation Results
• No administrative or judicial review of determination of high level of payment error BUT determination must be made
• Failure to follow one or more requirements in Benefit Integrity Manual does not necessarily affect validity
• Not sufficient to argue better or more precise methods are available
45
Defending Against Extrapolation Results
• Can challenge validity of sampling methodology based on “the actual statistical validity of the sample as drawn and conducted”
• Test: Was the sample statistically valid?
• Contractor has burden of establishing sample was in fact random and statistically valid
46
Defending AgainstExtrapolation Results
• Procedural Challenges– Did the contractor provide work papers
allowing for review and replication of sampling results for every stage of the process?
– Did the contractor follow the guidelines?• Medicare: MPIM• Medicaid: State requirements
– Were allowed claims included in overpayment sample calculation?
– Were calculations performed correctly in the audit and at each level of appeal?
47
Defending Against Extrapolation Results
• Substantive Challenges
– Need a statistician to make your case.
• Where can you find one?
– “One size does NOT fit all.”
– It is not your job to explain how it should be done.
48
Defending Against Extrapolation Results
• Examples of Substantive Challenges
– Is the sample representative?
– Is the sample statistically significant?
• Is the sample size reliable?
• Is the sample within the required precision and confidence levels?
49
Hold Auditors Accountable• Did they know what should be done statistically?
– Requires consultation with statistical expert
• Did they do what is necessary to create and audit a probability sample (not just say they did)?
• Have to test the data to be sure
• Did they accurately follow chosen methodology key assumptions (accurately execute the methodology, use proper randomization, use the correct formulae and accurately measure variables of interest)?
• All of these criteria must be met for extrapolation
• Did they select a random sample of sufficient size made up of independent observations that are normally distributed, randomly selected and representative?
50
Common Substantive Issues
• Sample size not associated with established precision or confidence levels
• Incorrect use of formulas
• Use of wrong formulas - choose wrong method
• Use of inapplicable methodology – simple, stratified, cluster, multi-stage
• Non-representative sample
• Fail to meet key assumptions of statistics – math basis of statistics
• Exclusion of zero paid claims
• Accuracy outside of recommended range – too little precision
• Reporting incorrect precision and/or confidence levels51
Defending Against Extrapolation Results
• Obtain all documentation related to sampling calculations
– Consider provider’s prior audit history
• Know appeal timelines and requirements for each level
– No new evidence after reconsideration absent good cause
– Request for ALJ Hearing – 42 C.F.R. § 405.1014(a)(3)
• Include information on each sample claim to be appealed
• File request within 60 calendar days after the party receives the last reconsideration for the sample claims
• Assert the reasons for disagreement with how the statistical sample and/or extrapolation was conducted in the request for hearing
52
Defending Against Extrapolation Results
• Understand reasons for denial at each level
• Present reasons in written protest or position paper
• Prepare for oral testimony at ALJ hearing
53
Questions?
54
Faculty
Anna M. Grizzle, Esq.MemberBass, Berry & Sims PLC150 Third Avenue SouthSuite 2800Nashville, TN [email protected](615) 742 7732
Pat Maykuth, Ph.D.PresidentResearch Design Associates, Inc.
721 E Ponce de LeonDecatur, GA 30030www.researchdesignassociates.compm@researchdesignassociates.com(404) 373 4637
55
