View
219
Download
0
Category
Tags:
Preview:
Citation preview
• Provider X is identified for billing excessive ambulance services. A decision was made to pull all his/her ambulance services for a specified two years period.
Results: - 3,000 claims, 7,000 lines and $1.8M in payments.
Scenario
What is Sampling?
• Sampling - is the selection of observations to acquire some knowledge of a statistical universe (population).
• From the characteristics of samples, we can infer the characteristics of universes, if the sample is representative of the universe.
• In order for statistics to be good estimates of parameters, they must, on average, return the value of the universe parameter
• When the expected value of a statistic equals a universe parameter, we call the statistic an unbiased estimator of that universe parameter
How Do I Get a Representative Sample?
• How do you ensure that your statistic is an unbiased estimator?
How Do I Get a Representative Sample?
RANDOMIZATION!!!
• A sample that is randomly selected from a universe yields sample statistics that are unbiased estimates of the universe parameters
• Many software packages, such as SAS and RAT-STATS have a valid random number generator
Randomization
• Another idea behind random sampling is that each sampling unit has a known probability of being selected
Probability Sampling
• Universe : An event or things of interest that the researcher wishes to investigate.
Eg. All Medicare beneficiaries that received a left heart catheterization from Dr. John Doe between January 1, 2007 and June 30, 2008 paid up to September 30, 2008.
Sampling Terms
• Samples are usually drawn by taking a subset of sampling units from the total universe
• Sampling units are non-overlapping collection of elements from the universe that cover the entire universe (eg claims, beneficiaries)
Sampling Terms
• We can infer the values of the universe from the sample by the use of estimation
• Ideally, we would like gather information from the sample and then estimate that value for the entire universe
• These estimates calculated from the sample data are called statistics
Estimation
• In an simple random sample where we had sampled 100 units out of 1000, suppose we had a $5,000 total overpayment from the sample
• The Mean Total Overpayment would then be:
Estimation
000,50$100
50001000
top yN
Why Should I Care?
• HCFA Ruling 86-1 allows the use of statistical sampling for the purpose of estimating a provider’s overpayment to the Medicare trust fund
• Thus, we can use sampling to estimate overpayments to providers and avoid having to review the entire universe!
• CMS guidelines for Statistical Sampling for Overpayment Estimation
• Program Integrity Manual Section 3.10
• Some of the issues addressed are:– Methodologies– Sample Size– Estimation techniques
CMS Sampling Guidelines
• This replaces and clarifies (for older cases) the old HCFA Sampling Guidelines Appendix (CR 1363)– “This program memorandum (PM) provides clarified
guidance and direction for Medicare carriers to use when conducting statistical sampling for overpayment estimation. The attached replaces the prior Sampling Guidelines Appendix for reviews conducted after issuance of this PM. For reviews conducted prior to this issuance, the attached are a clarification to aid interpretation of the earlier instructions, particularly where specific numbers are suggested”
Sampling Guidelines
• Simple Random Sampling
• Cluster Sampling
• Stratified Sampling
• Other Methodologies
Sampling Methodologies
• This is the most straightforward method of sampling
• X number of sampling units are randomly selected from Y total sampling units in the Universe
• Each sampling unit has an equal probability of being selected
Simple Random Sampling
• A cluster sample is a probability sample in which each sampling unit is a collection, or cluster, of elements
• A good example is the random selection of beneficiaries, then selecting all relevant claims from each beneficiary
Cluster Sampling
• A stratified random sample is one obtained by separating the universe elements into non-overlapping groups, called strata, and then selecting a simple random sample from each stratum
• An example of this would be samples involving multiple procedure codes, selecting simple random samples from each code
• Stratified random sampling generally has less sampling variability that other sampling designs
Stratified Random Sampling
Stratified Random Sampling – Proportional Allocation Example
Universe = 1000 Units
99211 99212 992159921499213
50 150 50300 450
Sample = 100 Units
99211 99212 99213 99214 99215
5 15 30 45 5
• PIM 3.10 states about sample sizes:– “It is neither possible nor desirable to specify a minimum sample
size that applies to all situations”– “…real-world economic constraints must be taken into account.
As stated earlier, sampling is used when it is not administratively feasible to review every sampling unit in the target universe. In practice, sample sizes may be determined by available resources. That does not mean, however, that the resulting estimate of overpayment is not valid as long as proper procedures for the execution of probability sampling have been followed. A challenge to the validity of the sample that is sometimes made is that the particular sample size is too small to yield meaningful results. Such a challenge is without merit as it fails to take into account all of the other factors that are involved in the sample design”
PIM 3.10 – Sample Sizes
• CSA procedure:– If we can, we like to pull at least 10% of the
universe, however, this is not a rule that is set in stone
– We must, at a minimum, pull at least 30 sampling units to satisfy distribution requirements through the central limit theorem
PIM 3.10 – Sample Sizes
• PIM 3.10 also states:– “In most situations the lower limit of a one-
sided 90 percent confidence interval should be used as the amount of overpayment to be demanded for recovery from the physician or supplier. The details of the calculation of this lower limit involve subtracting some multiple of the estimated standard error from the point estimate, thus yielding a lower figure.”
PIM 3.10 – Overpayment
• It further states that:– “This procedure, which, through confidence interval estimation,
incorporates the uncertainty inherent in the sample design, is a conservative method that works to the financial advantage of the physician or supplier. That is, it yields a demand amount for recovery that is very likely less than the true amount of overpayment, and it allows a reasonable recovery without requiring the tight precision that might be needed to support a demand for the point estimate. However, you are not precluded from demanding the point estimate where high precision has been achieved.”
PIM 3.10 – Overpayment
• What we really do then is calculate the Mean Total Overpayment and subtract a multiple of the standard error from it to achieve the lower level of the confidence interval
PIM 3.10 – Overpayment
• Below is the formula for the total variance for cluster sampling
PIM 3.10 – Overpayment
11
2
2
n
yy
nN
nNNV
n
iti
op
• Look at how the overpayments work:
PIM 3.10 – Overpayment
OP w/ Small Variance (Large n) OP w/ Large Variance (Small n)
Mean Total Overpayment
Mean Total Overpayment
90% Upper Limit
90% Lower Limit
90% Upper Limit
90% Lower Limit
$$ O
verp
aym
ent
Sample Size Comparison
Analysis Variable : CLUSTAMT
N Mean Std Dev Sum Minimum Maximum
20 17,462.81 30,551.07 34,9256.24 0.00 81,603.20
10 17,462.81 31,388.24 17,4628.12 0.00 81,603.20
5 18,323.58 35,639.19 91,617.92 0.00 81,603.20
Estimation Of Total Amount Of
Refund & It's Lower 1-sided 90% C.I.
Difference if sample Size 5-10 beneficiary: $184,569.45 Difference in sample Size 10-20 beneficiary: $207,578.67
Sample Size
Univ. Size
Mean Total Overpayment
Std. Error
90% 1-sided Lower Bound
20 44 $768,363.73221,995.
08$483,766.04
10 44 $768,363.73383,912.
92$276,187.37
5 44 $806,237.70660,329.
49$91,617.92
Bottom Line
• Large Sample Sizes– Use when the expected
overpayment is large
– Use in high profile cases
– Resource intensive
– Increase precision even more using stratified sampling plans
• Small Sample Sizes– Use when the expected
overpayment is small
– Use in routine, low $ cases
– Not as resource intensive
– Does not work as well for stratified sampling
Sub-samples
• It is often beneficial to evaluate a sub-sample before moving to a full statistical sample. (sample size of about 30)
• Get a good idea of the point estimate (Mean Total Overpayment).
• Sampling for Consent Settlements.
Summary for Sampling
• Define the Universe
• Determine the sampling methodology
• Create the sampling Frame
• Determine sample size
• Create your sample
After Sampling review is completed
• Perform overpayment Projection
Recommended