The Post-Survey Process

George Mason UniversityCollege of Nursing and Health Science

Regulatory Requirements for Health SystemsSummer 2004

Used with Permission

Scoring and Decision Process-

Executive Summary . . .

EP Scoring are based on a three-point scaleType I and supplemental recommendations are replaced with “requirements for improvement” and supplemental findingsThe surveyor leaves a final report on siteFinal decision comes after acceptance of the evidence of standards compliance (ESC)

Scoring Guidelines

The scoring guidelines divide EP’s into different categories

Category A (EP’s scored yes or no)Category B (EP’s address situations in which the literal intent is met, but there is need to evaluate the quality or comprehensiveness of the effort)Category C (EP’s that address frequency)

How is each category scored?

EP’s are scored on a 3 point scale,with 0 as insufficient compliance, 1 as partial compliance, and 2 as satisfactory compliance“A” EP’s, are scored only 0 or 2 (yes or no), unless there was a track record issue leading to a score of 1(partial compliance)

New Scoring and Decision Process

3-point vs 5-point Element of Performance scoring scaleStandards identified as compliant or not compliantSimplified aggregation process No grid element or summary grid score calculationSummary score based on number of non-compliant standardsStatistically-based thresholds for Conditional and Preliminary Denial of Accreditation (2 and 3 standard deviations above the mean Summary Score)Revised accreditation decision categoriesMeasure-based follow-up (Evidence of Standards Compliance and Measure of Success)No scores shared with HCO

Standard Level Scoring

The EP’s are aggregated to determine standards complianceStandards are either in compliance or not in complianceThere is no partial compliance at the standard level

Number of not compliant standards

Standard (STD)(2 Point Scale)

Element of Performance (EP)(3 Point Scale)

Overall Program Decision

CURRENT MODEL

Program Follow-up

Summary Grid Score

Grid Element Score

Standard (STD)(5 Point Scale)

Measurable Characteristic (MC)(5 Point Scale)

PREVIOUS MODEL

New Scoring and Decision Process:

Previous Aggregation and Decision Process

Standard Score• Worst Measurable Characteristic score• Measurable Characteristic set score

“If at least 2 MCs are scored 5 or worse, the standard score is 5”

Summary Grid Score• Convert each grid element score into points (0-4) • Add the points for each converted grid element score

• Determine the maximum number of points (# of scored elements x 4)• Divide actual points by maximum points and multiply the result by 100

• Conditional Accreditation = Summary Grid Score < 80• Preliminary Denial of Accreditation = Summary Grid Score < 50

Grid Element Score• Standard scores are weighted through capping• Worst standard score after applying caps

Current Aggregation and Decision Process

Count of the standards scored 0 (not compliant)

• Conditional Accreditation = Summary Score between 2 and 3 Standard Deviations above the mean.

• Preliminary Denial of Accreditation = Summary Score greater than 3 Standard Deviations above the mean.

Standard Score• Score each Element of Performance

(0 =Insufficient Compliance, 1 = Partial Compliance, 2 = Satisfactory Compliance)• Standard score = 0 (not compliant) if any one EP is scored 0 (insufficient

compliance) or a predetermined percent of EP’s are scored 1 (partial compliance).

What kind of follow-up is there?

The HCO has 90 days* after survey for the first 18 months (January 2004 through June 2005) to send JCAHO “evidence of standards compliance” (ESC) – i.e., what it has done to come into compliance with the standard, or what evidence proves it was in compliance at the time of the on-site surveyAt this time the HCO submits an indicator or measure of success that they will use to assess sustained compliance over time, as applicableFour months after approval of the ESC, the HCO submits data on their measure of success to demonstrate track record – in all cases this is an audit process

*Note: HCO’s surveyed after July 1, 2005 will be allowed 45 days for ESC submission; this timeframe is subject to revision by Accreditation Committee after review and analysis of actual submission data

Remember, a Measure of Success (MOS) is . . .

A numerical or other quantitative measure usually related to an audit that validates that an action was effective and sustainedSubmitted via the extranetSubmitted on an electronic form with space limited to a brief indication of the numerical measure – often just a numerator and denominator with definitions of each.

What about revisions?

There is no longer any need for revisions…During the 90 days* after the on-site survey, the HCO can send information to JCAHO to demonstrate what they have done to come into compliance (corrective evidence), OR, to demonstrate that they were in compliance at the time of survey (clarifying evidence)During this time period, the HCO maintains its current accreditation status

*Note: HCO’s surveyed after July 1, 2005 will be allowed 45 days for ESC submission; this timeframe is subject to revision by Accreditation Committee after review and analysis of actual submission data

The final accreditation decision

Is made after the JCAHO receives and approves the HCO’s evidence of standards compliance (ESC) and identified measure of successIf an acceptable ESC is received, then the HCO will receive an “Accredited” decision

If no ESC submission within the required timeframe . . .

After 90 days*, the HCO will receive a “Provisional Accreditation” decisionThis decision will be disclosable

*Note: HCO’s surveyed after July 1, 2005 will be allowed 45 days for ESC submission; this timeframe is subject to revision by Accreditation Committee after review and analysis of actual submission data

Aligning the decision and reporting process

The revised process differentiates between accreditation categories rather than within a single categoryThe revised decision process emphasizes compliance with all standards all the time, and continuous improvement in key safety and quality areas

Accreditation Decisions

Published category of “Accredited with Requirements for Improvement” is eliminatedNew decision rules for Conditional and Preliminary Denial of Accreditation (PDA) decisions

Final Report Format

Standards Out of ComplianceThe Primary Critical Focus Area XXX may be vulnerable as evidenced by:

Standard Standard TextProgramRecommendationElement of PerformanceSecondary Critical Focus Area (as appropriate)

Post-Survey Sequence

PreviousPreliminary left on siteFinal mailed within 35 daysRevision processWritten progress reports (1, 4 & 6 month)2nd generation failures (can result in CON)

CurrentFinal Report of findings left on siteReport on extranet within 48 hoursNo revisionsESC in 90 days during the first 18 monthsFour months after an approved ESC, HCO submits a Measure of Success (MOS)

Quality Reports . . .

Will replace performance reportsWill include new information relative to quality and safety

National Patient Safety Goal PerformanceNational quality improvement goal performance/ ORYX core measuresOptional disease specific care or other certificationsSpecial recognitions/achievements (e.g., Codman award winner, Magnet Hospital)

Public Disclosure

Performance ReportDecisionOverall ScoreSummary Data ComparisonsRecommendations for Improvement on all surveys

Quality ReportDecisionNo ScoreRecommendations for Improvement for provisional, conditional, PDA and DA decisionsNational Patient Safety GoalsNational Quality GoalsQuality Distinctions

Certifications, Awards

Shared Visions—New Pathways: Triennial Accreditation Cycle

Example of HCO Surveyed in July 2002

-6 0 1 15 18 19 30 33 35.5 36 39 40 41 44

July 2002

Full Survey

JCAHO gives HCO extranet access to Periodic Performance Review and Priority Focus Process output

Organization completes Periodic Performance Review; Organization identifies areas of non-compliance and develops corrective action plan

Organization returns Periodic Performance Review and corrective action plans with definition of MOS

Organization completes extranet Application for Accreditation– made available 6-9 months prior to survey due date

Standards Interpretation Group conducts phone interview with HCO; reviews and approves corrective action plan

JCAHO runs Priority Focus Process and sends output to organization

PFP output is delivered to surveyor with itinerary for review prior to survey

July 2005

Full Survey

On-site Survey

- Tracer Methodology

- Systems Tracers on key issues

- Validation of implementation of corrective action plan from Periodic Performance Review

- Final report left on site

Organization submits Evidence of Standards Compliance and Measures of Success (if recommendations during on-site survey)

Month:

Organization’s accreditation status is not impacted if corrective action plan is approved

Organization completes extranet Application for Accreditation

Survey scheduled

Onsite Survey scheduled

Organizations submit quarterly core measure data (as core measures are implemented across accreditation programs)

Quality Report posted on the web

Decision rendered

Organization submits data for MOS to JCAHO