Upload
aiden-barr
View
219
Download
0
Tags:
Embed Size (px)
Citation preview
Randomized Controlled Trials: Randomized Controlled Trials: Allocation ConcealmentAllocation Concealment
Kenneth F. Schulz, PhDKenneth F. Schulz, PhD
Fixation on Sequence GenerationFixation on Sequence Generation
Traditionally, many medical researchers Traditionally, many medical researchers mistakenly consider the sequence mistakenly consider the sequence generation process as “randomization”generation process as “randomization”
They frequently ignore allocation They frequently ignore allocation concealment concealment
Without adequate allocation concealment, Without adequate allocation concealment, proper random sequences can be subvertedproper random sequences can be subverted
For example, a properly generated sequence For example, a properly generated sequence posted on a bulletin board posted on a bulletin board
Allocation ConcealmentAllocation Concealment
Prevents selection bias by Prevents selection bias by concealing the allocation sequence concealing the allocation sequence from those assigning participants to from those assigning participants to intervention groupsintervention groups
Until the moment of assignmentUntil the moment of assignment
Mechanisms for Selection BiasMechanisms for Selection Biasin RCTsin RCTs
If those responsible for entering If those responsible for entering participants know, or can detect, the participants know, or can detect, the upcoming treatment allocations:upcoming treatment allocations: Decide entrance based on that knowledgeDecide entrance based on that knowledge
Channel participants with a better prognosis Channel participants with a better prognosis to the experimental group and those with a to the experimental group and those with a poorer prognosis to the control group, or vice poorer prognosis to the control group, or vice versaversa
In any case, they introduce bias into the In any case, they introduce bias into the treatment comparisontreatment comparison
Do Not Confuse Allocation Do Not Confuse Allocation ConcealmentConcealment with with BlindingBlinding
Allocation concealment seeks to prevent Allocation concealment seeks to prevent selection bias, protects assignment selection bias, protects assignment sequence before and until allocation, and sequence before and until allocation, and can always be successfully implementedcan always be successfully implemented
In contrast, blinding seeks to prevent In contrast, blinding seeks to prevent ascertainment bias, protects sequence ascertainment bias, protects sequence after allocation, and cannot always be after allocation, and cannot always be successfully implementedsuccessfully implemented
Importance of Allocation Importance of Allocation ConcealmentConcealment
Unclearly concealed and inadequately Unclearly concealed and inadequately concealed trials, compared to concealed trials, compared to adequately concealed trials, adequately concealed trials, exaggeratedexaggerated the estimates of an the estimates of an interventionintervention’’s effectiveness by s effectiveness by 30% to 30% to 40%,40%, on average on average
Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273:408-412.
Similar Results on Allocation Similar Results on Allocation Concealment in a Separate StudyConcealment in a Separate Study
Replicated our methods in different Replicated our methods in different subject areas (digestive & circulatory subject areas (digestive & circulatory diseases and mental health) diseases and mental health)
Examined 11 meta-analysesExamined 11 meta-analyses
Inadequately concealed trials Inadequately concealed trials exaggeratedexaggerated estimates of effectiveness by estimates of effectiveness by 37%37%
[Moher D, et al. Does quality of reports of randomised trials [Moher D, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-affect estimates of intervention efficacy reported in meta-analyses? analyses? LancetLancet 1998; 352: 609-13.] 1998; 352: 609-13.]
Allocation Concealment: SummaryAllocation Concealment: Summary
Adequate allocation concealment emerges Adequate allocation concealment emerges from our analyses as crucial to reducing from our analyses as crucial to reducing biasbias
Without it, the whole point of Without it, the whole point of randomisation vanishes and bias may randomisation vanishes and bias may distort resultsdistort results
Our results support Mosteller and his Our results support Mosteller and his colleaguescolleagues
“When the randomization leaks, the trial’s guarantee of lack of bias runs down the drain”
“When the randomization leaks, the trial’s guarantee of lack of bias runs down the drain”
Method of allocation of treatment in 208 controlled trials in head injury
Dickinson K, et al., BMJ 2000;320:1308-1311.
Method of allocation No. of trials
Method of allocation No. of trialsAdequateAdequate
Centralised randomisation by telephone 1
Numbered/coded identical containers administered sequentially 11
Randomisation scheme controlled by pharmacy 8
Sequentially numbered, sealed, opaque envelopes 2
Not adequateNot adequate
Other 18
Date of birth 1
Day of week 3
Alternation 3
Not stated 161
Actual allocation method with the Actual allocation method with the method just specified as “random”method just specified as “random”
Difficult (Peto and I. Chalmers experience)Difficult (Peto and I. Chalmers experience)
MA of OCsMA of OCs
David G. knew the PI who put him in David G. knew the PI who put him in contact with the statisticiancontact with the statistician
Described a “great” method . . . Post-it Described a “great” method . . . Post-it notes!notes!
Seemed indignant . . . We asked for Seemed indignant . . . We asked for clarificationclarification
Perfect also for stratification—different colors!!Perfect also for stratification—different colors!!
Personal Accounts of Deciphering Personal Accounts of Deciphering Assignment SequencesAssignment Sequences
Conducted over 20 epidemiological Conducted over 20 epidemiological workshops for medical residents and workshops for medical residents and medical junior facultymedical junior faculty
Each included 20-25 participantsEach included 20-25 participants
Asked how many of the participants had Asked how many of the participants had deciphered, or had witnessed someone deciphered, or had witnessed someone else decipher, an assignment sequenceelse decipher, an assignment sequence
With assurance of anonymity, more With assurance of anonymity, more than half related at least one than half related at least one instance of decipheringinstance of deciphering
Not half of all trialsNot half of all trials No accurate denominatorNo accurate denominator
Nevertheless, not a rare occurrenceNevertheless, not a rare occurrence
Personal Accounts of Deciphering Personal Accounts of Deciphering Assignment Sequences Assignment Sequences (cont.)(cont.)
Decipherings Run the Gamut Decipherings Run the Gamut From Simple to IntricateFrom Simple to Intricate
Simple the most frequent and usually Simple the most frequent and usually take advantage of inadequate take advantage of inadequate allocation concealment schemes:allocation concealment schemes: Posting sequence on a bulletin boardPosting sequence on a bulletin board Opening unsealed envelopesOpening unsealed envelopes Translucent envelopes to a light bulbTranslucent envelopes to a light bulb Opening many envelopes that were not Opening many envelopes that were not
sequentially numberedsequentially numbered
Needed to circumvent more adequate Needed to circumvent more adequate allocation schemesallocation schemes With sequentially numbered drug With sequentially numbered drug
containerscontainers Based on appearance of tablets in Based on appearance of tablets in
unsealed containersunsealed containers Appearance of the labelAppearance of the label
With central allocation, obtaining the With central allocation, obtaining the next few allocations at oncenext few allocations at once
Envelopes to a “hot light” in radiologyEnvelopes to a “hot light” in radiology
Intricate Decipherings Less FrequentIntricate Decipherings Less Frequent
Attempted to Decipher a Numbered Attempted to Decipher a Numbered Container SchemeContainer Scheme
Gave upGave up
Found attending physician rifling Found attending physician rifling the P.I.’s files for the assignment the P.I.’s files for the assignment sequencesequence
Horrified? No, impressed with his Horrified? No, impressed with his brilliance and proceeded to helpbrilliance and proceeded to help
Should be kept in a locked locationShould be kept in a locked location
Ignorance Is Not BlissIgnorance Is Not Bliss
““At the annual meeting of the Society for At the annual meeting of the Society for Clinical Trials, held in Houston, … Schulz Clinical Trials, held in Houston, … Schulz reported data showing that the experts’ fears reported data showing that the experts’ fears about ignorance are justified. Schulz about ignorance are justified. Schulz examined 250 reports of clinical trials and examined 250 reports of clinical trials and came up with an intriguing finding…”came up with an intriguing finding…”
““At the annual meeting of the Society for At the annual meeting of the Society for Clinical Trials, held in Houston, … Schulz Clinical Trials, held in Houston, … Schulz reported data showing that the experts’ fears reported data showing that the experts’ fears about ignorance are justified. Schulz about ignorance are justified. Schulz examined 250 reports of clinical trials and examined 250 reports of clinical trials and came up with an intriguing finding…”came up with an intriguing finding…”
[Rachel Nowak. Problems in clinical trials go far beyond misconduct. [Rachel Nowak. Problems in clinical trials go far beyond misconduct. Science 1994; 264: 1539.]Science 1994; 264: 1539.]
Ignorance Is Not BlissIgnorance Is Not Bliss
“Most physicians are not trained in basic scientific principles, let alone clinical trials”
“Most physicians are not trained in basic scientific principles, let alone clinical trials”
““I fantasize that in the future (training) I fantasize that in the future (training) will become a requirement for running will become a requirement for running clinical trials”clinical trials”
““I fantasize that in the future (training) I fantasize that in the future (training) will become a requirement for running will become a requirement for running clinical trials”clinical trials”
John Gallin, Director of NIH’s Clinical CenterJohn Gallin, Director of NIH’s Clinical Center
Stephen George, Duke University, Chair of the Statistics Stephen George, Duke University, Chair of the Statistics Committee for NCI, NIH Cancer Clinical Cooperative GroupsCommittee for NCI, NIH Cancer Clinical Cooperative Groups
Rigorous Trials Annoy HumansRigorous Trials Annoy Humans
InvestigatorsInvestigators Certain Ps to benefitCertain Ps to benefit May want the results of study to May want the results of study to
reveal the “truth”reveal the “truth”
Proper trial procedures attempt Proper trial procedures attempt to impede human inclinationsto impede human inclinations Hence, aspects of properly Hence, aspects of properly
conducted RCTs annoy conducted RCTs annoy investigatorsinvestigators
Without Methodological RigourWithout Methodological Rigour
The challenge of deciphering may frequently The challenge of deciphering may frequently become too great a temptation to resistbecome too great a temptation to resist Even without intent to biasEven without intent to bias
““The only way to get rid of a temptation is to The only way to get rid of a temptation is to yield to it.” yield to it.” Oscar WildeOscar Wilde
Deciphering may just reflect human Deciphering may just reflect human inquisitiveness and ingenuityinquisitiveness and ingenuity
Does not necessarily indicate scientific Does not necessarily indicate scientific malevolencemalevolence
Without Methodological RigourWithout Methodological Rigour
Or deciphering may reflect deliberate Or deciphering may reflect deliberate acts to alter findingsacts to alter findings
Whatever the motivations, innocent Whatever the motivations, innocent or naïve or deliberate, those actions or naïve or deliberate, those actions undermine the validity of the trialundermine the validity of the trial
Investigators must Investigators must devote diligent, devote diligent, persistent attentionpersistent attention to randomization to randomization
Minimal Standards: Common Minimal Standards: Common Allocation Concealment ApproachesAllocation Concealment Approaches
Sequentially numbered, Sequentially numbered, opaque, sealed envelopes opaque, sealed envelopes (SNOSE)(SNOSE)
Pharmacy controlPharmacy control
Numbered or coded containersNumbered or coded containers
Central randomizationCentral randomization
Realistically, these standards Realistically, these standards should be should be exceededexceeded
Envelopes for Allocation Envelopes for Allocation ConcealmentConcealment
More susceptible to More susceptible to manipulation through human manipulation through human ingenuityingenuity
Less than ideal method of Less than ideal method of allocation concealmentallocation concealment
If used, investigators must If used, investigators must diligently develop and monitor diligently develop and monitor the processthe process
Envelopes Envelopes (Cont.)(Cont.)
onlyonly after the P’s after the P’s name and other name and other details are written on details are written on the appropriate the appropriate envelopeenvelope
SNOSESNOSE
Ensure that the envelopes are opened Ensure that the envelopes are opened sequentiallysequentially
Envelopes Envelopes (Cont’d)(Cont’d)
Pressure-sensitive or carbon Pressure-sensitive or carbon paper insidepaper inside Supports enrolment Supports enrolment Encourages orderly openingEncourages orderly opening Promotes proper assignmentPromotes proper assignment Creates valuable audit trailCreates valuable audit trail
Cardboard or aluminum foil Cardboard or aluminum foil inside envelopeinside envelope
““Pharmacy Controlled”Pharmacy Controlled”
Compliance with randomization/allocation Compliance with randomization/allocation concealment methods questionableconcealment methods questionable
Aware of gross distortionsAware of gross distortionsPharmacy ran out of one drug . . .Pharmacy ran out of one drug . . .Alternate assignmentAlternate assignment
Should not assume pharmacists knowledgeShould not assume pharmacists knowledge
Ensure they, and all research partners, Ensure they, and all research partners, follow proper trial proceduresfollow proper trial procedures
““Numbered Containers”Numbered Containers”
Excellent – prevents foreknowledgeExcellent – prevents foreknowledgeMust take proper precautionsMust take proper precautions
Appearance – exactly the sameAppearance – exactly the same
Weight – exactly the sameWeight – exactly the same
Sound when shaken – exactly the sameSound when shaken – exactly the same
Lettering – exactly the sameLettering – exactly the same
Odor – exactly the sameOdor – exactly the same
SealedSealedDifference between concealment and blindingDifference between concealment and blinding
““Central Randomization”Central Randomization”
Frequently telephoneFrequently telephoneFaxFaxEmailEmailInternetInternet““Computer”Computer”
Assignments only provided after Assignments only provided after participants irrevocably enrolled participants irrevocably enrolled
Must check procedures for leaksMust check procedures for leaks Monitor process for adherenceMonitor process for adherence
RCTs: Anathema to the Human SpiritRCTs: Anathema to the Human Spirit
Must acknowledge the Must acknowledge the vagaries of human naturevagaries of human nature
Must establish methodological Must establish methodological safeguards that thwart safeguards that thwart attempts to contaminate trials attempts to contaminate trials with biaswith bias
Subversions of SequencesSubversions of Sequences
Conscious or subconscious?Conscious or subconscious?
Direct or compensatory?Direct or compensatory?
Doesn’t matterDoesn’t matter Biases the trialBiases the trial
ParticipantsParticipants
OutcomeOutcome OutcomeOutcome
PlaceboPlacebo
40% Non-40% Non-compliancecompliance
60% 60% ComplianceCompliance
Group representing the Group representing the policy of no treatmentpolicy of no treatment
25% Non-25% Non-compliancecompliance
75% 75% ComplianceCompliance
Group representing the Group representing the policy of oral treatmentpolicy of oral treatment
New Oral DrugNew Oral Drug
Randomize
RCT Compared the Effectiveness of Clofibrate in RCT Compared the Effectiveness of Clofibrate in Preventing Cardiac Deaths in Men Who Had Preventing Cardiac Deaths in Men Who Had
Survived a Myocardial InfarctionSurvived a Myocardial Infarction
ClofibrateClofibrate PlaceboPlacebo
5 Year mortality5 Year mortality 20.2%20.2% 20.9%20.9% (p = .55)(p = .55)
Authors state that:Authors state that:
• One can justify almost any conclusion, dependent upon the analysis One can justify almost any conclusion, dependent upon the analysis chosenchosen
• Manipulating deviates leads to severe biasManipulating deviates leads to severe bias
• Can you ever do so?Can you ever do so?
Eliminating deviates from Eliminating deviates from clofibrateclofibrate
(80% adherence)(80% adherence)15.0%15.0% 20.9%20.9% (p < .05)(p < .05)
Eliminating deviates from Eliminating deviates from both groupsboth groups 15.0%15.0% 15.1%15.1%
Exclusions After Exclusions After RandomizationRandomization
Can introduce bias and should Can introduce bias and should be carefully scrutinizedbe carefully scrutinized
All randomized patients should All randomized patients should be analyzed, and analyzed as be analyzed, and analyzed as part of the group to which they part of the group to which they were initially assignedwere initially assigned
ITT (Intention-to-treat)ITT (Intention-to-treat)
Exclusions of LFU Damage Exclusions of LFU Damage Internal ValidityInternal Validity
Without outcomes from those lost to Without outcomes from those lost to follow-up, investigators have little choice follow-up, investigators have little choice but to exclude them from the analysis but to exclude them from the analysis
Any losses damage internal validityAny losses damage internal validity
However, differential rates of loss among However, differential rates of loss among comparison groups cause major damage comparison groups cause major damage
Investigators must minimize their losses Investigators must minimize their losses to follow-upto follow-up
Retention of Trial ParticipantsRetention of Trial Participants
Minimizing losses exudes difficultiesMinimizing losses exudes difficulties Eliminating losses may be impossible Eliminating losses may be impossible
But investigators too frequently profess But investigators too frequently profess insurmountable difficultiesinsurmountable difficulties
Most should work harder to obtain Most should work harder to obtain higher follow-up rateshigher follow-up rates
Investigators must commit adequate Investigators must commit adequate attention and resources to develop and attention and resources to develop and implement procedures to minimize implement procedures to minimize losseslosses
Innovative Twists that Cultivate Innovative Twists that Cultivate High Follow-up RatesHigh Follow-up Rates
Establish many conveniently placed FU Establish many conveniently placed FU facilities facilities
Too often investigators expect participants to Too often investigators expect participants to visit a single, inconvenient locationvisit a single, inconvenient location
Shortening data collection form caters to the Shortening data collection form caters to the participants’ wishesparticipants’ wishes
Long sessions lead them to vote with their feetLong sessions lead them to vote with their feetBe considerateBe considerateFoster follow-up by not overburdening Foster follow-up by not overburdening Also may engender higher quality data on theAlso may engender higher quality data on themain itemsmain items
ENDEND
Meta-analysis of polyglycolic acid versus Meta-analysis of polyglycolic acid versus catgut for perineal repair on short term paincatgut for perineal repair on short term pain
1 adequately concealed trial … 1 adequately concealed trial … OR = .89OR = .894 unclearly concealed trials … OR = .444 unclearly concealed trials … OR = .44
Protective effect of polyglycotic acid was 51% lower in the unclearly concealed trialProtective effect of polyglycotic acid was 51% lower in the unclearly concealed trial
(ROR = .49; 95% CI of .35 - .69)(ROR = .49; 95% CI of .35 - .69)
Meta-analysis of PolyglycolicMeta-analysis of Polyglycolic
Meta-analysis of corticosteroids vs. none, after Meta-analysis of corticosteroids vs. none, after preterm rupture of membranes, on respiratory preterm rupture of membranes, on respiratory distress syndrome (RDS)distress syndrome (RDS)
3 adequately concealed trials … OR = 0.723 adequately concealed trials … OR = 0.724 unclearly concealed trials … OR = 0.534 unclearly concealed trials … OR = 0.53
Protective effect of corticosteroids was exaggerated by 27% in the unclearly concealed trials
Protective effect of corticosteroids was exaggerated by 27% in the unclearly concealed trials
ROR = 0.73; 95% CI of 0.35 - 1.50ROR = 0.73; 95% CI of 0.35 - 1.50
Meta-analysis of CorticosteriodsMeta-analysis of Corticosteriods
Randomization Process (P 138B)Randomization Process (P 138B)
In the 4 OB/GYN Journals, found the 2 treatment group sizes to In the 4 OB/GYN Journals, found the 2 treatment group sizes to be much too similarbe much too similar
ExpectedExpected ObservedObserved
5 ( 5%)5 ( 5%) 00
48 (50%)48 (50%) 88
Proper approach to: 4 OB/GYN Journals
4 General Medical Journals
Method of generation of random numbers
32% 49%
Randomization concealment
23% 26%
Both 9% 15%
Separation of Executer and Separation of Executer and GeneratorGenerator
Person(s) who prepare the randomization Person(s) who prepare the randomization scheme should not determine eligibility, scheme should not determine eligibility, administer treatment, or assess outcomeadminister treatment, or assess outcome
Neglected critical elementNeglected critical element
Obviously important—regardless of Obviously important—regardless of methodological quality, access to sequence methodological quality, access to sequence and thus opportunity to introduce biasand thus opportunity to introduce bias
Faults in this element may be the crack Faults in this element may be the crack though which much of the bias seeps into though which much of the bias seeps into controlled trialscontrolled trials
““Not quite as random as I pretended”Not quite as random as I pretended”[Martyn C. Not quite as random as I pretended. Lancet [Martyn C. Not quite as random as I pretended. Lancet
1996; 347:70.]1996; 347:70.]
“Bias in the way treatments are assigned can be a more powerful determinant of the outcome of the trial than the treatments that are being investigated.”
“Bias in the way treatments are assigned can be a more powerful determinant of the outcome of the trial than the treatments that are being investigated.”
““Trialists … often find it hard to prevent Trialists … often find it hard to prevent their hopes that a new treatment might their hopes that a new treatment might prove to be beneficial from evolving intoprove to be beneficial from evolving intoa belief that it really is so.”a belief that it really is so.”
““Trialists … often find it hard to prevent Trialists … often find it hard to prevent their hopes that a new treatment might their hopes that a new treatment might prove to be beneficial from evolving intoprove to be beneficial from evolving intoa belief that it really is so.”a belief that it really is so.”
RCT in NorwayRCT in Norway
Examined the effect of Chlorhexidine Examined the effect of Chlorhexidine compared to placebo on postpartum compared to placebo on postpartum infection and Strep B infections in infection and Strep B infections in neonatesneonates
Reported as a randomized controlled Reported as a randomized controlled trial in the published articletrial in the published article
Insert published article
RCT in NorwayRCT in Norway
Midwives knew on which shelf the Midwives knew on which shelf the Chlorhexidine and placebo were kept in Chlorhexidine and placebo were kept in the cabinet and just proceeded to the cabinet and just proceeded to administer whatever they deemed administer whatever they deemed appropriateappropriate
Even if they forgot the shelf, they could Even if they forgot the shelf, they could smell the differencesmell the difference
No monitoring of the allocation processNo monitoring of the allocation process
Letter from a Physical TherapistLetter from a Physical Therapist
A A “friend of mine”“friend of mine” conducted an RCT conducted an RCT comparing 2 training regimens following comparing 2 training regimens following surgical repair of the knee for anterior surgical repair of the knee for anterior cruciate ligament damagecruciate ligament damage ““randomization method used was alternating randomization method used was alternating
assignment”assignment”
Referred to one patient as Referred to one patient as “difficult”“difficult” and and asked the friend to which group she was asked the friend to which group she was assignedassigned Friend responded Friend responded “that she was in the group “that she was in the group
that required slightly less contact and that her that required slightly less contact and that her inclusion was by his choice”inclusion was by his choice”
““Not so blind, after all”Not so blind, after all”
Sometimes subverting can work against a Sometimes subverting can work against a treatment’s apparent effectivenesstreatment’s apparent effectiveness
[Wallich P. Not so blind after all. Scientific American; May 1996.][Wallich P. Not so blind after all. Scientific American; May 1996.]
“Medical staff … may try to help their sickest patients by sneaking them into the treatment group instead of the control group.”
“Medical staff … may try to help their sickest patients by sneaking them into the treatment group instead of the control group.”
““The drug would then have to be significantly The drug would then have to be significantly better than conventional treatment just to appear better than conventional treatment just to appear equal in efficacy”equal in efficacy”
““The drug would then have to be significantly The drug would then have to be significantly better than conventional treatment just to appear better than conventional treatment just to appear equal in efficacy”equal in efficacy”
RCTs: Anathema to the RCTs: Anathema to the Human SpiritHuman Spirit
Scientific need to obtain unbiased data Scientific need to obtain unbiased data from an inherently biased source – from an inherently biased source – human beingshuman beings
Many trialists are unaware of the Many trialists are unaware of the rationale behind RCTs rationale behind RCTs
And the need for trial rigorAnd the need for trial rigor
Some may intellectually grasp the need, Some may intellectually grasp the need, but once immersed in a trial, have many but once immersed in a trial, have many contradictory interests (e.g. episiotomy contradictory interests (e.g. episiotomy trial in Canada)trial in Canada)
Anathema: A Psychologist’s ViewAnathema: A Psychologist’s View
””Fundamental conflict” between scientific Fundamental conflict” between scientific and human imperatives … “Many and human imperatives … “Many probably think they already know what probably think they already know what treatment is best.”treatment is best.” Geoffrey Scobie, psychologist at the Geoffrey Scobie, psychologist at the
University of Glasgow; Doctors cheat University of Glasgow; Doctors cheat because they know what’s bestbecause they know what’s best
New Scientist, 16 December 95, p. 10New Scientist, 16 December 95, p. 10
Of course, what we think we ‘know’ is Of course, what we think we ‘know’ is frequently wrongfrequently wrong
Reporting of Exclusions, Double-blinding, and Schedule Reporting of Exclusions, Double-blinding, and Schedule Generation Related to the Level of Allocation Concealment Generation Related to the Level of Allocation Concealment for 250 Trialsfor 250 Trials
Authors Authors
ReportedReported
Adequately Adequately ConcealedConcealed
(n=79)(n=79)
Unclearly Unclearly ConcealedConcealed
(n=150)(n=150)
Inadequately Inadequately ConcealedConcealed
(n=21)(n=21)
No ExclusionsNo Exclusions(Apparent)(Apparent)
53%53% 67%67% 76%76%
Double-blindingDouble-blinding 73%73% 39%39% 14%14%
Adequate generation Adequate generation of scheduleof schedule
29%29% 15%15% 0%0%