The impact of the STARD checklist on the reporting of studies of diagnostic accuracy in obstetrics and gynecology

literature

Class 1: Ana Costa, André Sena, Duarte Pinto, Inês Brandão, João Silva, Lídia Ana Costa, André Sena, Duarte Pinto, Inês Brandão, João Silva, Lídia

Sousa, Luís Barros, Márcia Marinho, Maria Rocha, Pedro Falcão, Rafael Silva, Sousa, Luís Barros, Márcia Marinho, Maria Rocha, Pedro Falcão, Rafael Silva,

Sara Tapa, Susana Franco.Sara Tapa, Susana Franco.

Supervisors: Altamiro da Costa Pereira, MD, PhD; Cristina Santos, MD, PhD

Introduction to Medicine; Porto Faculty of Medicine 2006

Index

1) Introduction

2) Objective

3) Methods

4) Acknowledgements

• Obstetrics and Gynecology are branches of medicine.(*)

• These areas depend on diagnostic tests.(*)

• Studies of the new diagnostic tests, and their development, are reported in scientific articles. (**)

Introduction

(*) - Cunningham, F.Gary. Williams obstetrics, 26ª edição (2005)

- Berek, Jonathan S. Novak’s gynecology 12ª edição (1996)

(**) - Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and accurate reporting of studios of diagnostic accuracy: the STARD initiative.Annuals of internal Medicine. 2003; 138:40-44

• The reports not always are well made and this may lead to unwanted consequences. (*)

• STARD was created to guideline the reporting of studies of diagnostic accuracy. (*)

Introduction

(*) - Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and accurate reporting of studios of diagnostic accuracy: the STARD initiative.Annuals of internal Medicine. 2003; 138:40-44

To evaluate the impact of the introduction of the STARD checklist on the reporting of diagnostic accuracy, in obstetrics and gynecology literature.

Objective

Articles’ search

Articles’ selection

Selected Articles’ analysis

Statistical analyze of the extracted data

Methods

Query used in MedLine:

(((((((("sensitivity and specificity"[All Fields] OR "sensitivity and specificity/standards"[All Fields]) OR "specificity"[All Fields]) OR "screening"[All Fields]) OR "false positive"[All Fields]) OR "false negative"[All Fields]) OR "accuracy"[All Fields]) OR (((("predictive value"[All Fields] OR "predictive value of tests"[All

Fields]) OR "predictive value of tests/standards"[All Fields]) OR "predictive values"[All Fields]) OR "predictive values of tests"[All Fields])) OR (("reference value"[All Fields] OR "reference values"[All Fields]) OR "reference values/standards"[All Fields]) OR ((((((((((("roc"[All Fields] OR "roc analyses"[All Fields]) OR "roc analysis"[All Fields]) OR "roc and"[All Fields]) OR "roc area"[All Fields]) OR "roc auc"[All Fields]) OR

"roc characteristics"[All Fields]) OR "roc curve"[All Fields]) OR "roc curve method"[All Fields]) OR "roc curves"[All Fields]) OR "roc estimated"[All Fields]) OR "roc evaluation"[All Fields]) OR ("likelihood ratio"[All

Fields])) AND

(hum reprod update[Journal] OR Obstet Gynecol[Journal] OR Menopause [Journal] OR Hum Reprod[Journal] OR Obstet Gynecol Surv[Journal] OR Fertil Steril[Journal] OR Am J Obstet

Gynecol[Journal])

• This query was based on a search strategy in PubMed (MEDLINE) for publications about the evaluation of diagnostic accuracy, suggested by a research article:

Devillé, W. L. et. al., Conducting systematic reviews of diagnostic studies: didactic guidelines

Articles’ search

Methods

Methods

Articles’ search

Methods

Articles’ selection

• Creation of 2 groups (with 2 elements each*):

- one group read and analyse the abstract of each article published in 1998

- the other do the same for the those published in 2005

*Note: each element of the group works independently and in the end, if occurs a disagreement about de inclusion of exclusion of an article the entire article is read. Even so, if no agreement is achieved, an expert e called to make the final decision

• The article is a report of a study of diagnostic accuracy

• The diagnostic test is used in obstetrics and/or genecology

• Was published in 1998 or in 2005

Inclusion criteria:

Methods

Articles’ selection

• The article could not be found in the Internet, the facilities of the

School of Medicine or the local libraries

• The article is written in languages other than English

• The article is a clinical trial, a letter, an editorial, a practice

guideline, a meta-analyse or a review

Exclusion criteria:

Methods

Articles’ selection

STARD checklist Section and topicSection and topic ItemItem

DescribeDescribeValueValue

Article Article scorescore

TITLE/ABSTRACT/TITLE/ABSTRACT/KEYWORDSKEYWORDS 11 The article as a study on diagnostic accuracy (recommend MeSH heading 'sensitivity and specificity')The article as a study on diagnostic accuracy (recommend MeSH heading 'sensitivity and specificity') 11

INTRODUCTIONINTRODUCTION22

The research question(s), such as estimating diagnostic accuracy or comparing accuracy between tests or across The research question(s), such as estimating diagnostic accuracy or comparing accuracy between tests or across participant groupsparticipant groups 11

METHODSMETHODS       11

ParticipantsParticipants 33 The study population: the inclusion and exclusion criteria, setting(s) and location(s) where the data were collectedThe study population: the inclusion and exclusion criteria, setting(s) and location(s) where the data were collected 11

  44

Participant recruitment: was this based on presenting symptoms, results from previous tests, or the fact that the Participant recruitment: was this based on presenting symptoms, results from previous tests, or the fact that the participants had received the index test(s) or the reference standard?participants had received the index test(s) or the reference standard? 11

  55

Participant sampling: was this a consecutive series of patients defined by selection criteria in (3) and (4)? Participant sampling: was this a consecutive series of patients defined by selection criteria in (3) and (4)? If not specify If not specify how patients were further selected.how patients were further selected. 11

  66

Data collection: were the participants identified and data collected before the index test(s) and reference standards were Data collection: were the participants identified and data collected before the index test(s) and reference standards were performed (prospective study) or after (retrospective study)?performed (prospective study) or after (retrospective study)? 11

Reference standardReference standard 77 The reference standard and its rationaleThe reference standard and its rationale 11

Test methodsTest methods88

Technical specification of material and methods involved including how and when measurements were taken, and/or cite Technical specification of material and methods involved including how and when measurements were taken, and/or cite references for index test(s) and reference standardreferences for index test(s) and reference standard

0,50,50,50,5

  99

Definition and rationale for the units, cutoffs and/or categories of the results of the index test(s) and the reference Definition and rationale for the units, cutoffs and/or categories of the results of the index test(s) and the reference standardstandard

0,50,50,50,5

  1010

The number, training and expertise of the persons (a) executing and (b) reading the index test(s) and the reference The number, training and expertise of the persons (a) executing and (b) reading the index test(s) and the reference standardstandard

0,50,50,50,5

1111Whether or not the reader(s) of the index test(s) and reference standard were blind (masked) to the results of the other Whether or not the reader(s) of the index test(s) and reference standard were blind (masked) to the results of the other test(s) and describe any information available to themtest(s) and describe any information available to them

0,50,50,50,5

Statistical methodsStatistical methods1212

Methods for calculating measures of diagnostic accuracy or making comparisons, and the statistical methods used to Methods for calculating measures of diagnostic accuracy or making comparisons, and the statistical methods used to quantify uncertainty (e.g. 95% confidence intervals)quantify uncertainty (e.g. 95% confidence intervals) 11

  1313 Methods for calculating test reproducibility, if doneMethods for calculating test reproducibility, if done

0,50,50,50,5

Methods

Selected Articles’ analysis

Methods

Selected Articles’ analysis STARD checklist Section and topicSection and topic ItemItem DescribeDescribe ValueValue

Article Article scorescore

RESULTSRESULTS      

ParticipantsParticipants 1414 When study was done, including beginning and ending dates of recruitmentWhen study was done, including beginning and ending dates of recruitment 11

  1515

Clinical and demographic characteristics (e.g. age, sex, spectrum of presenting symptom(s), comorbidity, Clinical and demographic characteristics (e.g. age, sex, spectrum of presenting symptom(s), comorbidity, current treatment(s), recruitment center)current treatment(s), recruitment center)

11

  1616

How many participants satisfying the criteria for inclusion did or did not undergo the index test and/or the How many participants satisfying the criteria for inclusion did or did not undergo the index test and/or the reference standard; describe why participants failed to receive either test (a flow diagram is strongly reference standard; describe why participants failed to receive either test (a flow diagram is strongly recommended) recommended)

11

Reference standardReference standard 1717 Time interval and any treatment administered between index and reference standardTime interval and any treatment administered between index and reference standard 11

  1818

Distribution of severity of disease (define criteria) in those with the target condition; describe other Distribution of severity of disease (define criteria) in those with the target condition; describe other diagnoses in participants without the target condition diagnoses in participants without the target condition

11

Test resultsTest results1919

A cross tabulation of the results of the index test(s) by the results of the reference standard; for continuous A cross tabulation of the results of the index test(s) by the results of the reference standard; for continuous results, the distribution of the test results by the results of the reference standardresults, the distribution of the test results by the results of the reference standard

11

  2020

Indeterminate results, missing responses and outliers of index test(s) stratified by reference standard result Indeterminate results, missing responses and outliers of index test(s) stratified by reference standard result and how they were handledand how they were handled

11

   2121 Adverse events of index test(s) and reference standardAdverse events of index test(s) and reference standard 11

EstimationEstimation 2222 Estimates of diagnostic accuracy and measures of statistical uncertainty (e.g. 95% confidence intervals)Estimates of diagnostic accuracy and measures of statistical uncertainty (e.g. 95% confidence intervals) 11

  2323

Estimates of variability of diagnostic accuracy between subgroups of participants, readers or centers, if Estimates of variability of diagnostic accuracy between subgroups of participants, readers or centers, if donedone

11

  2424 Measures of test reproducibility, if doneMeasures of test reproducibility, if done

0,50,50,50,5

DISCUSSIONDISCUSSION 2525 The clinical applicability of the study findings The clinical applicability of the study findings 11

Bossuyt, P. M. et. al., Towards Complete and Accurate Reporting of Studies of Diagnostic Accuracy: The STARD Initiative, Annals

of Internal Medicine Vol 138 – nº1, 7 January 2003

Methods

Selected Articles’ analysis

• 3 groups ():

- G1: Title, Abstract, Keywords, Introduction

- G2: Methods

- G3: Results, Discussion

• 1 group (α):

- Makes the final decision when occur disagreements within groups in the classification of the article

- In the end, analyses 1/3 of the articles, randomly selected

• Final score of each article.

• Mean score of the articles for each year (1998 and 2005)

• Comparison of the 2 means

• Interobserver variability

• The status of each STARD item in each year

Methods

Statistical analyze of the extracted data

Points to analyse:

• Creation of a data base, in SPSS

• Determination of the scores of each article

• Execution of an Inter-class correlation coefficient and the limits of Bland and Altaman

• Determination of mean, range, and standard deviation of each item of the STARD, for each year

• Determination of mean score of the articles, for each year

• Execution of an unpaired two-tailed t-test for independent samples (significance level 0,05)

Methods

Statistical analyze of the extracted data

How ?

Results (Articles’ search)

Planning

• Professor Altamiro da Costa Pereira

• Professor Cristina Santos

Acknowledgments