Ann. Coll. Surg. H.K. (2000) 4, 87–90 Audit

Annual Scientific Meeting of The College of Surgeons ofHong Kong: A 5-year audit

DWH Lee ( ), ACW Chan ( ), PBS Lai ( ), TMK Fung ( )and SCS Chung ( )*Department of Surgery, Prince of Wales Hospital, The Chinese University of Hong Kong, Shatin, NT, Hong Kong.

Objective: To examine the quality of surgical research in Hong Kong by auditing the results of abstractssubmitted for annual scientific meetings held by The College of Surgeons of Hong Kong.Materials and Methods: From 1993 to 1998, all abstracts submitted for the meetings were evaluated andclassified according to the hierarchy of evidence. The institutions of the presenting authors were recorded.Ovid MEDLINE search was then conducted to determine whether the abstracts were subsequently publishedin an indexed medical journal.Results: In total, 165 abstracts were submitted for the meeting in a 5-year period. Randomized controlledtrials constituted 12% of this research. Participation from community surgeons showed an encouraging trendfrom 17% in 1993 to 63% in 1998. In the first 4 years, 38% of abstracts were of publishable quality. More than half of the prospective randomized controlled trials were selected for presentation and subsequentlypublished. Those abstracts selected for presentation have a higher chance of being published subsequently(P < 0.03).Conclusions: The randomized controlled trial remains a powerful research tool but is scarce in Hong Kongsurgical research.

in order to promote local research. Selected abstractsby external adjudicators are presented at the meet-ings, and a prize is awarded to the best paper pre-sented. We aimed to examine the quality of surgicalresearch in Hong Kong by auditing the results of theabstracts submitted to the meeting in the last 5 years.

Materials and Methods

All submitted abstracts were evaluated and classifiedinto one of four categories according to hierarchy ofevidence3 by 2 authors (DWHL and PBSL) and vali-dated by a senior author (ACWC): (i) category one,prospective randomized controlled trials that includedlaboratory or interventional studies; (ii) category two,prospective non-randomized studies that includedcohort studies or those trials which did not specify the nature of randomization; (iii) category three, non-

*Author to whom all correspondence should be addressed.Email: sydneychung@cuhk.edu.hkReceived 28 October 1999; accepted 12 January 2000.

Introduction

Surgical research has been criticized recently as notbeing conducted in a scientific manner and thusbeing meaningless.1 It is a general perception thatclinical decisions should be based on the best avail-able evidence, for example the results of prospectiverandomized controlled trials.2 In a review of the litera-ture, randomized controlled trials (RCT) persistentlycomprised less than 10% of articles in surgical jour-nals.3,4 A similar phenomenon is also noted at inter-national clinical meetings.5

Since 1993, an annual scientific meeting has beenorganized by the College of Surgeons of Hong Kong

Key words: abstracts, audit of research quality, randomized controlled trial, research quality.

88 DWH Lee et al.

prospective comparative studies that included case-controlled studies, retrospective comparisons; and(iv) category four, non-comparative studies thatincluded cross-sectional surveys, case series andcase reports. The institutions of the presenting authorswere classified as university-based hospital, com-munity hospital and overseas institutions, and an Ovid MEDLINE search was conducted to determinewhether the submitted abstracts were subsequentlypublished in an indexed medical journal.

Results

Over the last 5 years, a total of 165 abstracts weresubmitted to meetings and 46 abstracts were selectedfor oral presentation at meetings. The results are sum-marized (Table 1). The majority of the abstracts sub-mitted belong to category 4, and there is a trend risingfrom 44% in the first year to 70% in the fifth year. Cat-egory 1 abstracts constituted an average of 12% inthe first 5 years (Fig. 1). In the first 3 years, the major-

ity of abstracts were submitted from university-basedhospitals. This phenomenon had changed in the mostrecent 2 years. Participation by community surgeonshas increased from 17% in 1993 to 63% in 1998 (Fig.2). An average of 38% of abstracts in the first 4 yearswere published in an indexed medical journal. Theresults for 1998 are not yet available because it takeslonger than a year for a paper to be accepted andpublished in a journal.

More than half category 1 abstracts were selectedfor presentation and subsequently published (53%and 50%, respectively) (Figs 3,4). Abstracts selectedfor presentation have a higher likelihood of being published subsequently (Table 2, Chi-squared test, P = 0.03).

Discussion

In the literature, RCT persistently comprise less than10% of research output.3–5 The majority of the researchwere non-comparative studies or case series. Theseuncontrolled studies were criticized for not being ableto provide strong evidence for assessing treatment or

Table 1. Summary of abstracts 1993–98 submitted to Collegeof Surgeons of Hong Kong Annual Scientific Meetings since1993

Year No. Category (%) Institution (%) Publishedabstracts 1 2 3 4 Univ. CH OS articles (%)

1993 23 13 26 17 44 83 17 0 391994 37 8 27 11 54 84 16 0 431995 46 11 17 22 50 76 24 0 411996 29 17 21 14 48 31 62 7 241998 30 10 10 10 70 30 63 7 NA

Univ., university hospital; CH, community hospital; OS, overseasinstitutions; NA, not yet available.

Table 2. Comparison of presented abstracts with publishedabstracts in College of Surgeons of Hong Kong Annual Scientific Meetings (1993–97)

Presented Not presented Total

Published 19 32 51Not published 17 67 84Total 36 99 135

P = 0.03.

80

70

60

50

40

30

20

10

0

Abs

trac

ts p

er c

ateg

ory

(%)

1993 1994 1995 1996 1998

13

26

1744

8

2711

54

11

1722

50

1721

1448

10 10 1070

Categories per year

Fig. 1. Categories of abstractssubmitted to College of Sur-geons of Hong Kong AnnualScientific Meetings since 1993,according to hierarchy of evidence: �, category 1; �, category 2; , category 3; ,category 4.

Audit of surgical research 89

answering specific questions and were believed to bemeaningless.1 The lack of RCT presented at surgicalsociety meetings has been attributed to a lack offunding, lack of initiative and the difficulty of perform-ing such trials.2 During the last 5 years, the majority of Hong Kong studies were non-comparative and RCT only comprised approximately 12% of local sur-gical research. Although the number of RCT wasscarce, this was a comparable result to Western societies.

In the era of evidence-based medicine, there is ageneral perception that clinical practice should bebased on the best available evidence, such as RCT.2,6

In this series, more than half of the category 1abstracts were selected for presentation at themeeting and subsequently published in an indexedmedical journal. This reflects that RCT is a powerfulresearch tool and that our peers eagerly await theresults of RCT to guide their clinical practice.

The implementation of specialist examinations in

100

90

80

70

60

50

40

30

20

10

0

91

9

0

16

84

0

76

24

0

31

62

7

30

63

7

1993 1994 1995 1996 1998

Abstract submissions per year

Inst

itutio

nof

abst

ract

auth

ors

(%)

Fig. 2. Category of institutionsof the authors of abstracts submitted to College of Surgeons of Hong Kong AnnualScientific Meetings since 1993:�, university hospital; �, com-munity hospital; , overseasinstitution.

53

48

24

15

1 2 3 4

60

50

40

30

20

10

0

Abstract no. (category)

Cat

egor

yof

abst

ract

s(%

)

Fig. 3. Percentage of each of four categories of abstracts sub-mitted to College of Surgeons of Hong Kong Annual ScientificMeetings since 1993 selected for presentation.

60

50

40

30

20

10

01 2 3 4

5047

41

30

Category per year

Cat

egor

yof

abst

ract

s(%

)

Fig. 4. Percentage of each of four categories of abstracts sub-mitted to College of Surgeons of Hong Kong Annual ScientificMeetings in the period 1993–97 that were subsequently published in indexed medical journals.

90 DWH Lee et al.

1995 has raised awareness of the need for researchtraining among higher surgical specialist trainees.Because the majority of trainees have worked outsideuniversity hospitals, it is not surprising to see the par-ticipation rate from community hospital surgeons hasincreased in the last 2 years.

The overall quality of the annual scientific meetingcan be assessed by whether the submitted abstractswere of publishable quality for peer-reviewed journals.An average of 38% of the abstracts were published inindexed medical journals in the first 4 years studied.Those abstracts selected for presentation at themeeting have a higher chance of subsequently beingpublished. This also reflected the high quality of adju-dicators invited by the College to select the bestquality papers for presentation at the meetings.

To conclude, RCT is a powerful research tool but is

scarce in our surgical research. Increasing partici-pation from community surgeons may improve the situation in the future.

References

1 Horton R. Surgical research or comic opera: Questions, butfew answers. Lancet 1996; 347: 984–5.

2 Bell PR. Surgical research and randomized trials. Br. J. Surg.1997; 84: 737–8.

3 Solomon MJ, McLeod RS. Clinical studies in surgical journals.Have we improved? Dis. Colon Rectum 1993; 36: 43–8.

4 Haines SJ. Randomized clinical trials in the evaluation of surgical innovation. J. Neurosurg. 1979; 51: 5–11.

5 Barnes RW. Understanding investigative clinical trials. J. Vasc.Surg. 1989; 9: 609–18.

6 Sackett DL, Haynes RB, Tugwell P. Clinical epidemiology. ABasic Science for Clinical Medicine. Boston: Little, Brown andCo., 1985.