Embed Size (px)
Citation preview
This article was downloaded by: [University of Montana]On: 30 September 2014, At: 03:01Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Journal of EcotourismPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/reco20
A Delphi toolkit for ecotourismresearchHolly M. Donohoe aa Department of Tourism , Recreation and Sport Management,University of Florida , Gainesville, FL, USAPublished online: 15 Jan 2011.
To cite this article: Holly M. Donohoe (2011) A Delphi toolkit for ecotourism research, Journal ofEcotourism, 10:1, 1-20, DOI: 10.1080/14724040903418897
To link to this article: http://dx.doi.org/10.1080/14724040903418897
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.
This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
A Delphi toolkit for ecotourism research
Holly M. Donohoe∗
Department of Tourism, Recreation and Sport Management, University of Florida, Gainesville,FL, USA
(Received 5 September 2008; final version received 13 October 2009)
The Delphi is considered a legitimate and valuable research technique in a range ofresearch fields. Its value is based on its utility for forecasting, issue identification orprioritisation, and concept or framework development. However, systematic guidelinesfor designing and executing a Delphi are not widely available. Concomitantly, aplethora of methodological interpretations are proving a source of confusion for theDelphi architect. This paper explores the utility of a proposed generic Delphi toolkit(GDT) for designing and implementing a Delphi study. An ecotourism researchexample is introduced as a basis for testing the GDT and for a critical review of itsbenefits and challenges for ecotourism research.
Keywords: ecotourism; Delphi technique; research design
Introduction
It is now approaching 60 years since the first Delphi was conducted and more than 30 since itwas first applied by tourism researchers. During this time, the Delphi has broadened its fieldsand frequency of application. Despite the method’s growing popularity, the Delphi literaturehosts two compelling concerns that serve as the foundation for this paper. The first concern isrelated to methodological refinement. Methodological critique and evaluation are necessaryfor identifying opportunities and constraints and for encouraging methodological refinementand best practice (Alberts, 2007; Brown, 2007). In its place, the Delphi literature is dominatedby one-off case studies that lack sufficient detail about their design and execution to supportmethodological evaluation (Powell, 2003). Mullen (2003) and Rowe and Wright (1999)argued that our knowledge about the structure and potential of the Delphi is poor as aresult. This trend is inexorably linked to a second concern related to Delphi architecture.The literature has come to suggest that there is no reference standard for Delphi architectsand there has been difficulty in drafting a ‘generic’ or ‘universal’ description of themethod and its procedures (Linstone & Turoff, 2002). As a consequence, there exists realpotential for confusion and ad hoc Delphi design and this raises serious questions aboutthe method’s validity and reliability. Given that guidelines are essential for Delphi designand administration, that a lack of guidelines is contributing to confusion and that confusionis impeding methodological review, the development and evaluation of a ‘generic’ descriptionof the method is required for methodological refinement.
ISSN 1472-4049 print/ISSN 1747-7638 online
# 2011 Taylor & FrancisDOI: 10.1080/14724040903418897
http://www.informaworld.com
∗Email: [email protected]
Journal of EcotourismVol. 10, No. 1, March 2011, 1–20
Dow
nloa
ded
by [
Uni
vers
ity o
f M
onta
na]
at 0
3:01
30
Sept
embe
r 20
14
Day and Bobeva (2005) proposed a generic Delphi toolkit (GDT) for providing guidancefor Delphi research. If the GDT is substantiated as a well-proven and robust instrument, theauthors claim that the Delphi could be applied in a wider range of disciplines and methodologi-cal confusion could be reduced. The validity of the GDT has yet to be tested; therefore, theremainder of this paper addresses the call for Delphi research concerned with methodologicaltesting and refinement by testing the GDT. In doing so, this paper offers several contributionstowards addressing the aforementioned concerns. First, a critical review of the Delphi tech-nique is provided and the GDT is introduced. Second, a case study example – a researchagenda concerned with cultural sensitivity and ecotourism – is presented in order to test theutility of the GDT for guiding Delphi architecture and administration. By testing the validityand appropriateness of GDT, this paper seeks to consolidate understanding of the Delphitechnique and to offer insight into its application based on a ecotourism research case.
The Delphi technique
The Delphi is a technique used to systematically combine expert knowledge and opinion toarrive at an informed group consensus (Moeller & Shafer, 1994). The Delphi technique wasoriginally developed as a tool for soliciting opinion from a group of experts in order toinform a forecasting process and it is now accepted as a valid technique for a variety ofresearch purposes (Landeta, 2006). Accordingly, the Delphi technique has been appliedin a breadth of fields that include: environmental management (Gokhale, 2001; Wright,2006), business marketing and management (Chevron, 1998; Hayes, 2007), recreation(Austin, Lee, & Getz, 2008; Vaugeois et al., 2005), and health (de Meyrick, 2003). Inthe tourism case, the Delphi has been applied for the purposes of long-range travel andtourism planning (Moeller & Shafer, 1994), to predict change in the tourism industry(Lloyd, La Lopa, & Braunlich, 2000; Yeong, Keng, & Leng, 1989), to develop ecotourismand sustainable tourism management and evaluation frameworks (Miller, 2001; Spenceley,2005), and to develop and/or refine ecotourism definitions (Garrod, 2003).
The Delphi technique is a structured communication process that facilitates (but doesnot force) consensus (Linstone & Turoff, 1975). It is founded on the use of techniquesthat aim to develop from a group of experts, an agreed view or shared interpretation on atopic or issue where there is contradiction, controversy, or conflict (Day & Bobeva,2005). Individual expert contributions are collected through iterative survey rounds,group opinion is assessed, and opportunity is provided for individuals to revise their con-tributions (controlled feedback) (Linstone & Turoff, 1975). Evaluation of the Delphi tech-nique reveals a set of key Delphi attributes and benefits (Donohoe & Needham, 2009):
(1) Legitimacy: An established research technique.(2) Suitability: Well suited to complex problems where the contributions of experts
would contribute to advancing understanding and knowledge about the problem.(3) Proximity: ‘Virtual laboratory’ where physical meetings are not required.(4) Reflexivity: By design, participants think through concepts and questions – so that
quality and objectivity of the data collected and the credibility of the study’s find-ings may be enhanced.
(5) Flexibility: Methodological adaptation facilitates a comprehensive understandingof the research problem.
(6) Repetition: Multiple iterations move the group towards an informed judgementabout a complex problem.
(7) Anonymity: Reduces the risk(s) for group dynamics to influence outcomes.
2 H.M. Donohoe
Dow
nloa
ded
by [
Uni
vers
ity o
f M
onta
na]
at 0
3:01
30
Sept
embe
r 20
14
Delphi proponents advocate anonymity as a means for facilitating rational and reflexiveindividual contribution and for reducing the influence of dominant participants, the powerof persuasion, and the bandwagon effect that is associated with other group methods(Duboff, 2007; Linstone & Turoff, 2002; Rowe & Wright, 1999). Sunstein (2006)argued that anonymity is what distinguishes the Delphi from other group techniques andon this basis, Wolfers and Zitzwitz (2004) and Rowe and Wright (1999) have found theDelphi to be more accurate than other group techniques. However, anonymity does notrule out the possibility of researchers giving undue credence to the opinion of selectexperts, the tendency of the individuals to be influenced by the statistical measure ofgroup judgement, or the comments of panelists reported in summary reports (Wright,2006). Other potential problems associated with the Delphi have been comprehensivelyreviewed elsewhere. Representative examples include critiques and discussion related toexpert selection (Alberts, 2007; Phillimore & Goodson, 2004), questionnaire ambiguity(Sackman, 1975; Seely, Iglarsh, & Edgell, 1980), false consensus (Linstone & Turoff,1975), panel stability, the substantive quality of panelists contributions (Alberts, 2007;Mullen, 2003; Wright, 2006), and the validity of the results (Landeta, 2006; Powell,2003). Antecedent research suggests that the Delphi technique, like other methods, is notfree from flaws, bias, and other potential pitfalls or dangers (Briedenhann & Butts, 2006;Murray, 1979). It is recommended that Delphi administrators recognise and addresspotential constraints and the importance of professional judgement in control of theexercise (Delbecq, Van de Ven, & Gustafson, 1975; Powell, 2003). Furthermore, caremust be afforded the final justification for the technique on the basis of its utility, itssuitability for the research problem, its potential advantages, and the measures availablefor mitigating potential problems. Day and Bobeva (2005) contended that the GDT couldprovide a basis for such justification and the guidance for doing so. As reminder, thishas yet to be substantiated in practice.
The GDT
Day and Bobeva’s (2005) GDT is a three-stage model for designing and implementing aDelphi study. The model is informed by a comprehensive analysis of the Delphi literature.Commonalities between Delphi techniques used by other researchers are identified and syn-thesised into a set of generic Delphi guidelines. In this regard, the GDT is a reference stan-dard for the design, adaptation, and implementation of a Delphi study. The first stage,‘Exploration’, demands rigorous preparation where care is afforded the basis for consensus(purpose statement), the establishment of participant selection criteria, and the creation ofan expert panel; the design of a data collection and analysis instrument; the identification ofan initial set of issues to be tested through Delphi implementation and/or pilot testing. Dayand Bobeva (2005) presented a set of generic design criteria and common options to helpguide Delphi architects through this preparatory phase (Table 1). For the purposes of thispaper, the term ‘Exploration’ has been changed to ‘Preparation’ to simplify the languageand make clear the stage objectives.
The second stage, referred to as ‘Distillation’, typically involves three rounds of surveysthat are circulated to a predetermined expert panel. The literature indicates that a scopinground (Round 1), that is, a preliminary or general survey round, is commonly usedduring the preparation phase as a means to circulate and solicit reactions to an introductionpackage (initial problem and position statement) and/or pilot test surveys (Green, Hunter, &Moore, 1990; Hurd & McLean, 2004; Weber & Ladkin, 2003). This scoping round isoptional, though it is recommended as it may help to identify ambiguities, it allows
Journal of Ecotourism 3
Dow
nloa
ded
by [
Uni
vers
ity o
f M
onta
na]
at 0
3:01
30
Sept
embe
r 20
14
some degree of early validity testing and methodological refinement, improves the feasi-bility of administration, and allows the researcher to narrow the study’s focus (Garrod &Fyall, 2005; Powell, 2003). Following the optional scoping round, the Delphi movesinto the ‘Distillation’ stage where a survey is developed on the basis of the purpose state-ment and the outcomes of the preparatory phase. The survey is circulated to the expert paneland responses are collected (Round 2). The results are synthesised into a summary reportthat is circulated to participants with the subsequent round. The report may not onlyinclude simple statistical analysis (frequency tabulation, median, etc.), but it may alsoinclude representative comments or criticisms from participants (anonymous) and aconsensus status update. The preparation of the report allows the Delphi administratorsthe opportunity to analyse interim results and to monitor progress (or lack thereof)towards consensus while the distribution of the report allows participants the opportunityto reflect on their responses in light of the group’s judgement (Gordon, 1994; Rowe &Wright, 1999). Subsequent rounds are essentially the same as Round 2, with surveys(developed on the basis of the previous round) and summary reports provided to partici-pants. The purpose of the iterative rounds, and particularly the feedback element thatcomprise the ‘Distillation’ stage of the Delphi, is to move the group towards a convergenceof opinion (Day & Bobeva, 2005).
In the final round, a synthesis and/or evolving consensus statement is distributed and afinal judgement is requested from participants (Gordon, 1994). It is important to note thatadditional rounds may be necessary if the Delphi has not yet reached a critical point forstudy termination. To complete this assessment, there are two recommended approaches:extensive statistical testing and simple statistical reporting. Examples of extensive statisticaltesting for consensus include Okoli and Pawlowski’s (2004) use of Kendall’s W coefficientof concordance and Schmidt’s (1997) consensus ratio. Others prefer simple statistics formeasuring the degree of convergence (Day & Bobeva, 2005). For example, the use ofthe mean/median to measure the control tendency and the standard deviation as a determi-nant of convergence is common in the tourism literature (Garrod & Fyall, 2005; Greenet al., 1990; Kaynak & Macauley, 1984; Miller, 2001). Holey, Feeley, Dixon, and Whittaker(2007, p. 52) concur, adding that the evolution of consensus can also be ascertained bydescriptive analysis of group trends such as an increase in agreement percentages, conver-gence of range with standard deviations, and as a decrease in the number of commentsmade. When sufficient convergence has been achieved, the Delphi rounds cease and thegroup judgement is used to inform the final report and the initial research problem(Stage Three). ‘Convergence’ is commonly used in the literature to describe Delphi activi-ties and objectives during this stage; thus, it replaces ‘Distillation’ in the remainder of thispaper.
Table 1. Generic design tool: Delphi criteria and options for the preparation phase.
Generic criteria Design options
1. Purpose of the study Building, exploration, testing, evaluation2. Participants Homogeneous or heterogeneous groups3. Anonymity of panel Full or partial4. Number of rounds Between 2 and 105. Concurrency of rounds Sequential set of rounds6. Mode of operation Face-to-face or remote access7. Communication media Paper-and-pen, Internet-based, telephone, fax, email
Adapted after Day and Bobeva (2005).
4 H.M. Donohoe
Dow
nloa
ded
by [
Uni
vers
ity o
f M
onta
na]
at 0
3:01
30
Sept
embe
r 20
14
The third stage of the Delphi, named the ‘Utilisation’ stage, includes the developmentand dissemination of the final Delphi report. The results from the ‘Convergence’ phase areintegrated into a final consensus judgement and a report is developed on this basis. As anoption for Delphi administrators, a draft may be circulated to Delphi participants for one lastreview. Although a group judgement has been achieved at this point, circulation of the draftprovides participants with a final opportunity to reflect on their responses and comment onthe Delphi outcome. Commentary is then integrated into the final report. Considerationmust also be given to the impacts of the consensus judgement on the initial researchproblem during this stage of the Delphi. That is, the final report, and particularly the con-sensus statement, is to be applied to the original research purpose so that the central researchquestion may be addressed. Dissemination is a distinctive feature of this stage and dissemi-nation activities may be classified as both ‘short term’ and ‘long term’ in nature (Day &Bobeva, 2005). In the short term, the final report is distributed to Delphi participants andothers who may have assisted in the Delphi exercise and/or have an interest in itsoutcome (e.g. supporting organisations, research assistants, etc.). Long-term knowledgedissemination activities include related publications that address both the substance andthe structure of the study. Although, ‘Utilisation’ aptly describes some of the activities atthis stage, it does not describe the essential Delphi product. Therefore, ‘Consensus’ isused for the remainder of this paper to describe the third stage of a Delphi.
Delphi design and implementation: an ecotourism example
Using the GDT, the following sections critically assess the utility of the model for Delphiarchitecture and administration. Each of the generic stages is explored through a Delphistudy concerned with ecotourism and cultural sensitivity. The decisions involved in itsdesign and the critical issues revealed by its implementation serve as focus. The ambitionis to bring the GDT to life in order to test its utility for guiding Delphi architecture generally,and ecotourism research specifically.
Purpose of the study
The purpose of the research is to develop a cultural sensitivity definition and to test itsvalidity among a group of international ecotourism experts (Table 2). The Delphi techniqueis applied in order to identify and describe the definitional elements as well as theopportunities and barriers to culturally sensitive ecotourism. The results are to inform anexamination of the extent to which progress towards making ecotourism sensitive to
Table 2. Delphi response rates and convergence measures.
Round one Round two Round threeResponse ratea 60% 84% 77%
Convergence measures Mean scoreb 4.2 4.26 4.34Standard deviation 0.97 0.87 0.63
aThe research was completed between January 2008 and July 2008. As the research moved into the summer monthsand the busy ecotourism season, it was expected that participation would decline. Therefore, a larger initial groupwas sought in order to mitigate a potentially high attrition rate. This proved to be a useful strategy.bDefinitional appropriateness scores measured on a five-point Likert-type scale where: 5 ¼ very appropriate, 4 ¼appropriate, 3 ¼ somewhat appropriate, 2 ¼ not very appropriate, 1 ¼ not appropriate.
Journal of Ecotourism 5
Dow
nloa
ded
by [
Uni
vers
ity o
f M
onta
na]
at 0
3:01
30
Sept
embe
r 20
14
cultural differences (and similarities) in a global community is to proceed. In completingthese final steps, the research is to offer a practical product – a definition – that contributesto ecotourism discourse related to definitions, cultural sensitivity, and managementguidelines.
The reason for this research focus is directly linked to a critical development in the eco-tourism research and the recent identification of an ecotourism problem: the lack of culturalsensitivity in ecotourism theory and praxis. Since 2000, a growing number of ecotourismresearchers are suggesting that a set of values – embedded in ‘Western’ understandingsof human/environment relationship – are being superimposed on ‘other’ or ‘non-Western’ ecotourism destinations and their value systems (Backman & Morais, 2001;Braden & Prudnikova, 2008; Carrier & Macleod, 2005; Cater, 2006; de la Barre, 2005;Jamal, Borges, & Stronza, 2006; Stark, 2002; Weaver & Lawton, 2007). A lack of sensi-tivity to the cultural context of ecotourism presents perils related to goal achievement.The literature suggests that the real danger exists for model transference failure when asingle ‘ecotourism mould’ is used; there exists potential for benefits to be replaced withinsecurity, resentment, conflict, ecological degradation, and economic loss (Vivanco,2002). Cater (2006, p. 36) asserted that ‘if we uncritically accept Western-constructedecotourism as the be-all-to-end-all, we do so at our, and others’ peril’. In response tothese concerns, ecotourism researchers and practitioners are calling for increased culturalsensitivity, increased reflexivity in research and praxis, and a re-thinking of the so-called‘universal’ ecotourism knowledge. Sofeild (2007, p. 158) concurred: ‘where differentworld views and different aesthetics are involved, a greater degree of acceptance andunderstanding for difference is imperative’.
For ecotourism to truly exemplify sustainable development and the core ecotourismtenets, the ‘mould’ must be sensitive to culture (Cater, 2006; de la Barre, 2005; Honey,1999; Humberstone, 2004; McCool & Moisey, 2001; Stark, 2002; Vivanco, 2002). Inthe ideal case, cultural sensitivity is a key ecotourism attribute as culture has the potentialto affect ecotourism opportunities, experiences, and management (Jamal, Borges, &Stronza, 2006). Perceptions, attitudes, and values may also have the potential to affectopportunities, experiences, and management, and this cultural dimension has the capacityto affect ecotourism’s social relevancy (Nyiri, 2006; Sofeild, 2007). Therefore, the attributedenotes the sensitivity level of ecotourism to cultural differences and similarities and itultimately dictates the adaptation investments (e.g. policy development, trainingmodules, regulations, programmes, research agendas) required of ecotourism stakeholdersand leaders to make more acceptable the sharing of core definitions, planning and manage-ment tools and expertise, and international standards in the cultural mosaic of our globalhuman landscape. To achieve this culturally sensitive ideal, multiple (and overlooked)ecotourism realities need to be identified and the influences, potential conflicts, and out-comes of such phenomena must be understood. The latter requires an epistemology andmethodology that highlights and prioritises cultural sensitivity and that is counter-hegemonic by design (Denzin & Lincoln, 2005; Kincheloe & McLaren, 2005; Saukko,2005). This includes definitions, standards, research models, visions, and strategies thatare sensitive to the diversity of cultural values and identities that exist in the ecotourismlandscape (de la Barre, 2005). Given the contemporary ecotourism context and therapid development of ecotourism markets in emergent destinations – such as Bhutan,Cambodia, China, Thailand, and Vietnam, a response to this call – as Sofield suggests,is imperative.
This research problem and purpose statement is the foundation for all other designdecisions during the ‘Preparation’ stage of the Delphi.
6 H.M. Donohoe
Dow
nloa
ded
by [
Uni
vers
ity o
f M
onta
na]
at 0
3:01
30
Sept
embe
r 20
14
Methodology
The Delphi technique was selected over traditional group methods such as the group inter-view or the focus group, as a valid method for eliciting knowledge and opinion from agroup of international ecotourism experts. The method was judged to be a best-fit on thebasis of its suitability for tourism research (Donohoe & Needham, 2009; Garrod & Fyall,2005; Green et al., 1990) and its demonstrated utility for informing a definition develop-ment process (Garrod, 2003). Delphi attributes (described earlier in the text) may also beconsidered advantages that are congruent with the research purpose and problem. The struc-ture, utility, and purpose of the Delphi technique are congruent with the work of de la Barre(2005) and Stark (2002) who suggest a need for a ‘constructed space’ for consensus build-ing. Given that ecotourism has historically been advanced by non-inclusive knowledgesystems and that this process finds us now in a potentially calamitous predicament, itmakes the most sense to apply a ‘counter-hegemonic’ methodology that supports Haber-mas’s (1990) principle of ‘universalisability’ and the achievement of inclusive consensus.In this way, the product of the Delphi study is to reflect multiple interests, values, and exper-tise while eliminating the risk of furthering Cater’s (2006) ‘hegemonic sphere’ (an exclusiveor Western-derived model). Instead, the Delphi method offers a constructed space for con-sensus building. It facilitates reflexivity and it allows for alternative viewpoints and/orvoices of the ‘other’ to be heard.
It is important to acknowledge that the Delphi exercise is to produce a definition that ispositioned in both time and space. This is an important consideration because the conditionsthat are framing the exercise, including the global popularity of ecotourism and the recentdiscourse related to cultural sensitivity, are likely to affect the outcome. The experts who areto comprise the Delphi panel are also representative of a diversity of truths located in timeand space, and their opinions and contributions are likely to reflect this fact. Therefore, it isrecognised that the outcome of the research is contextual and subjective.
Implementing the Delphi
The preparation stage
The literature establishes that Delphi study objective achievement is intrinsically linked to acarefully designed and executed Delphi plan (Day & Bobeva, 2005; Miller, 2001; Philli-more & Goodson, 2004). Based on the GDT, a three-stage Delphi implementation planwas developed to guide the study (Figure 1).
In the preparation stage, the expert panel design is identified as the most importantdesign decision (Needham & de Loe, 1990). Therefore, a Delphi study requires that thedeclaration of selection criteria, the selection of experts, and the management of anexpert panel (Green et al., 1990; Linstone & Turoff, 2002) be carefully managed. ADelphi study does not rely on a statistically representative sample of participants toachieve valid results (Okoli & Pawlowski, 2004; Powell, 2003). Guidance suggests thatdesign decisions related to Delphi panel size, characteristics, and composition must ‘begoverned by the purpose of the investigation’, the scope of the problem, and the resourcesavailable (Cantrill, Sibbald, & Buetow, 1996, p. 69). The expertise represented on the panelmust therefore be congruent with the research topic and issues.
For this study, a panel of international ecotourism experts was desired so that the Delphicaptures a diversity of experience, knowledge, skills, and cultural perspectives. Guided bythe GDT, a seven-phase purposive sampling strategy was used for identifying, recruitingand selecting participants. It is detailed here because in application, a number of Delphi
Journal of Ecotourism 7
Dow
nloa
ded
by [
Uni
vers
ity o
f M
onta
na]
at 0
3:01
30
Sept
embe
r 20
14
design challenges were revealed and the decisions made to mitigate such challenges mayhelp to shed light on the technique’s strengths and weaknesses.
Phase 1 – panel size: The GDT suggests that there is variation in the requisite size for aDelphi panel, but limited guidance is provided for making panel size decisions (panel sizethat will best serve the research agenda). Thus, the researcher consulted the Delphi literature,and particularly the tourism literature, for guidance in this regard. Linstone (1978) assertedthat accuracy deteriorates with smaller panel sizes and improves with larger numbers. ForDelphi studies that are composed of a mixed group of experts (heterogeneous group), Brie-denhann and Butts (2006) recommend a larger group. The tourism literature containsexamples of panels in the range of 7–100 participants (Choi & Sirakaya, 2006; Greenet al., 1990; Kaynak & Marandu, 2006; Kuo & Chiu, 2006; Lloyd, La Lopa, & Braunlich,2000; Miller, 2001; O’Connor & Frew, 2004; Tsaur, Lin, & Lin, 2006; Weber & Ladkin,2003; Yeong, Keng, & Leng, 1989). The challenge in constructing a heterogeneous panelof international ecotourism experts is in ensuring that the panel includes a diversity of culturalbackgrounds, perspectives, and experience. Given the global context of the research and itscultural theme, this balance is imperative for producing a relevant and inclusive consensus.Thus a goal of 75–125 participants was judged to be congruent both with the aims of thestudy and Delphi research standards. Also, the goal was purposefully broad to facilitate
Figure 1. Three-stage Delphi approach.
8 H.M. Donohoe
Dow
nloa
ded
by [
Uni
vers
ity o
f M
onta
na]
at 0
3:01
30
Sept
embe
r 20
14
inclusion and to mitigate attrition between rounds (see Phase 5). A minimum panel size of 20participants was also determined on the basis of the research objectives as well as recommen-dations found in the literature (Archer, 1980; Linstone & Turoff, 2002). The minimum estab-lishes a critical level under which the Delphi panel can no longer be considered stable and theDelphi can no longer be considered a rigorous exercise (Garrod, 2003). Thus, a safeguard(minimum panel size and attrition rate/critical level monitoring) is in place in the case ofhigh attrition rates during the convergence stage. A balance between academic and pro-fessional experts as well as their global distribution was desired and relatively equal represen-tation throughout the Delphi exercise was maintained.
Phase 2 – panel expertise: A purposive sampling method was employed to select therelevant experts concerning the study topic: (1) ecotourism professionals from government,private industry and non-governmental organisations and (2) academics engaged in eco-tourism research and education. It is assumed that these groups, together (heterogeneouspanel), will contribute rich and varied data and information for the Delphi study. Theinclusion of both professional and academic experts is substantiated by Alberts (2007),Briedenhann and Butts (2006), Donohoe and Needham (2009), Sunstein (2006), andVaugeois et al. (2005) as a means to achieve a balance between differing approaches toand perspectives on ‘knowledge’, to mitigate the existent divide between research andpractitioner communities (knowledge sharing, communications, priorities, epistemology,etc.), and to achieve an inclusive Delphi outcome. An intriguing dichotomy is present,and it was expected that consultations with these groups would provide meaningfulresults. For this study, the ‘expert’ must be proficient in English and they must satisfy aminimum of one of the selection criteria (parameters for participant recruitment andselection).
(1) Current or previous experience in the public or governmental sector related to eco-tourism and/or nature-based tourism activities (minimum 5 years)
(2) Current or previous experience in the private sector related to ecotourism and/ornature-based tourism activities (minimum 5 years)
(3) Evidence of professional productivity in terms of peer-reviewed or professionalpublications and research and/or participation in academic or industry symposia
(4) Teaching portfolio that includes courses dedicated to tourism and/or ecotourism(minimum 5 years)
(5) Other – potential participants are to be afforded the opportunity to provide insightinto ecotourism expert qualities and/or attributes that may have been overlookedand that should be considered for participant selection.
The researcher recognises that the language requirement is a very important element inthis study as it introduced a potential bias – particularly because of the desired diversity of‘voices’. However, English is the primary language of the researcher and for this reason, aswell as logistical reasons such as time and budget, multiple language editions of the surveywere not developed. Instead, English served as the unifying language for the panel andwhere possible, language assistance was provided. For example, support was provided toEnglish-as-a-second-language participants in locations such as Canada, Brazil, China,Greece, and Mexico so to avoid unnecessary language-based exclusion or attrition. Theirony of only using English in a study of cultural sensitivity is acknowledged.
Phase 3 – expert recruitment: Portals through which ecotourism experts were identifiedand recruited include international peer-reviewed publications and organisations whereecotourism experts publish and/or seek membership. Peer-reviewed journals provide
Journal of Ecotourism 9
Dow
nloa
ded
by [
Uni
vers
ity o
f M
onta
na]
at 0
3:01
30
Sept
embe
r 20
14
publication portals where ecotourism experts publish their research findings. For example,the Journal of Ecotourism, the Journal of Sustainable Tourism, the International Journal ofTourism Research, and Tourism Management are established publications where ecotour-ism research is frequently published. Expertise is established through a rigorous peer-review process. In addition to the selection criteria, peer-review serves as an additionalmeasure of expertise. International organisations where ecotourism experts seek member-ship provide complementary portals for recruitment. The International EcotourismSociety (TIES) and the International Ecotourism Club (ECOCLUB) are global organis-ations where ecotourism experts (both academics and professionals) seek membership.Unique to these organisations is a strong representation of ecotourism professionals froma diversity of locations and cultural backgrounds. While academic experts are also rep-resented in the membership ledgers, the [total] diversity of ecotourism expert members isthe strength and utility of these portals. In total, six portals are included as they offer unpar-alleled access to ecotourism experts generally, and potential research participants specifi-cally. In the absence of these select portals, the international pool of ecotourism expertswould be difficult to access and the challenge of constructing the desired Delphi panelwould be magnified.
Phase 4 – recruitment strategies: Using the expert portals, direct and assisted strategieswere used to identify potential participants and to create a potential participants database.First, individuals with ecotourism research experience are identified through a strategicreview of papers published in the aforementioned journals within the last 5 years (2002–2007). This direct approach was expected to produce 100 individuals but it produced160. By way of the TIES portal, additional ecotourism experts were identified from themembership and expert lists found on the organisations’ public website. Individuals andtheir contact information were harvested and entered into the potential research participantdatabase. In addition to these direct methods, TIES and ECOCLUB assisted by circulating a‘call for participants’ to their membership through electronic or print newsletters and/orposting the ‘call’ on their website. Interested individuals were invited to submit an‘expression of interest’ to the researcher. This assisted approach was expected and didproduce a pool of approximately 600 potential participants for the Delphi exercise. Thelists were cross-referenced to remove duplications and the final pool included a total of706 individuals.
Phase 5 – panelist screening and selection: Potential participants were contacted byemail with an invitation to participate. The invitation contained a description of the researchand the participant selection criteria as well as a link to an Internet-based survey. The surveyrequired potential participants to indicate which, if any, of the expert selection criteria theysatisfy, to indicate their English language proficiency, to identify their nationality, to self-identify as an academic, professional, or combination thereof, and to provide their contactinformation. Respondents were provided with a 14-day period in which to respond. Atotal of 159 individuals responded while 157 satisfied the expert selection criteria.
Phase 6 – panel composition: While creating a panel that is statistically representativeof the World’s population of cultures is likely to be laborious and time consuming, it wasnot likely to add great(er) value to the study. Instead, the decision was taken to develop apanel comprising a diversity of individuals from a variety of locations. To establish diver-sity, two measures were employed. First, the number of nationalities, organised by conti-nent, provided a simple diversity measure (Figure 2). A diversity of nationalities wasdesired (i.e. at least 25) while a lack of diversity (i.e. few nationalities or a western predica-tion) was to serve as indication that cultural richness had not been achieved. This was notthe case and no additional recruitment was required. While this measure may be considered
10 H.M. Donohoe
Dow
nloa
ded
by [
Uni
vers
ity o
f M
onta
na]
at 0
3:01
30
Sept
embe
r 20
14
subjective by some, statistical sampling is acknowledged by the GDT as unnecessary forestablishing validity (Day & Bobeva, 2005). Instead, representativeness and validity isestablished by other rules of thumb that are best described on the basis of their appropriate-ness to the research at hand (Linstone & Turoff, 1975; Miller, 2001; Spenceley, 2005). Thepanelist’s locations provide a secondary measure of panel composition. The locations weremarked on an Internet-based map and the link was shared with participants. For both theresearcher and the participant group, the map provided an interactive and public view ofthe ‘laboratory’. A map with markers across the globe, while a very basic indication,was meant to complement the aforementioned diversity measure.
Phase 7 – initiate Delphi rounds: Once the participants were selected and the desiredbalance was achieved, the participants were sent a formal email invitation to participate. Alink to the round one Internet-based survey was provided in the letter. The Internetcomprised the primary communication medium in the preparatory stage. The Delphi invita-tions were administered by email, and the round one survey was administered using anInternet-based survey software provider. The advantage of using the Internet is that it isa convenient, economical, and quick method for transmitting communications andcollecting data (Denzin, 2004; Donohoe, 2008). It is an increasingly attractive option forDelphi administrators who struggle to mitigate the traditionally long waits betweenDelphi iterations when surface mail is the primary communication method (Day &Bobeva, 2005; Donohoe, 2009). While surface mail was offered as an alternative toparticipants who do not have the Internet access or who prefer the traditional paper-and-pen approach to surveys, this offer was not accepted by any of the Delphi panelists.Therefore, a critical design decision was made to continue Internet-based communicationsand e-surveys through subsequent rounds.
As the expert panel construction is considered the most important Delphi design feature,this very detailed and strategic approach was considered methodologically imperative.The care to detail and process afforded in this stage, as guided by the GDT, produced aDelphi panel consisting of 157 ecotourism experts (47 professional, 22 academic, 25both academic and professional) from over 40 countries (Figure 2).
It is important to note that the time required to complete the preparation stage variesconsiderably across Delphi studies. Defining the research problem and purpose can take
Figure 2. Delphi panel composition.
Journal of Ecotourism 11
Dow
nloa
ded
by [
Uni
vers
ity o
f M
onta
na]
at 0
3:01
30
Sept
embe
r 20
14
months of preparation, review, and preliminary investigation. The recruitment process canalso vary depending on the challenges that may present when identifying, approaching, andinviting potential participants to join the panel. A standard time frame for the preparationstage is therefore not defined by the GDT or elsewhere in the literature. In this case, thetotal time for the preparation stage was 1 year with the majority of this period dedicatedto defining the research problem and process. Once the panel recruitment process wasdefined, it took approximately 2 months to complete the seven-phase process describedhere. This would have been much longer if traditional communications methods wereused; the use of the Internet expedited communications and recruitment. A Delphi studyrequires more time and effort than many of the methodological alternatives available tothe ecotourism researcher. Therefore, it is recommended that the research architectbudget an appropriate amount of time for the design decisions and activities requiredduring the Delphi study preparation stage and that the Internet be considered as a communi-cation and time management tool.
The convergence stage
The GDT recognises that preparatory effort is necessary before initiating the convergencestage, but it does not distinguish this work as a separate stage (Day & Bobeva, 2005).Instead, the GDT suggests that the initial base of knowledge be ‘generated’ through thefirst round of the study and that it be complimented with a synthesis of the key issues ident-ified in the literature. Therefore, the first round was a scoping exercise designed to getparticipants reflecting about the relationship between cultural sensitivity and ecotourism.Round one asked: What are the three most important issues surrounding contemporaryecotourism?; How important is it for ecotourism to be explicitly sensitive to culture andcultural differences around the globe?; How appropriate is the working definition of culturalsensitivity for ecotourism (definition provided)?; and What cultural variables or attributesneed to find expression in the definition? Participants were provided with a one-monthperiod in which to respond. A response rate of 60% was achieved (Table 2) and theresults revealed that a majority agree (81%) that cultural sensitivity is a very important eco-tourism issue that is highly relevant to the contemporary context but neglected in theory andpraxis. The working definition for culturally sensitive ecotourism provided to the panelistswas purposefully broad and loosely structured so as to allow the group to inform the shapeand content of the definition over the course of the Delphi exercise. As expected, the firstdefinition resulted in a flurry of valuable comments, suggestions, and critiques that wereused to enhance the definition and inform the subsequent Delphi rounds.
The GDT suggests that subsequent rounds may be considered ‘evaluation and exten-sion’ of the results produced in the first ‘scoping’ round. Therefore, the focus of the remain-der of the study was on developing a group consensus definition for culturally sensitiveecotourism. This included two survey rounds that successfully captured hundreds of com-ments and suggestions from the Delphi panel. In each round, an enhanced definition wasintroduced and feedback was collected regarding its appropriateness. In order to assessthe level of convergence between rounds, simple mean scores and standard deviationwere calculated (Table 2). This approach is substantiated by the GDT as well as throughoutthe Delphi literature and in particular, the Delphi research within the tourism domain (Greenet al., 1990; Moeller & Shafer, 1994). As the definition was enhanced, the research revealeda convergence of group opinion and an increase in convergence between rounds. Betweenrounds two and three, the definitional enhancements were minimal, no significant change inthe mean scores was observed, but the increase in convergence (reduction in standard
12 H.M. Donohoe
Dow
nloa
ded
by [
Uni
vers
ity o
f M
onta
na]
at 0
3:01
30
Sept
embe
r 20
14
deviation value) was noteworthy. Thus, it was decided that consensus had been reached andthat further rounds would not produce additional convergence of opinion.
It is important to note that the design of the data collection instrument evolved betweenrounds. The GDT suggests that ‘creativity’ should guide any data collection mechanism andthat different ways of conceptualising the issue and structuring questions must be tried so asto match the expertise of the panel and their preferred communication mode. The GDTprovides standard guidance on structuring the survey questions in accordance with well-proven survey design practices. Moreover, the GDT suggests that the key is formulatingclear, concise, and unambiguous questions, together with clear instructions for the panelists.It is also suggested that the researcher assume a commonsense approach to survey designwhere care is afforded an aesthetically pleasing and a standard and technically easy formatso as to facilitate participation and mitigate attrition. For example, they suggest includingquestions and scales that are easily identifiable by the participant and where the participantdoes not require additional training in order to complete the survey. As recommended by theGDT, this study employed a combination of open and closed-ended questions and themajority of consensus building activities and measures were based on a five-pointLikert-type scale where panelists judged the appropriateness of the definition from ‘five’for very appropriate to ‘one’ not appropriate (Table 2).
It is also important to note that the time required to complete the convergence stagevaries across Delphi studies. Contributing factors include the time between rounds, thenumber of survey rounds required to achieve consensus, administration challenges, andattrition. While the literature contains reports of Delphi’s that required several months toseveral years (Brown, 2007; Hanafin et al., 2007; Hurd & McLean, 2004; Vaugeoiset al., 2005), 1 year appears to be the average length of time required for Delphi adminis-tration (Landeta, 2006). In this case, the survey rounds were administered between Januaryand July 2008. The total time required to complete the convergence stage was 6 months.
The consensus stage
The convergence stage produced a definition that is considered the most appropriate to aninternational panel of ecotourism experts. The process assures that the definition has under-gone testing and that a consensus judgement has been rendered regarding its value andappropriateness. Before declaring ‘consensus’, a draft final report was circulated to theDelphi panelists with a request for final comments. A few panelists submitted final rec-ommendations and these were integrated into the final consensus definition (Figure 3). Afinal Delphi report was then prepared and disseminated to the panelists, supporting organ-isations, and other interested stakeholders. The final result is a definition for culturallysensitive ecotourism.
Discussion: critical issues for Delphi administration
The Delphi is a legitimate research technique that is proving useful in a growing range ofdisciplines and fields. In the ecotourism case, it has been shown that the Delphi is an effec-tive technique for the generation of opinion from a geographically dispersed group ofecotourism experts (Garrod & Fyall, 2005; Miller, 2001). The GDT introduced by Dayand Bobeva (2005) can be a valuable source of guidance for Delphi architects in allstages of ecotourism research administration. In this case, the GDT has provided a usefulinstrument for moving through each of the Delphi stages and the researcher was able tocarefully consider and operationalise the utility, structure, and substance of a Delphi for
Journal of Ecotourism 13
Dow
nloa
ded
by [
Uni
vers
ity o
f M
onta
na]
at 0
3:01
30
Sept
embe
r 20
14
the study of cultural sensitivity and ecotourism. In doing so, the exercise revealed a numberof GDT benefits. First, the GDT provides a comprehensive summary of the critical issuesthat must be considered when designing a Delphi study. The ‘checklist’ format of the GDTfacilitates critical reflection on the research problem, the choice of methodology, and theassociated research design and administration decisions. It offers a range of options aswell as a set of recommendations which have proven to be particularly useful for thefirst-time Delphi architect. Second, the GDT describes a set of ‘optional’ iterationswhere researchers can validate the research purpose (scoping round) and the research pro-ducts (draft final report circulation). These options provide researchers with an additionalset of tools that can ensure that the methodology is a ‘best-fit’ for the research, that themethod will produce meaningful data and by extension, and that the results of theDelphi study can be substantiated. As such, the GDT provides researchers with a referencestandard for designing, adapting, and implementing a Delphi study.
In addition to the panel design, instrument design, communication method, timerequirements, and convergence measures, there are other design and implementationissues that have been revealed by the implementation of the Delphi study. Four issueswere selected for discussion here as they are judged to be highly relevant to the ecotourismresearch context and for the methodogical refinement process.
Issue 1: analysis of the Delphi results
The GDT warns about the potential data analysis problems that can arise. However, the dis-cussion is limited to a specific data collection instrument and little guidance is provided forthose using a breadth of instruments and techniques (as is recommended by the GDT). Thisstudy revealed the complexity of processing a large, rich, and varied data set comprisingnumerical and non-numerical data. At this time, there are few tools available for processingdata of this kind and this presents the Delphi administrator with additional time constraints
Figure 3. Definitional enhancement process.
14 H.M. Donohoe
Dow
nloa
ded
by [
Uni
vers
ity o
f M
onta
na]
at 0
3:01
30
Sept
embe
r 20
14
(Day & Bobeva, 2005). As it is recommended that the time between rounds be kept to aminimum – less than 2 months, the researcher maintained strict deadlines so as to avoidattrition between rounds, to avoid ‘survey fatigue’, and to mitigate temporal delay.However, the data and recommended timelines presented the researcher with considerablechallenges in maintaining the research schedule and delivering the analysis reports to thepanelist with the subsequent round. Although deadline adherence served to mitigate attri-tion and to expedite the Delphi process, additional time or flexibility could have allowedfor greater data analysis creativity and the identification of multiple perspectives. In thisregard, the GDT provided limited guidance on time management.
Issue 2: timing of the Delphi surveys
The GDT recommends that the surveys be dispatched so as to capture the maximum avail-ability of the panelists and to achieve an acceptable response rate. This approach has beensubstantiated in the literature, and the work of Briedenhann and Butts (2006) confirmed theimportance of knowing the situational context so as to avoid distractions and attrition. Theirstudy of rural tourism in Africa coincided with the World Forum on Sustainable Develop-ment in Johannesburg and this caused a decline in participation as panelists abandoned theDelphi to attend the event. For this study, care was afforded the timing of the surveys so thatthey did not coincide with the active ecotourism season for professionals and the activeresearch or vacation seasons for academics. The surveys were initiated in January and ahigh response rate was achieved and maintained throughout. Although attrition didoccur, the response rate improved and attrition rates decreased as the rounds progressed.This may be evidence of the panelists level of commitment to the study and/or the culturalsensitivity/ecotourism issue. It may also be the result of the efforts to inform the panelists ofthe Delphi process, goals, and timelines at the outset so as to mitigate confusion and disil-lusionment. It is therefore recommended that researchers become informed of the contextfor administration and that the Delphi be timed to avoid known distractions so as toachieve a desirable response rate. It is also recommended that panel include individualswho are interested in the research topic and that they are well informed so as to prevent frus-tration with the process and loss of interest over the course of the exercise.
Issue 3: communications
The GDT recommends that the researcher maintain a high level of communication for anumber of reasons: it supports participant commitment, it enables an adequate level ofresponse, and it mitigates attrition. For this study, significant time and effort were dedicatedto communications. Through all research stages, the Internet served as the primary com-munication medium. Potential Delphi participants were identified through Internetresources (online journals and databases), and all communications were conductedthrough email. This made communications with a global network of experts possible,rapid, and efficient while recruitment costs were reduced as postage and travel were notrequired. A website was also created to serve as the central research communicationportal. The interactive online map and the survey summary reports were posted andshared through this page. Participants frequented the webpage and they reported thatadded value as a very useful communication portal for the study. The Delphi surveyswere administered through a ‘user-friendly’ online survey provider. Postage costs forsurvey delivery and return were eliminated and data collection and analysis costs wereminimal. While the time required for a Delphi study is substantial, the GDT suggests
Journal of Ecotourism 15
Dow
nloa
ded
by [
Uni
vers
ity o
f M
onta
na]
at 0
3:01
30
Sept
embe
r 20
14
that the time required can be decreased through the use of e-surveys and internet-basedtools. Although this Delphi study was completed in a 6-month period, the reliance onInternet-based communications revealed several challenges – both expected and unex-pected. Technological challenges presented the most significant barrier to participantrecruitment and data collection. Select participants reported difficulties such as an inabilityto open survey links, to download the survey, or to submit their survey responses. In mostcases, difficulty was the result of user error and administrator assistance served to resolvethe issue. In other cases, computer hardware such as operating systems, Internet connec-tions, and technical failures were the source for difficulty. For some, Internet access wasnot available (infrastructure limited in remote locations and developing countries); it wasinterrupted by electricity failures or unreliable connections (or in one case, cablesdamaged by terrorists). Technological failures at the research centre also caused delaywhen the server collapsed and the network remained inaccessible for 7 days. Finally,some email addresses were inactive and this resulted in a significant amount of returnedemails. These challenges are not unique to the ecotourism case, but given the nature ofecotourism operations and research, it is often the case that experts are located in areaswhere Internet communications are not always possible. To maximise communicationsand time savings as well as to mitigate technological challenges, it is recommended thatthe researcher offer a pen-and-paper survey alternative, invest in developing clear directionsfor survey completion, and pilot test online surveys and communications and that theymake themselves or an assistant available for technical support throughout the durationof an Internet-based Delphi study.
Issue 4: panel management
The GDT identifies panel stability as a significant challenge for Delphi administrators.Panel stability is understood to represent minimal attrition rates, preferred panel size man-agement, and preferred panel composition or balance (Garrod & Fyall, 2005). A lack ofpanel stability presents a considerable barrier to maintaining the expert group and achievingthe Delphi’s objectives (Garrod, 2003). In this case, the researcher consulted the Delphi lit-erature for a more comprehensive set of recommendations. The addition of new members,in order to replace withdrawals, is not a recommended mitigation measure. Murray (1979,p. 155) suggested that it damages the ‘very core of the Delphi procedure’ and ‘the resultsthat emerge must be suspect’. Instead, it is recommended that Delphi architects develop aninitial expert sample list that reflects the predetermined expert selection criteria, establish aminimum requisite group size and/or a preferred group size, and develop a panel stabilitymanagement plan accordingly (Choi & Sirakaya, 2006; Gordon, 1994). Garrod and Fyall(2005) recommended that panel stability be assessed periodically throughout the processwith a quality control measure based on predetermined panel composition preferencesand criteria. Based on their experiences, Pan, Vega, Vella, Archer, and Parlett (1995,p. 32) advocated that the ‘sample size should be as large as possible to allow for subsequentdrop-outs, yet small enough to ensure the respondents are all experts in their fields’. In theworst case, Garrod (2003) recommended Delphi termination if the desired panel compo-sition and size are compromised by attrition between rounds. Given the nature of theresearch, the importance of the panel composition, and the established panel sizeminimum, the latter is particularly pertinent. These recommendations complemented theguidance provided by the GDT and were therefore integrated into the Delphi studydescribed here. For ecotourism researchers, a similar path would serve to avoid confusionand study compromise.
16 H.M. Donohoe
Dow
nloa
ded
by [
Uni
vers
ity o
f M
onta
na]
at 0
3:01
30
Sept
embe
r 20
14
Conclusion
The review of the design and implementation of this Delphi study has illuminated thepotential utility of the GDT for ecotourism research. It makes possible the sharing ofvalues across a broad ecotourism stakeholder spectrum, the convergence of groupopinion on contemporary and relevant ecotourism issues, and the delivery of outcomesthat may have important implications for ecotourism management. Furthermore, bringingtogether ecotourism experts is a particularly arduous task given the nature of the environ-ments in which ecotourism activities occur. Experts are spread across the globe and areoften located in remote areas. Bringing these individuals together to discuss pertinentissues presents logistical and financial challenges. The Delphi offers a constructed spacefor examining ecotourism free from the aforementioned challenges. While the examinationof the GDT has illuminated other potential challenges associated with Delphi design andimplementation, the researcher is optimistic that the guidance provided by the GDT andcomplimented by a literature review, that the level of care afforded the research design,and that study implementation controls will mitigate potential challenges and maximisepotential opportunities. The GDT discussed in this paper has been proposed by Day andBobeva (2005) as a Delphi guide. Its utility as a guide has been assessed and it has beenfound to be practical and user-friendly. Furthermore, its utility for research design maybe enhanced, its limitations can be mitigated, and its potential challenges can be addressedwhen additional resources are consulted. For these reasons, it is suggested that ecotourismresearchers consult the GDT for guidance when considering, designing, or conducting aDelphi study.
As the ecotourism research domain moves through its ‘coming of age’ (Weaver &Lawton, 2007, p. 1168), it is important to consider not only emergent themes and knowl-edge voids, but also its methodological evolution. In Backman and Morais’s (2001)review of the methodological approaches used in the ecotourism literature, they concludethat the current methodological state of the domain is relatively immature and narrowlyfocused. The Delphi technique, unlike the commonly applied interview, participant obser-vation, or content analysis methods, is unique because it has not been the focus of signifi-cant attention or critical review in the ecotourism literature. In fact, Garrod (2003) reportedthat the ecotourism literature is very lean in this regard. This paper represents an attempt toaddress this void and to contribute to ecotourism’s methodological ‘coming of age’. It ishoped that it will ignite interest in the Delphi method, critical discourse related to its suit-ability for ecotourism research and that it will inspire other researchers to benefit from itsadvantages, attributes, and applications.
AcknowledgementsThe author wishes to express gratitude to Delphi participants for their expertise and contributions andto TIES and the ECOCLUB for their support.
ReferencesAlberts, D.J. (2007). Stakeholders or subject matter experts, who should be consulted? Energy Policy,
35, 2336–2236.Archer, B. (1980). Forecasting demand: Quantitative and intuitive techniques. Tourism Management,
1(1), 5–12.Austin, D.R., Lee, Y., & Getz, D.A. (2008). A Delphi study of trends in special and inclusive
recreation. Leisure/Loisir, 32(1), 163–182.
Journal of Ecotourism 17
Dow
nloa
ded
by [
Uni
vers
ity o
f M
onta
na]
at 0
3:01
30
Sept
embe
r 20
14
Backman, K.F., & Morais, D.B. (2001). Methodological approaches used in the literature. In D.B.Weaver (Ed.), The encyclopedia of ecotourism (pp. 597–610). Wallingford, Oxon: CABI Publishing.
Braden, K., & Prudnikova, N. (2008). The challenge of ecotourism development in the Altay region ofRussia. Tourism Geographies, 10(1), 1–21.
Briedenhann, J., & Butts, S. (2006). The application of the Delphi technique to rural tourism projectevaluation. Current Issues in Tourism, 9(2), 171–190.
Brown, C.A. (2007). The opt-in/opt-out feature in a multi-stage Delphi method study. InternationalJournal of Social Research Methodology, 10(2), 135–144.
Cantrill, J.A., Sibbald, B., & Buetow, S. (1996). The Delphi and nominal group techniques in healthservices research. International Journal of Pharmacy Practice, 4(2), 67–74.
Carrier, J.G., & Macleod, D.V.L. (2005). Bursting the bubble: The socio-cultural context of ecotour-ism. Journal of the Royal Anthropological Institute, 11, 315–334.
Cater, E. (2006). Ecotourism as a Western construct. Journal of Ecotourism, 5(1&2), 23–39.Chevron, J.R. (1998). The Delphi process: A strategic branding methodology. Journal of Consumer
Marketing, 15(3), 254–264.Choi, H.C., & Sirakaya, E. (2006). Sustainability indicators for managing community tourism.
Tourism Management, 27, 1274–1289.Day, J., & Bobeva, M. (2005). A generic toolkit for the successful management of Delphi studies. The
Electronic Journal of Business Research Methodology, 3(2), 103–116.Delbecq, A.L., Van de Ven, A.H., & Gustafson, D.H. (1975). Group techniques for program plan-
ning: A guide to nominal and Delphi processes. Glenview, IL: Scott, Foresman and Co.de la Barre, S. (2005). Not ‘ecotourism’?: Wilderness tourism in Canada’s Yukon territory. Journal of
Ecotourism, 4(2), 92–107.de Meyrick, J. (2003). The Delphi method and health research. Health Education, 103(1), 7–16.Denzin, N.K. (2004). Prologue: Online environments and interpretive social research. In M.D. Johns,
S.-L.S. Chen, & G.J. Hall (Eds.), Online social research: Methods, issues, and ethics (pp. 1–12).New York: Peter Lang.
Denzin, N.K., & Lincoln, Y.S. (2005). Paradigms and perspectives in contention. In N.K. Denzin &Y.S. Lincoln (Eds.), The Sage handbook of qualitative research (3rd ed., pp. 183–190). ThousandOaks, CA: Sage Publications, Inc.
Donohoe, H. (2009). Internet-based Delphi administration: The evolution of methodological best-practices for tourism research. Travel and Tourism Research Association 40th AnnualConference Proceedings, Honolulu, HI.
Donohoe, H.M. (2008). Internet-based leisure research: Opportunities and constraints (Pesquisa NoLazer Associados a Internet: Oportunidades e Entraves). Motriz. Revista de Educacao Fısica,UNESP, 14(1), 1–8.
Donohoe, H.M., & Needham, R.D. (2009). Moving best practice forward: Delphi characteristics,advantages, potential problems, and solutions. International Journal of Tourism Research,11(5), 415–437.
Duboff, R.S. (2007, September 28). The wisdom of (expert) crowds. Harvard Business Review.Retrieved January 3, 2008, from http://hbr.org/2007/09/the-wisdom-of-expert-crowds/ar/1
Garrod, B. (2003). Defining marine ecotourism: A Delphi study. In B. Garrod & J.C. Wilson (Eds.),Marine ecotourism: Issues and experiences (pp. 17–36). Clevedon: Channel View Publications.
Garrod, B., & Fyall, A. (2005). Revisiting Delphi: The Delphi technique in tourism research. In B.W.Ritchie, P. Burns, & C. Palmer (Eds.), Tourism research methods: Integrating theory and practice(pp. 85–98). Wallingford: CAB International.
Gokhale, A.A. (2001). Environmental initiative prioritization with a Delphi approach: A case study.Environmental Management, 28(2), 187–193.
Gordon, T.J. (1994). The Delphi method. AC/UNU millennium project: Futures research method-ology. Retrieved November 23, 2007, from http://www.futurovenezuela.org/_curso/5-delphi.pdf.
Green, H., Hunter, C., & Moore, B. (1990). The application of the Delphi technique in tourism. Annalsof Tourism Research, 17, 270–279.
Habermas, J. (1990). Discourse ethics on a program of philosophical justification. In C. Lenhardt &S.W. Nicholsen (Eds.), Moral consciousness and communicative action. Cambridge, MA: MITPress.
Hanafin, S., Brooks, A.-M., Carroll, E., Fitzgerald, E., Gabhainn, S.N., & Sixsmith, J. (2007).Achieving consensus in developing a national set of child well-being indicators. SocialIndicators Research, 80, 79–104.
18 H.M. Donohoe
Dow
nloa
ded
by [
Uni
vers
ity o
f M
onta
na]
at 0
3:01
30
Sept
embe
r 20
14
Hayes, T. (2007). Delphi study of the future of marketing of higher education. Journal of BusinessResearch, 60, 927–931.
Holey, E.A., Feeley, J.L., Dixon, J., & Whittaker, V.J. (2007). An exploration of the use of simplestatistics to measure consensus and stability in Delphi studies. BMC Medical ResearchMethodology, 7, 1–6.
Honey, M. (1999). Ecotourism and sustainable development: Who owns paradise? Washington, DC:Island Press.
Humberstone, B. (2004). Standpoint research: Multiple versions of reality in tourism theorising andresearch. In J. Phillimore & L. Goodson (Eds.), Qualitative research in tourism: Ontologies,epistemologies and methodologies (pp. 119–136). London: Routledge.
Hurd, A., & McLean, D. (2004). An analysis of the perceived competencies of CEOs in public parkand recreation agencies. Managing Leisure, 9, 96–110.
Jamal, T., Borges, M., & Stronza, A. (2006). The institutionalisation of ecotourism: certification,cultural equity and praxis. Journal of Ecotourism, 5(3), 145–175.
Kaynak, E., & Macauley, J.A. (1984). The Delphi technique in the measurement of tourism marketpotential: The case of Nova Scotia. Tourism Management, 5(2), 87–101.
Kaynak, E., & Marandu, E.E. (2006). Tourism market potential analysis in Botswana: A Delphi study.Journal of Travel Research, 45, 227–237.
Kincheloe, J.L., & McLaren, P. (2005). Rethinking critical theory and qualitative research. In N.K.Denzin & Y.S. Lincoln (Eds.), The Sage handbook of qualitative research (3rd ed.,pp. 303–342). Thousand Oaks, CA: Sage Publications, Inc.
Kuo, N.-W., & Chiu, T. (2006). The assessment of agritourism policy based on SEA combination withHIA. Land Use Policy, 23, 560–570.
Landeta, J. (2006). Current validity of the Delphi method in social sciences. TechnologicalForecasting and Social Change, 73, 467–482.
Linstone, H.A. (1978). The Delphi technique. In J. Fowles (Ed.), Handbook of futures research.London: Greenwood Place.
Linstone, H.A., & Turoff, M. (1975). The Delphi method: Techniques and applications. London:Addison-Wesley Publishing Company, Inc.
Linstone, H.A., & Turoff, M. (2002). General applications: Introduction. In M. Turoff & H.A.Linstone (Eds.), The Delphi approach: Techniques and applications (pp. 71–79). Reading,MA: Addison-Wesley Publishing Company Inc. Retrieved November 11, 2007, from http://www.is.njit.edu/pubs/delphibook.
Lloyd, J., La Lopa, J.M., & Braunlich, C.G. (2000). Predicting changes in Hong Kong’s hotel industrygiven the change in sovereignty from Britain to China in 1997. Journal of Travel Research, 38,405–410.
McCool, S.F., & Moisey, R.N. (Eds.). (2001). Tourism, recreation and sustainability: Linking cultureand the environment. Wallingford: CABI Publishing.
Miller, G. (2001). The development of indicators for sustainable tourism: Results of a Delphi surveyof tourism researchers. Tourism Management, 22, 351–362.
Moeller, G.H., & Shafer, E.L. (1994). The Delphi technique: A tool for long-range travel and tourismplanning. In J.R.B. Ritchie & C.R. Goeldner (Eds.), Travel, tourism and hospitality research: Ahandbook for managers and researchers (2nd ed., pp. 473–480). New York: John Wiley & Sons,Inc.
Mullen, P.M. (2003). Delphi myths and reality. Journal of Health Organization and Management,17(1), 37–52.
Murray, T.J. (1979). Delphi methodologies: A review and critique. Urban Systems, 4, 153–158.Needham, R.D., & de Loe, R. (1990). The policy Delphi: Purpose, structure, and application. The
Canadian Geographer, 34(2), 133–142.Nyiri, P. (2006). Scenic spots: Chinese tourism, the state, and cultural authority. Seattle, WA:
University of Washington Press.O’Connor, P., & Frew, A.J. (2004). An evaluation of methodology for hotel electronic channels of
distribution. Hospitality Management, 23, 179–199.Okoli, C., & Pawlowski, S.D. (2004). The Delphi method as a research tool: An example, design
considerations and applications. Information and Management Systems, 42, 15–29.Pan, S.Q., Vega, M., Vella, A.J., Archer, B.H., & Parlett, G.R. (1995). Mini-Delphi approach: An
improvement on single round techniques. Progress in Tourism and Hospitality Research, 2(1),27–39.
Journal of Ecotourism 19
Dow
nloa
ded
by [
Uni
vers
ity o
f M
onta
na]
at 0
3:01
30
Sept
embe
r 20
14
Phillimore, J., & Goodson, L. (2004). Progress in qualitative research in tourism. In J. Phillimore & L.Goodson (Eds.), Qualitative research in tourism: Ontologies, epistemologies and methodologies(pp. 3–29). London: Routledge.
Powell, C. (2003). The Delphi technique: Myths and realities. Journal of Advanced Nursing, 41(4),376–382.
Rowe, G., & Wright, G. (1999). The Delphi technique as forecasting tool: Issues and analysis.International Journal of Forecasting, 15, 353–375.
Sackman, H. (1975). Delphi assessment, expert opinion, forecasting, and group processes. SantaMonica, CA: Rand Corporation.
Saukko, L. (2005). Methodologies for cultural studies. In N.K. Denzin & Y.S. Lincoln (Eds.), TheSage handbook of qualitative research (3rd ed., pp. 343–356). Thousand Oaks, CA: SagePublications, Inc.
Schmidt, R. (1997). Managing Delphi surveys using non-parametric statistical techniques. DecisionSciences, 28(3), 763–774.
Seely, R.L., Iglarsh, H., & Edgell, D. (1980). Utilising the Delphi technique at international confer-ences: A method for forecasting international tourism conditions. Travel Research Journal, 1,30–35.
Sofeild, T. (2007). Tourism and natural landscapes: Universal principles or culturally determinedvalues? In H.M. Donohoe & W.J. Li (Eds.), Leisure in context: Implications for human develop-ment and well-being (pp. 150–159). Hangzhou: Zhejiang University Press.
Spenceley, A. (2005). Nature-based tourism and environmental sustainability in South Africa. Journalof Sustainable Tourism, 13(2), 136–170.
Stark, J.C. (2002). Ethics and ecotourism: Connections and conflicts. Philosophy and Geography,5(1), 101–113.
Sunstein, C.R. (2006). Infotopia: How many minds produce knowledge. Oxford: Oxford UniversityPress.
Tsaur, S., Lin, Y., & Lin, J. (2006). Evaluating ecotourism sustainability from the integrated perspec-tive of resource, community and tourism. Tourism Management, 27, 640–653.
Vaugeois, N., Rollins, R., Hammer, K., Randall, C., Clark, B., & Balmer, K. (2005). Realizingimproved knowledge transfer in the leisure industry: A new agenda and strategies to link research-ers and professionals, Travel and Tourism Research Association Canada Chapter ConferenceProceedings, Kelowna, BC, Canada.
Vivanco, L. (2002). Escaping from reality. The Ecologist, 32(2), 26–30.Weaver, D.B., & Lawton, L.J. (2007). Twenty years on: The state of contemporary ecotourism
research. Tourism Management, 28, 1168–1179.Weber, K., & Ladkin, A. (2003). The convention industry in Australia and the United Kingdom: Key
issues and competitive forces. Journal of Travel Research, 42, 125–132.Wolfers, J., & Zitzwitz, E. (2004). Prediction markets. Journal of Economic Perspectives, 18(2),
107–126.Wright, T.S.A. (2006). Giving ‘Teeth’ to an environmental policy: A Delphi study at Dalhousie
University. Journal of Cleaner Production, 14, 761–768.Yeong, Y.W., Keng, K.A., & Leng, T.L. (1989). A Delphi forecast for the Singapore tourism industry:
Future scenario and marketing implications. European Journal of Marketing, 23(11), 15–26.
20 H.M. Donohoe
Dow
nloa
ded
by [
Uni
vers
ity o
f M
onta
na]
at 0
3:01
30
Sept
embe
r 20
14
