Green Public Procurement in academic literature: a survey
Sandra Lange1, Jan Telgen2, Fredo Schotanus3
Abstract
The first author’s master thesis (published elsewhere) is a comprehensiveliterature review of 16 years of academic research on Public Procurement.Here we focus on the literature on Green Public Procurement. The originalliterature review concerns papers published in English in academic jour-nals covered by Scopus and Web of Science in the 1997-2012 timeframe.Through a rigorous review process we identified 378 publications on PublicProcurement in general, out of which 29 were in the area of Green PublicProcurement. A first observation is that less than 10% of academic paperson Public Procurement deal with Green Public procurement, so Green Pub-lic Procurement can hardly be considered a dominant topic in Public Pro-curement academic research. But there are more observations to be made.Even though 29 papers is only a limited number to base any comparisonson we find a number of striking differences with the general Public Procure-ment literature. To mention a few: relatively low percentage of academicpapers in typical purchasing and supply management journals, wide spreadof authorship and countries studied, relatively low number of citations, noquantitative modelling.
1. Introduction
Public procurement is a powerful tool to make governments moreefficient. It accounts for 13% to 20% of worldwide GDP [3], meaningthat a significant proportion of all produced products and services arebought by governments. Many researchers emphasized the powerfuleffects that public procurement can have on fostering green produc-tion [13] [18]. To achieve such socially desirable outcome, govern-ments operate as both regulators, passing laws and regulations, andmarket participants [10] [21]. And in times of global supply chains,
1This work was based upon Sandra Langeas master thesis at the University of Twente2Professor of Public Procurement, University of Twente, PO Box 217, 7500 AE Enschede, The
Netherlands, corresponding author [email protected] professor, University of Twente
1
public procurement is not only a means to improve upon social out-comes on domestic markets, but also internationally [21].
This paper is a detailed analysis of findings obtained on the sub-ject of Green Public Procurement (GPP) during a comprehensiveliterature review of 16 years of academic research on Public Pro-curement. While Public Procurement is a highly fragmented field, asystematic overview of research is still lacking. Crossan and Apaydin(2010) state that “fragmentation of the field prevents us from seeingthe relations between these facets and ultimately impedes consolida-tion of the field.” [11, p. 1154]. By showing which topics have beenaddressed by research, to what extent, detecting possibly understud-ied, as well as mature sub-fields, and by enabling researchers withthe tools to conduct syntheses on findings for sub-fields, the field isdeveloped to a new state of more clarity and unification. The prac-tical impact of this work is therefore indirect, by stimulating andenabling a research agenda to derive at generalizable findings.
This literature review provides an overview of the most influentialscientific literature published on the topic of Green Public Procure-ment. Moreover, the overall state of GPP research is assessed, pro-viding insights into the maturity of the field. According to Cheon,Grover and Sabherwal (1993) mature research fields are character-ized by studying a variety of different topics and applying variousresearch methods instead of narrowly focusing on few [9]. Therefore,this review focuses on addressed topics, as well as employed method-ologies and their development over time. The literature review isfocused on the past sixteen years (1997 to 2012).
2
2. Methodology
The methodology was informed by Wynstra (2010) [28]. For re-viewing the Journal of Purchasing and Supply Management ’s pub-lications of the years 1994 to 2009 he developed an extensive listof categories each article was classified into. The main categoriesemployed by Wynstra are: topic, research strategy, data collection,data analysis, type of product, type of purchase, as well industryand sector. With respect to the topics, the author classified eacharticle into a maximum of three subject categories, while another,similar review conducted by Carter and Ellram (2003) [7] on theJournal of Supply Chain Management categorized each article intoonly one subject category which summarized the article’s focus thebest. We believe that Wynstra’s approach yields a more accurate rep-resentation of research topics, since procurement subjects are oftenresearched against a clear background and therefore categorizationinto only one subject field will under-represent the other(s). More-over, this categorization enables us to look more specifically, whichtopics have been studied in the context of the green procurementstrategy.
Each of the main categories was further divided by Wynstra intosub-categories. He also gathered general article data including pub-lication year, contributing authors, institutions and citations. Allmain categories employed in his research were adopted with the ex-ception of “type of purchase”, as a scoping study of the PP fieldhad revealed that this category was irrelevant. While the main cat-egories were mostly adopted, the sub-categories were modified. Histopic sub-categories were to a great extent not applicable to thisreview, since his research mainly focused on private sector procure-ments. The research strategies were all adopted except for “expertinterviews/Focus group” and “laboratory experiment”, which wereinstead grouped as data collection methods. Wynstra’s subcategories
3
for data collection are very specific, such as distinguishing betweenfour kinds of questionnaires. As this level of specificity may falselycreate the impression of variety, the subcategories were simplified.The same applies to his thirty-nine items list of data analysis tech-niques, which was simplified to only distinguish between quantitativeand qualitative research. The product types were amended to ac-count for works while the product type combination “good/service”was omitted. Nine industry and sector categories were adopted fromthe Wynstra classification scheme yet further extended during thedata extraction stage of the review. Instead of collecting institutedata, the country of the institute was noted per article. While insti-tutional data would have provided interesting insights into institu-tional collaborations, as well as most active institutions on the field,this review adopted a global perspective limited to cross-country asopposed to cross-institutional differences. Finally, his categorizationswere extended with data on studied country/ies, publishing journaland the procuring government level. This latter study character-istic is specific to public procurement and inclusion in the reviewprovided valuable information on the context of conducted research.Specifying the research context is according to Denyer, Tranfield andvan Aken (2008) supportive to practical relevance [14]. The completeclassification scheme may be obtained from the authors upon request.The classification scheme was directly transcribed into an SPSS dataextraction form, which was later used to conduct the analyses.
This review’s objective was to assess the overall status developmentof Green Public Procurement research, its predominant study charac-teristics and research designs, as well as addressed topics. The statuswas operationalized as annual publications. Publication counts arean effective outcome measure to assess the scientific importance ofa research field [11] and their development is a reliable indicationwhether the relevance of the field changed. To further character-ize the time developments, publishing countries and journals were
4
assessed over time as well as authorships. It is believed that thesevariables provide a meaningful overview of the main stakeholders ofthe field.
Predominant study characteristics were assessed against the indus-tries and sectors from which the government procures, the procuringgovernment level, studied country/ies and types of products pro-cured. There was no limitation for those categories with regard tomaximum sub-categories per article.
Most frequently employed research strategies were assessed againstthe research methodology, time dimension, research strategy anddata collection methods. While there was no limit as to how manydata collection methods each article was grouped into, the other cat-egories held mutually exclusive sub-categories. This exclusivenesswas only breached when articles clearly articulated to have utilizedmore than one strategy.To detect developments over time, the sixteen years of research havebeen subdivided into equal time intervals of four years each, inspiredby Wynstra (2010) [28] and Carter and Ellram (2003) [7].
Searches were conducted by use of the databases Scopus and Webof Science. Both databases are well-established, multi-disciplinaryresearch platforms, holding a wide variety of peer-reviewed journals,and they are being kept up to date. We chose for two databases toensure all relevant papers are included, since it is possible that onedatabase omits relevant research [11].
To assess whether high impact papers differ from low and mediumimpact publications with regard to study characteristics and researchdesigns, we conducted citation analyses based on mean scores ofthe Scopus and Web of Science citation counts. We included bothdatabases’ citation counts as citations differ per database and there-
5
fore reliance on only one source may over- or undervalue individualpapers. A mean citation count is believed to provide a more realisticassessment of each paper’s scientific impact. Also, not every paperis enlisted in both databases, therefore considering only one of themcould mean that some papers could not be assigned an impact as-sessor although they may be of value to research. A shortcoming ofcitation analysis is that recent papers have had less time to accumu-late citations.
Searches were limited to English articles, published in peer-reviewedjournals. While some authors have critiqued peer-review to be anobscure process [4], potentially biased by knowing the identity ofthe author [23], the limitations of this research did not allow for aquality appraisal of each included article. Therefore, it is believedthat peer-review was the best available measure to have some qualityappraisal in place. It is assumed that high impact research on thesubject of Green Public Procurement will have been translated intoEnglish and that therefore no high impact papers have been disre-garded from the review based on the language restriction.
The timeframe chosen for this systematic review are the years be-tween January 1, 1997 and December 31, 2012. Around the millen-nium a number of initiatives have been launched by the National In-stitute of Governmental Purchasing, Inc. (NIGP) to foster academiato pay more attention to the until then largely neglected field ofpublic procurement [26] [8]. Those included, beyond others, undera partnership agreement with the Florida Atlantic University (FAU)the establishment of the Public Procurement Research Center, aswell as the launch of the first scholarly journal on the field, the Jour-nal of Public Procurement. We decided on setting the cut-off yearfor this review a few years before the launch of those initiatives to,among other things, be able to assess their impact on the field. Thefinal year of consideration, 2012, was the most current research year
6
when this systematic review was initiated in 2013 and was thus cho-sen to represent the most recent developments.
Search terms were developed by testing individual term’s effective-ness against batches of twenty search results. As the main literaturereview’s objective was to assess the status of public procurementat large and not green public procurement in specific, search termswere developed, which address the main research field. After the testbatches had revealed many terms to be ineffective in that they didnot add any new or relevant results, the following are the final searchterms utilized in combination.
• Variations of public: public; government
• Variations of procurement: procurement; purchasing; con-tracting; buying; commissioning
For the database Scopus, searches for the search terms were restrictedto title, abstract and keywords of the article. The proximity oper-ator of W/5 was included between two consecutive search terms toinclude results where the two search terms appear within five words.Scopus advises researchers to use a proximity operator of either 3, 4or 5, if they wished to find the search terms within phrases [2]. Tolower the threat of falsely omitting relevant literature, we utilized thewidest of the advised proximity operators. For the Web of Sciencedatabase searches were restricted to the topic subject and title. Inline with the Scopus searches, the proximity operator NEAR/5 wasused. For the Scopus database searches were restricted to the subjectarea of Social Sciences Humanities. For the Web of Science databasesearches were restricted to the subject areas Science Citation IndexExpanded and Social Sciences Citation Index.
Search results were assessed for relevance in a three-step processbased on Bettany-Saltikov (2010) by comparing title, abstract and
7
the full text against stipulated criteria for in- and exclusion [6]. Themain rationale was that we only wanted to include articles that werestrictly on the topic of public procurement and which provided ex-emplars of current practices, best or worst, as well as guidelines forpractice and research. The criteria may be requested from the au-thors. Relevance assessments were partly conducted in a team offour. The other three researchers could not finish all batches dueto personal agendas that no longer allowed for the time intensive ef-fort. As a result, the title assessment was fully conducted in a team,while the abstract assessment was only partly conducted in a team,and the final full text assessment was conducted by the first authoralone. The remaining 378 articles were then coded against elevenmain categories and subsequently anaylzed by means of descriptivestatistics using the software SPSS.
3. Findings and discussion
3.1. Status of Green Public Procurement Research
3.1.1. Annual Publications
Only 29 articles, translating to 7.7% of the papers included in thecomprehensive literature review, were on the topic of Green PublicProcurement. While the systematic review regarded the time frame1997 to 2012, the first relevant GPP article only got published by theyear 2003. As Figure 1 shows, publications remained low throughoutthe following years, yet with an increase by 2011, which continued in2012. While the annual publications on Green Public Procurementare still low, this increase indicates a raise in scientific awareness.Very likely the annual publications will continue to increase in theyears succeeding 2012.
8
Figure 1: Publication trend
3.1.2. Authorships
59 authors published relevant GPP articles over the years under re-view. Of those, only six authors published more than one article,the most active being Lutz Preuss who published four. Those fourpublications made Preuss the overall fourth most active publisherof Public Procurement research at large. The two authors Parikka-Alhola and Walker published three relevant articles on Green PublicProcurement and they are also amongst the most active authors onPP research overall with their amount of GPP publications. Thisindicates that while the PP research field at large is fragmented witha high amount of individual authors who each only published one ortwo articles, the sub-field of Green Public Procurement appears tohave been approached by researchers more specialized on that par-ticular sub-field. The otherwise high amount of authors with merelyone publication, 89.8%, poses to a scattered field, which is assessedby many researches unfamiliar with GPP. It is believed that researchquality enhances with specialized scientists who are knowledgeableand experienced on the field. In this respect the relatively youngfield of Green Public Procurement research shows some promisingindications of emerging to a well-grounded field.
9
Table 1: Authorships
Authorship Frequency Percent
Single author 6 20.7%Two authors 14 48.3%Three authors 5 17.2%Four authors 2 6.9%Five authors 1 3.4%Six authors 0 0.0%Seven authors 1 3.4%
Total 29 100.0%
48.3% of reviewed paperswere published inco-authorships between two.This is followed by 20.7%of single author papers and17.2% of articles published incollaboration between three.13.8% of articles were pub-lished in collaboration be-tween more than three au-thors, with one paper writtenby seven researchers, whichis the highest amount of au-thors observed for all 378 pa-pers on the PP field at large. With the exception of the very firstpublication in 2003, collaborations appear to have been most promi-nent from the start. This is very different to what we observed forthe PP field at large, where collaborations only became most promi-nent over time.
3.1.3. Publishing Journals
24 journals published GPP literature. The Journal of Cleaner Pro-duction is the most active publicist on the field with a total of 3relevant articles. 20 journals published only one article. This highfragmentation shows that there is no clear, most knowledgeable jour-nal on the field, which is in line with what we observed for the PPfield at large. Few journals have a clear (public) procurement or pub-lic administration background. Instead, the majority are specializedon environmental issues.
A limitation of this review, which was detected during the complete
10
review, is the search restriction to scientific databases which havedemanding requirements on journals to be listed. We found that dueto those requirements the well-established Journal of Public Pro-curement was not listed in the Web of Science database at all, whileScopus only includes articles from 2012 on. Journals specialized onnew, emergent fields such as GPP may not have had enough timeto accumulate those requirements. Therefore, it is likely that thisreview omitted journals with higher publications.
3.1.4. Publishing Countries
Each paper originated from at least one and a maximum of sevencountries, depending on authorship. Per reviewed article the countryof each author was listed, even if they came from the same country.12 countries published relevant articles. The UK were the most activewith 15 publications, closely followed by Finland with 11. With theexception of Japan and South Africa, which each have only published1 article, the GPP research field is not dominated by few highlypublishing countries. This is different to what we observed for thecomplete review of PP research, which showed a strong dominanceby publications from the UK and USA. Notably, 8 of the countries,which published relevant GPP articles, are Western European. Thus,we can still see a dominance, yet by a region instead of by individualcountries. Overall, more countries became active on the field of GPPover the course of time, which further indicates a raise in relevanceof the topic.
3.2. Predominant Study Characteristics
3.2.1. Procuring Government Levels
The majority of papers, 48.3%, did not specify the procuring gov-ernment level. This high proportion of research imprecision is com-parable to what we found during the complete review, where 56.1%
11
of articles did not specify the government level. Moreover, the timeseries analysis shows that the proportion of unspecified governmentlevels increased over time. Specifying the research context makesfindings of more direct practical applicability, as it also enables otherresearchers to assess reasons for contradictory findings.The local government level was studied most frequently, in 27.6% ofpapers, closely followed by the municipal level, which was researchedin 24.1% of papers. The proportion of papers studying the municipallevel increased over time, while the local level’s relevance proportion-ately decreased. Notably, merely one paper studied the federal level.While the overall dispersion as well as development over time is thesame as observed for the main PP research field, the federal level wasstudied close to equally often as the municipal one by articles on PPat large.
Table 2: Government levels
Government Frequency Percent
Governmental 1 3.4%Municipal 7 24.1%Local 8 27.6%Unspecified 14 48.3%
Total 30 103.4%
We further conducted analy-ses which juxtapose the vari-able of interest with the pub-lishing country. For theseanalyses it must be notedthat each contributing au-thor’s affiliation was consid-ered. For instance, whilethere were only two paperspublished by Norwegian au-thors, both were written inco-authorships and thus by in
total five Norwegian authors. Accordingly, the sum of the countries’publications is greater than the sum of reviewed GPP publications.As observed on the basis of a cross-analysis of publishing countryagainst studied government level, the one paper on the federal leveloriginated in the USA. All countries but the USA and France showhigh proportions of unspecified government levels. Especially the
12
Netherlands appear most imprecise in that all five authors, who con-tributed to a total of two articles, did not specify the procuringgovernment level. Further, all countries but the USA merely stud-ied one government level. The USA are the only publishing country,which addressed all three government levels.
A further cross-analysis assessed differences with respect to paperimpact. For this purpose, mean scores of the Scopus and Web ofScience citation counts were calculated. Since observations of singlepapers would have been impossible to generalize, citation counts weregrouped into decadic categories of 0 - 10, 10.5 - 20, and so on, up tothe highest citation of the complete review, 135. A first observationmade on the basis of this cross-analysis is that all twenty-nine articlesreceived low mean citations. The highest citation category is 20.5 to30, and the highest mean citation a GPP paper received was 23. Adetailed look at the data showed that the citation counts were com-parable across the two databases, thus the mean citation presentsa solid representation of the true impact of each paper. Overall,papers published in the final year under review, 2012, received lowcitations between 0 and 3, while three of them did not accumulateany citations in either database. This demonstrates that in fact ci-tation analysis discriminates against recent publications. However,the paper with the highest mean impact, 23, was published in 2009,whereas the first relevant publication of 2003 received a mean of 11citations. Thus, we can still see that time is not the only determinantof citations, but that scientific impact is most influential. While theoverall low citations are a testament to the field’s recency, they alsoindicate a low relevance of the GPP field to other fields.
The data show that papers with more citations are more contextspecific in that none of the highest category papers of 20.5 to 30 meancitations did not articulate the government level, while only 16.7% ofthe medium impact papers did. This sharply contrasts with 65% of
13
low impact papers not specifying the government level. This findingis in line with the previously made annotation that good research isspecific, since only specific research’s findings can be tested againstdifferent contextual backgrounds regarding generalizability.
3.2.2. Product Types
For product types we could also observe a high proportion of im-precision: 41.4% did not specify a product type. This proportion isnoticeably higher than what we found for the PP field at large, where28.6% of articles did not specify the product type. Furthermore, allthree of the highest impact papers did not specify the product types.Procurement requirements vary across different product types andtherefore this variable is key in putting research and its findings intoperspective and making it of practical relevance. Works were rarelystudied, by only two papers. In contrast, in the complete review ofthe PP field at large, works were found to increasingly have gained inrelevance, making up 25% of papers published in the final four years2009 to 2012. A further contrast is that goods were most frequentlystudied in GPP papers (41.4%) while for PP at large, goods wereleast prominent (18%). Over the course of time, goods’ relevanceto GPP research remained relatively stable, while services’ increasednotably. This may indicate an imminent change in rank order ofthose two product types.
The country analysis uncovered that while works have overall beenrarely studied, they were considered relevant by five of the twelvepublishing countries. Thus, there still is potential to expand researchactivity on this particular product class. Finland, Italy, Sweden andthe UK addressed all product types. Dutch and Norwegian pub-lications, on the contrary, did not address a particular product inany publication. According to the time series analysis of publishingcountries, both countries were active in either one (Norway) or both
14
Table 3: Product types per interval
Product type 2001 - 2004 2005 - 2008 2009 - 2012Frequency Percent Frequency Percent Frequency Percent
Goods 5 50.0% 7 38.9%Services 1 100.0% 1 10.0% 8 44.4%Works 1 10.0% 1 5.6%Unspecified 5 50.0% 7 38.9%
Total 1 100.0% 12 120.0% 23 127.8%
(Netherlands) of the two final four year intervals. Thus, while thisimprecision with respect to the important variable of product classmay indicate an exploratory state of research, published in the earlyyears, this is not confirmed by our data. Both countries are thereforeadvised to make their research more specific, especially the Nether-lands, which also never studied any particular government level.With respect to differences across varying paper impacts we findthat the degree of impreciseness increases with each higher citationcategory: while 30% of the low impact papers did not specify aproduct type, 50% of the medium impact papers did and 100% ofthe highest impact papers.
3.2.3. Industries and Sectors
7 different industries and sectors were studied, yet each with a lowrelevance of only 1 to 2 papers (Table 4). The majority of papersdid not specify the providing private industry or sector. Wheneverthose papers referred to services, they were grouped under a pooledcategory ”services, not specified”. Whenever articles referred to ei-ther goods or did not specify a product type while not specifyingan industry or sector, they were categorized under ”private sector,not specified”. 69% of GPP papers were categorized under ”privatesector, not specified”, which is higher than for the PP field at large,where 38.1% of articles fell into this category. This difference is
15
most likely attributable to the overall higher prominence of goods aswell as unspecified product types for GPP research. A further 20.7%were categorized under ”services, not specified”, which is in line with22.5% of the PP articles categorized accordingly. The proportionateshares of these two pooled categories increased over time, howeveraccomplished by an increase in specified industries and sectors.
Table 4: Industries and sectors
Industry / sector Frequency Percent
Private sector, not specified 20 69.0%Services, not specified 6 20.7%Construction 2 6.9%Catering 2 6.9%Defense sector 1 3.4%Professional services 1 3.4%Manufacturing 1 3.4%Transportation 1 3.4%SMEs 1 3.4%
Total 35 120.7%
Finland and Sweden were most versatile in that they addressed 3specific industries and sectors, while the myjority of countries onlyaddressed the pooled categories.Once again, lower impact papers show greater variation in that thetwenty articles falling into the first citation category of 0-10 meancitations addressed all industries and sectors but small and mediumenterprises (SMEs). The medium impact papers were all catego-rized into either the pooled services or pooled goods and unspecifiedprivate category. Highest impact paper with 20.5-30 citations onlyaddressed the pooled goods category as well as SMEs.
16
3.2.4. Studied Countries
While 12 countries published GPP research, 28 different countriesand regions were studied.
Table 5: Countries studied per paper
# Countries Frequency Percent
1 23 79.30%2 2 6.90%3 1 3.40%7 2 6.90%none 1 3.40%
Total 29 100.00%
We found a strong home biasof researchers in that themajority studied their homecountry or the country of theinstitution they were associ-ated with. The majority ofpapers, 79.3%, merely stud-ied one country. As publicprocurement practices varyacross countries, embeddedin specific institutional sys-tem contexts, valuable in-sights can be gained from
critically assessing the home country’s practices against those ofother countries.
The UK was studied most frequently, in 9 papers, whereas 19 coun-tries and regions were only studied once. Further, only 6 of the stud-ied countries are non-European, which indicates that applicability ofresearch findings on GPP is limited to Europe.
3.3. Predominant Research Designs
3.3.1. Methodologies
While it was aspired to only group each paper into either the qual-itative or quantitative category, 1 paper clearly followed a mixedapproach and was therefore categorized as applying both methods.The 29 reviewed GPP articles mainly applied qualitative data analy-sis methods, overall in 79.3% of papers. While during the first eight
17
years, 2001-2008, qualitative research was conducted in all papers,quantitative methods became more prominent in the final four years,2009-2012, when 38.9% of papers applied them. While still 66.7%of papers conducted qualitative research in this final interval, thisdevelopment towards more balance in research methodologies indi-cates maturation. This development also observed for the PP fieldat large, yet with a stronger balance of 48.9% versus 55% in the finalfour years for the qualitative and quantitative methods, respectively.
Table 6: Methodologies
Methodology Frequency Percent
Qualitative 23 79.3%Quantitative 7 24.1%
Total 30 103.4%
All countries, except for Fin-land, Italy and Norway, con-ducted qualitative analysesin the majority of their pa-pers. Finland and Italyapplied both the qualitativeand quantitive analyses closeto eqally often, whereas Nor-way conducted quantitativeanalyses in 100% of its publi-
cations, while qualtitative were stil conducted in 60% of Norwegianpapers.With respect to differences across paper impact categories we findthat lower impact papers show more balance in research method-ologies, while highest impact papers exclusively relied on qualitativeanalyses. This is most likely attributable to the fact that papers withhigher citations were published in the earlier years.
3.3.2. Time Dimension
82.8% of GPP papers designed their research cross-sectional and notlongitudinal. This preference for the snapshot perspective stayedhigh throughout all reviewed years, yet the longitudinal design gainedsome more relevance over time. This development is in line with what
18
we found during the complete review, where overall 79.6% of the 378reviewed articles applied a cross-sectional design.While the majority of countries relied exclusively on the cross-sectionaldeisgn, the 1 paper by Japan was designed longitudinal, whereas Fin-land and the USA still adopted this design in the majority of theirpublications, in 63.6% and 71.4% of papers, respectively.
Table 7: Time dimension per publishing country
Publishing country Cross-sectional Longitudinal
Finland (11) 36.4% 63.6%France (2) 100.0%Germany (4) 100.0%Italy (8) 100.0%Japan (1) 100.0%Netherlands (5) 100.0%Norway (5) 100.0%South Africa (1) 100.0%Spain (4) 100.0%Sweden (6) 100.0%UK (15) 100.0%USA (7) 28.6% 71.4%
As with research methodology, lower impact papers were more bal-anced than higher impact papers. According to Babbie (2006), gen-eralizations should only be made with great caution from cross-sectional studies [5]. Researchers on the field of Green Public Pro-curement are advised to adopt the longitudinal design more fre-quently to make findings meaningful to other contexts.
3.3.3. Data Collection Methods
Interviews and reviews of non-academic literature were most promi-nent in GPP research, each applied by 44.8% of papers. Literaturewas characterized as non-academic when it pertained to non-scienticliterature, such as books, law texts, business reports, web-sites, man-
19
Figure 2: Data sources per interval
uals, contracts and similar written sources. Questionnaires were uti-lized by 20.7% of papers, followed by secondary analysis, 13.8% andreviews of scientific literature, 10.3%. Focus groups were only con-ducted by one paper, whereas no article was based on results fromexperiments. One paper did not specify its data sources. Specify-ing data sources makes research more transparent and enables otherresearchers to test the findings. Accordingly, all researchers are ad-vised to articulate their data sources clearly to develop the researchfield to a more systematic science. Notably, however, is that withregard to data collection imprecision, GPP appears much more spe-cific than the PP field at large, where every fifth article (20.4%) didnot articulate its sources. Besides this difference, the rank order ismostly the same as for PP at large.
Over the course of time, more data collection methods were utilized,
20
and in the final four years all methods except for experiments weredeployed. This is a positive development, as it enriches research withversatility and thus increases generalizability of findings. Moreover,both interviews as non-academic literature reviews were applied con-tinuously less often over time. As survey methods, such as interviewsand questionnaires, record self-reports, potentially biased by factorssuch as respondents’ experience level, their findings have limited va-lidity. Podsakoff and Organ (1986) describe the main problems ofself-reports to be the consistency motif, social desirability and non-verifiability [22, pp. 533 - 535]. Non-scientific literature, too, canhold whitewashed, non-verifiable information, such as corporate web-sites or business reports. Accordingly, the decrease in prominence ofboth these methods is a further positive trend. Secondary analysesshow a slight increase in utilization. By utilizing past researchers’efforts and findings, secondary analysis has the potential to developthe field of Green Public Procurement research to a state of provenfindings and generally accepted paradigms.
Most countries showed variety in data collection in that each applied3 to 4 methods with relatively balanced proportions. Exceptions areFrance, the Netherlands, South Africa and Spain, which each onlydeployed one data source. Japan’s one reviewed article did not spec-ify its sources.
While lower impact papers adopted all data collection methods andare thus the most diversified, utilization of scientific literature reviewsis higher per citation category: while 40% of the low impact papersreviewed scientific literature, 50% of te medium impact papers didand 66.7% of the highest impact papers. This indicates that higherimpact articles utilized past researchers efforts and findings more,which is important in developing the GPP field to a state of definitefindings.
21
3.3.4. Research Strategies
Single case studies were most prominent in Green Public Procure-ment research, conducted by 41.4% of papers, followed by surveyresearch, 27.6%, and multiple case studies, 17.2%. Case studiesare valuable in developing an understanding of one particular phe-nomenon or event holistically. A shortcoming of single case studiesis their generalizability to other units, as the information gatheredis strongly embedded in the constructs of the unit of analysis [20].
Figure 3: Data sources per interval
GPP research relied on this unit specific assessment noticeably moreoften than the complete PP field (28.3%). Eisenhardt (1989) statedthat case studies are particularly suited for new research areas [16],and thus their high prominence may be attributable to the recencyof the GPP field. Multiple case studies improve generalizability since
22
findings can be compared, analyzed and contrasted [17]. Survey re-search has the aforementioned shortcoming of limited verifiability.The overall low prominence of literature studies, conducted in 10.3%of papers, and most notably meta-studies, conducted in one paper,shows that research findings have rarely been synthesized in the past.As research on Green Public Procurement is still in its infancy, thisappears logical. Still, researchers should keep in mind that synthe-ses deliver generalizable findings with broader applicability than casestudies or survey research.None of the reviewed papers conducted quantitative modelling. Thedispersion is mostly the same for the complete public procurementresearch field with the sole difference that quantitative modelling hasoverall been conducted in 17.7% of PP articles. Over the course oftime, more research strategies were applied, which is a positive mat-uration indicator.
Finland is most diversified in that it applied all strategies but surveyresearch with close to equal proportions. This is in line with whatwe found during the complete review: more active publishers showgreater variety in their research designs and are thus most mature.Moreover, Finland is the only publishing country that utilized meta-studies, while it is also strong on literature studies.Papers with lower citations show greater variation in employed re-search strategies: while the first citation category employed all strate-gies, the second only utilized 3 and the third only 2 research strate-gies. Remarkably, the relevance of multiple case studies also increasesper citation category: where 66.7% of highest impact papers con-ducted multiple case studies, 33.3% of medium impact and merely5% of low impact papers did.
23
3.4. Topics
While for the complete review we only analysed how often each topicwas addressed, this focused analysis of Green Public Procurementpapers enabled us to look specifically which topics were addressedagainst the background of GPP. 11 papers addressed a second topic,which translates to 37.9%. The majority of papers thus exclusivelylooked at GPP as a whole concept without studying specific pro-curement decisions. Overall 5 topics were addressed. Legal aspectswere studied most often, by 4 papers in total. Legal aspects werean issue from the beginning on, in fact the very first, relevant articleon GPP was on this topic. Selection was studied by 3 papers andsupplier relations by a further 2. Contracting and procurement toolswere only studied once each. The tool studied was a priority rankingscheme for green procurement. Although this model was developedbased on qualitative attributes data (see [25]) this kind of tool in-dicates that GPP decisions can well be determined mathematicallyafter qualitative data informed a scheme. Accordingly, researchersmay start utilizing quantitative modelling, which we earlier found tohave been a never utilized research strategy.Over the three intervals in which GPP research was published, theamount of topics studied continuously increased. In the final fouryears all topics except for PP tools were assessed. This indicatesa maturation of GPP research from the state of exploration to onewhere specific research problems are assessed.
9 of the publishing countries studied specific topics. This shows thatGPP research is maturing on a global level. Again, Finnish researchis the most developed in that its papers addressed 2 topics, selec-tion and contracting, whereas the other countries studied 1 topic.Legal aspects have been most relevant globally in that they wereassessed by 4 countries, followed by selection and supplier relations,each assessed by 2 countries.
24
Table 8: GPP topics per interval
Topic 2001 - 2004 2005 - 2008 2009 - 2012Frequency Percent Frequency Percent Frequency Percent
Supplier rela-tions
2 25.0%
Selection 3 37.5%Contracting 1 12.5%Legal aspects 1 100.0% 1 50.0% 2 25.0%PP tool 1 50.0%
Total 1 100.0% 2 100.0% 8 100.0%
Figure 4: Topics per publishing country
3.4.1. Study Characteristics per Topic
With respect to addressed government levels per sub-topic of GPPresearch we find that the majority of papers did not specify theprocuring level. While this lack of context inhibits practical rele-vance the high proportion is mainly attributable to the topic of legalaspects of which all four papers did not articulate the level. Besidesthis imprecision, each government level was addressed by the sub-topics, the municipal slightly more often.Services were most frequently studied against each of the sub-topics
25
apart from the procurement tool, which was only directed at theprocurement of goods. Goods were the second most often studied,studied by 4 papers as opposed to 7 on services, whereas the pro-curement of works was never studied.3 industries and sectors were addressed by the 5 sub-topics exclud-ing the two pooled categories. Each of these, namely professionalservices, catering and transportation, were only assessed in 1 papereach, whereas the majority of papers were grouped into the pooledcategories of unspecified private providers of services, goods and un-specified products. The topic of selection was assessed most diverse,against 2 specific industries and sectors, professional services andtransportation. Only the topic of supplier relations was also stud-ied against a specific industry, catering, while all other sub-topics‘papers did not specify the providing industry or sector. This lackof context directly contradicts these papers interest to develop theGPP research further to a state of resolving more specific procure-ment decisions.All sub-topics with the exception of the procurement tool, have beenstudied against multiple countries, which increases generalizability.While 10 countries have been studied against the 5 sub-topics, 3of them are non-European countries, namely Brazil, South Africaand the USA. Where GPP research generally shows a clear tendencytowards Europe, this is a positive sign of research findings becomingmore relevant to countries outside Europe.
3.4.2. Research Strategies per Topic
Qualitative data analysis was most prominent per sub-topic, exceptfor the subject of selection of which all three papers conducted quan-titative analyses. According to Babbie (2006), quantification makesfindings more “explicit” [5, p. 23], while also fostering syntheses, thusthe comparison and pooling with other findings. For the topic of se-lection, decisions are often characterized by economic assessments
26
of the best supplier or partner. Accordingly, a high prominence ofquantitative research appears understandable. It is, however, be-lieved that research on the selection of green suppliers could alsogain from employing both methodologies more equally, since thiswill yield a more complete understanding of the phenomenon. Thiscall for a more balanced approach is applicable to all sub-topics.With respect to the applied time dimension the image across the5 sub-topics is close to balanced except for the subject of legal as-pects which was exclusively studied cross-sectional. An explanationfor this contrast to the overall predominantly applied snapshot per-spective is that articles which studied GPP in general possibly onlylooked at what this strategy entails and how it could be implemented.In contrast, articles with a more specific issue at stake conceivablywere more interested in assessing the long-term effects of followingthe GPP strategy. While an effectiveness assessment should followeach newly introduced procurement procedure, GPP in specific isconcerned about the long-term effects of taken measures.Regarding utilized data sources we find that papers on the sub-topicsmade use of 4 of the collections methods: questionnaires, interviews,secondary analyses and non-academic literature reviews. Articles onselection, the second most prominent subject, were most versatile inthat they deployed 3 of the methods and each with an equal pro-portion. The most often studied sub-topic of legal aspects utilized2 different collection methods, yet non-academic literature reviewswere conducted in all 4 papers on the subject. This high proportionis attributed to the study of law texts and directives, which informedthose researches. We believe that this sub-field could gain interest-ing insights from also consulting practitioners’ experiences with legaldirectives and frameworks. Neglecting personal experiences may oth-erwise result in research of little or detrimental practical relevance.Single case studies were the most prominently applied research strat-egy, conducted by all topics except for contracting, which one paperinstead conducted a literature study. As annotated earlier, single
27
case studies provide deep insights into single phenomena, yet lackgeneralizability to other contexts. Multiple case studies, which in-crease generalizability, have not been conducted by any of the sub-jects. Articles on supplier selection again appear most diversified,having utilized 3 of the strategies with equal shares.
4. Conclusion
This review analyzed the overall status of Green Public Procurementresearch published between 1997 and 2012. We found that the fieldis still largely neglected, with 29 total publications and a moderatepublication rate since the first paper of 2003. A slight upward trendwas observed, which started in 2011 and continued in 2012. This mayindicate that GPP will become of more relevance in the upcomingyears. This increase in scientific awareness is further substantiatedby the fact that an increasing amount of countries published relevantresearch during the later years.Green Public Procurement research is dominated by Europe. Aspublic procurement practices are country specific, embedded in dif-ferent institutional contexts, findings can thus only be applied tonon-European countries with great caution. This inhibition is cor-roborated by the finding that 79.3% of reviewed papers only stud-ied one country, of which European countries were most prominent.Contrasting different countries’ practices generates valuable insightsfor improving domestic procurements.Research collaborations are common and although a high amountof individual authors only published one article, specialized authorsemerged whose publications made them some of the most influentialpublishers on the field of Public Procurement research at large. Suchspecialization is important as research gains quality from researcherswho are more literate on the field.Overall, the field received low citations. The highest mean citationa paper received is 23 (as of March 2014), the majority of reviewed
28
papers received 10 mean citations or less. While this can partly beattributed to the recency of publications, it may also indicate a lowrelevance of green procurement practices to other fields.With respect to study characterisctis and research designs we foundthat although the field is still clearly in its infancy it developed toa diversified state. Even though only 29 relevant research paperswere published in the timeframe under consideration, all variablesof interest were studied. Diversity, as a indicator of maturation,is paramount for deriving at definite findings, tested against variedbackgrounds and by varied means.With the exception of experiments, all data collection methods havebeen utilized by research, as well as all research strategies but quan-titative modelling. Interviews and non-academic literature reviewswere most prominent data sources, while case studies and surveyresearches were most often utilized research strategies. Both theseresearch designs pose to a practice-oriented field. Shortcomings ofthem refer, paradoxically, to practical relevance as all those measureshave limited reliability and their findings are diffcult to generalize.The mostly neglected measures to pool findings (meta-analyses) orapply them to further analyses (scientific literature studies) disablethe field from deriving at definite findings, which can be applied bypractitioners. However, the field is still emerging and as we foundthat more diverse designs were utilized over time those obersvationsdo not pose an alarming research status.The federal government level was only studied once, which indi-cates that GPP is more relevant to lower government levels. Goodswere most often studied, whereas services’ relevance continuouslyincreased over time. Works were only researched in 2 papers. 7 dif-ferent industries and sectors were studied, yet each with a low shareof 1 to 2 papers.Observed shortcomings of published research mainly refer to a highlevel of context inspecificity: 48.3% of reviewed papers did not specifythe procuring government level, a further 41.4% did not refer to any
29
type of product. 69% of the articles were grouped into the pooledcategory of unspecified private industry or sector providing goods orunspecified products. 20.7% of papers were grouped into the pooledcategory of unspecified private industry or sector providing services.Imprecision inhibits both practical applicability of findings, as it alsodisables other researchers to directly assess validity, as well as reasonsfor contradictory findings.In the final years an increasing amount of papers were published witha specific Green Public Procurement problem at stake. This devel-opment shows that research on the field is surpassing its exploratorystatus.
5. Review Limitations
A limitation of this review is that the categorizations of all includedarticles were done by the main author alone. To make the findingsobjective, systematic literature reviews should be conducted withina team of researchers [27] [24]. As this limitation was known atthe beginning of the research project, this paper aimed at makingthe review process highly transparent to enable other researchers toreplicate the work and test the findings.A further limitation regards the exclusion of articles published inother languages than English, which may have yielded a languagebias: Egger et al. (1997) found that for medical sciences researcheswere more likely to be translated in English, if they found significantresults [15].Restricting the search for relevant papers to only two databases mayhave omitted relevant papers, since even the renowned databasesScopus and Web of Science do not hold all relevant articles. Thisapprehension was confirmed by the fact that ten years of publicationsfrom the Journal of Public Procurement were missed because thejournal was rejected by Thomson Reuters, and only accepted byElsevier in 2012.
30
It is generally considered important to include grey literature in asystematic literature review to develop a more complete overview[27] [24] [19]. Grey literature refers to “multiple document typesproduced on all levels of government, academics, business, and orga-nization in electronic and print formats not controlled by commercialpublishing i.e. where publishing is not the primary activity of theproducing body.” [1]. Due to its nature, grey literature is difficultto locate and can be abundant, which would have exceeded the timerange of this review. Moreover, this type of literature does not satisfythe research aim of presenting an overview of only the most influ-ential scientific literature. According to Davies (2000) publicationbias may adversely affect the validity of findings as journals tend tofavour publishing positive results [12].
6. Directions for Future Research
To address this review’s limitations, other researchers are encouragedto replicate the work to test the findings.While the review provides a detailed overview of researched topicsand the designs applied to study them, researchers should take thesefindings into account when designing upcoming studies. To increaseversatility and increase generalizability of findings, subjects shouldbe assessed against different backgrounds and by different means.
31
References
[1] URL http://www.greynet.org/greynethome.html.
[2] URL http://help.scopus.com/Content/h_
bscssrchFASTprox.htm.
[3] February 2013. URL http://www.oecd.org/gov/ethics/
meetingofleadingpractitionersonpublicprocurement.htm.
[4] Lawrence K. Altman. The ingelfinger rule, embargoes, and jour-nal peer review-part 1. The La, 347(9012):1382–1386, May 1996.doi:10.1016/S0140-6736(96)91016-8.
[5] Earl R. Babbie. The Practice Of Social Research. ThomsonWadsworth, 11th edition, January 2006.
[6] Josette Bettany-Saltikov. Learning how to undertake a system-atic review: part 2. Nursing Standard, 24(51):47–56, August2010. doi:10.7748/ns2010.08.24.51.47.c7943.
[7] Craig R. Carter and Lisa M. Ellram. Thirty-five years ofthe journal of supply chain management: Where have webeen and where are we going? Journal of Supply ChainManagement, 39(1):27–39, March 2003. doi:10.1111/j.1745-493X.2003.tb00152.x.
[8] Rosalyn Y. Carter and Rick Grimm. Journal of public procure-ment under the fau-nigp partnership. Journal Of Public Procure-ment, 1(1):3–8, 2001. URL http://ippa.org/jopp/download/
vol1/Carter%20&%20Grimm.pdf.
[9] Myun J. Cheon, Varun Grover, and Rajiv Sabherwal. The evo-lution of empirical research in is - a study in is maturity. Infor-mation & Management, 24(3):107–119, 1993. doi:10.1016/0378-7206(93)90060-7.
32
[10] Jeong-Wook Choi. A study of the role of public pro-curement can public procurement make society better?2010. URL http://www.ippa.org/IPPC4/Proceedings/
13ProcurementPreferences/Paper13-4.pdf.
[11] Mary M. Crossan and Marina Apaydin. A multi-dimensionalframework of organizational innovation: A systematic review ofthe literature. Journal of Management Studies, 47(6):1154–1191,September 2010. doi:10.1111/j.1467-6486.2009.00880.x.
[12] Philip Davies. The relevance of systematic reviews to educa-tional policy and practice. Oxford Review of Education, 26(3 &4):365–378, 2000. doi:10.1080/713688543.
[13] Catherine Day. Buying green: the crucial role of publicauthorities. Local Environment, 10(2):201–209, April 2005.doi:10.1080/1354983042000388214.
[14] David Denyer, David Tranfield, and Joan Ernst van Aken.Developing design propositions through research synthe-sis. Organization Studies, 29(3):393–413, March 2008.doi:10.1177/0170840607088020.
[15] Matthias Egger, Tanja Zellweger-Zahner, Martin Schneider,Christoph Junker, Christian Lengeler, and Gerd Antes. Lan-guage bias in randomised controlled trials published in englishand german. The Lancet, 350(9074):326–329, August 1997.doi:10.1016/S0140-6736(97)02419-7.
[16] Kathleen M. Eisenhardt. Building theories from case study re-search. The Academy of Management Review, 14(4):532–550,October 1989. doi:10.5465/AMR.1989.4308385.
[17] Kathleen M. Eisenhardt and Melissa E. Graebner. The-ory building from cases: Opportunities and challenges.Academy of Management Journal, 50(1):25–32, February 2007.doi:10.5465/AMJ.2007.24160888.
33
[18] Edeltraud Gunther and Lilly Scheibe. The hurdle analysis. aself-evaluation tool for municipalities to identify, analyse andovercome hurdles to green procurement. Corporate Social Re-sponsibility and Environmental Management, 13(2):61–77, 2006.doi:10.1002/csr.92.
[19] Sally Hopewell, Steve McDonald, Mike J. Clarke, and MatthiasEgger. Grey literature in meta-analyses of randomized trials ofhealth care interventions (review). The Cochrane Library, (2),2007. doi:10.1002/14651858.MR000010.pub3.
[20] Mary M. Kennedy. Generalizing from single case stud-ies. Evaluation Quarterly, 3(4):661–678, November 1979.doi:10.1177/0193841X7900300409.
[21] Christopher McCrudden. Using public procurement to achievesocial outcomes. Natural Resources Forum, 28(4):257–267,November 2004. doi:10.1111/j.1477-8947.2004.00099.x.
[22] Philip M. Podsakoff and Dennis W. Organ. Self-reports in organizational research: Problems andprospects. Journal of Management, 12(4):531–544, 1986.doi:10.1177/014920638601200408.
[23] Arnold S. Relman. Peer review in scientific journals-what goodis it? The Western Journal Of Medicine, 153(5):520–522,November 1990. URL http://www.ncbi.nlm.nih.gov/pmc/
articles/PMC1002603/.
[24] Denise M. Rousseau, Joshua Manning, and David Denyer. Ev-idence in management and organizational science: Assemblingthe field’s full weight of scientific knowledge through synthe-ses. The Academy of Management Annals, 2(1):475–515, 2008.doi:10.1080/19416520802211651.
[25] Mary Swanson, Arthur Weissman, Gary Davis, Maria LeetSocolof, and Kim Davis. Developing priorities for greener
34
state government purchasing: a california case study.Journal of Cleaner Production, 13(7):669–677, June 2005.doi:10.1016/j.jclepro.2003.12.011.
[26] Khi V. Thai. Public procurement re-examined. Journal OfPublic Procurement, 1(1):9–50, 2001. URL http://ippa.org/
jopp/download/vol1/Thai.pdf.
[27] David Tranfield, David Denyer, and Palminder Smart. To-wards a methodology for developing evidence-informed man-agement knowledge by means of systematic review. BritishJournal of Management, 14(3):207–222, September 2003.doi:10.1111/1467-8551.00375.
[28] Finn Wynstra. What did we do, who did it and did itmatter? a review of fifteen volumes of the (european) jour-nal of purchasing and supply management. Journal of Pur-chasing & Supply Management, 16(4):279–292, December 2010.doi:10.1016/j.pursup.2010.09.003.
35
A. Bibliography of Reviewed ArticlesT
itle
Yea
rJou
rnal
Cit
atio
ns
Sco
pu
sC
itat
ion
sW
oSM
ean
cita
-ti
ons
Au
thor
s
Ad
dre
ssin
gsu
stai
n-
able
dev
elop
men
tth
rou
ghp
ub
lic
pro
-cu
rem
ent:
Th
eca
seof
loca
lgo
vern
men
t
2009
Su
pp
lyC
hai
nM
anag
e-m
ent:
An
Inte
rnat
ion
alJou
rnal
2818
23P
reu
ssL
.
Th
ehu
rdle
anal
-ysi
s.A
self
-ev
alu
atio
nto
olfo
rm
un
icip
alit
ies
toid
enti
fy,
anal
-yse
and
over
com
ehu
rdle
sto
gree
np
rocu
rem
ent
2006
Cor
por
ate
Soci
alR
e-sp
onsi
bil
ity
and
Envi-
ron
men
tal
Man
agem
ent
2222
Gu
nth
erE
.,S
chei
be
L.
Fos
teri
ng
sus-
tain
abil
ity
thro
ugh
sou
rcin
gfr
omsm
all
bu
sin
esse
s:p
ub
lic
sect
orp
ersp
ecti
ves
2008
Jou
rnal
ofC
lean
erP
rod
uct
ion
2518
21.5
Wal
ker,
H,
Pre
uss
,L
Con
tinu
edon
nex
tp
age
36
Bibliogra
phyofReviewed
Articles–co
ntinued
from
previouspage
Tit
leY
ear
Jou
rnal
Cit
atio
ns
Sco
pu
sC
itat
ion
sW
oSM
ean
cita
-ti
ons
Au
thor
s
Su
stai
nab
lep
ro-
cure
men
tin
pra
c-ti
ce:
Les
son
sfr
omlo
cal
gove
rnm
ent
2007
Jou
rnal
Of
Envir
onm
en-
tal
Pla
nn
ing
An
dM
anag
e-m
ent
2115
18T
hom
son
,J,
Jac
kso
n,
T
Bu
yin
gin
toou
rfu
ture
:S
ust
ain
-ab
ilit
yin
itia
tive
sin
loca
lgo
vern
men
tp
rocu
rem
ent
2007
Bu
sin
ess
Str
ateg
yan
dth
eE
nvir
on-
men
t
1616
Pre
uss
L.
Dev
elop
ing
pri
or-
itie
sfo
rgr
een
erst
ate
gove
rnm
ent
pu
rch
asin
g:a
Cal
ifor
nia
case
stu
dy
2005
Jou
rnal
Of
Cle
aner
Pro
du
ctio
n
1913
16S
wan
son
,M
,W
eiss
man
,A
,D
avis
,G
,S
oco
lof,
ML
,D
avis
,K
Con
tinu
edon
nex
tp
age
37
Bibliogra
phyofReviewed
Articles–co
ntinued
from
previouspage
Tit
leY
ear
Jou
rnal
Cit
atio
ns
Sco
pu
sC
itat
ion
sW
oSM
ean
cita
-ti
ons
Au
thor
s
Exp
erie
nce
sw
ith
gree
nin
gsu
pp
lier
s.T
he
Un
iver
sita
tA
uto
nom
ad
eB
arce
lon
a
2008
Jou
rnal
ofC
lean
erP
rod
uct
ion
1712
14.5
Bal
aA
.,M
un
ozP
.,R
iera
dev
all
J.,
Yse
rnP
.
Gre
enp
rocu
rem
ent
inN
orw
ay;
asu
r-ve
yof
pra
ctic
esat
the
mu
nic
ipal
and
cou
nty
leve
l
2009
Jou
rnal
ofE
nvi-
ron
men
tal
Man
agem
ent
1311
12M
ich
else
n,
O,
de
Boer
,L
Mak
ing
the
mar
-ke
tw
ork
for
the
envir
onm
ent:
Ac-
cep
tan
ceof
(som
e)’g
reen
’co
ntr
act
awar
dcr
iter
iain
pu
bli
cp
rocu
rem
ent
2003
Jou
rnal
ofE
nvir
onm
en-
tal
Law
1111
Ku
nzl
ik,
P
Con
tinu
edon
nex
tp
age
38
Bibliogra
phyofReviewed
Articles–co
ntinued
from
previouspage
Tit
leY
ear
Jou
rnal
Cit
atio
ns
Sco
pu
sC
itat
ion
sW
oSM
ean
cita
-ti
ons
Au
thor
s
Gre
ener
pu
bli
cp
urc
has
ing:
op-
por
tun
itie
sfo
rcl
imat
e-fr
ien
dly
gove
rnm
ent
pro
-cu
rem
ent
un
der
WT
Oan
dE
Uru
les
2006
Cli
mat
eP
ol-
icy
119
10va
nA
ssel
t,H
,va
nd
erG
rijp
,N
,O
oste
rhu
is,
F
Envir
onm
enta
lcr
i-te
ria
inth
ep
ub
lic
pu
rch
ases
abov
eth
eE
Uth
resh
old
valu
esby
thre
eN
ord
icco
untr
ies:
2003
and
2005
2009
Eco
logi
cal
Eco
nom
ics
127
9.5
Nis
sin
enA
.,P
arik
ka-
Alh
ola
K.,
Rit
aH
.
Con
tinu
edon
nex
tp
age
39
Bibliogra
phyofReviewed
Articles–co
ntinued
from
previouspage
Tit
leY
ear
Jou
rnal
Cit
atio
ns
Sco
pu
sC
itat
ion
sW
oSM
ean
cita
-ti
ons
Au
thor
s
Pro
tect
ing
the
en-
vir
onm
ent
thro
ugh
pu
bli
cp
rocu
re-
men
t:T
he
case
ofS
outh
Afr
ica
2008
Nat
ura
lR
esou
rces
For
um
117
9B
olto
nP
.
Ali
fecy
cle
ap-
pro
ach
toG
reen
Pu
bli
cP
rocu
re-
men
tof
bu
ild
ing
mat
eria
lsan
del
-em
ents
:A
case
stu
dy
onw
ind
ows
2011
En
ergy
106
8T
aran
tin
i,M
,L
opri
eno,
AD
,P
orta
,P
L
Lif
ecy
cle
app
roac
hin
the
pro
cure
men
tp
roce
ss:
Th
eca
seof
def
ence
mat
erie
l
2006
Th
eIn
tern
a-ti
onal
Jou
rnal
ofL
ife
Cycl
eA
sses
smen
t
86
7H
och
sch
orn
er,
E,
Fin
nve
den
,G
Con
tinu
edon
nex
tp
age
40
Bibliogra
phyofReviewed
Articles–co
ntinued
from
previouspage
Tit
leY
ear
Jou
rnal
Cit
atio
ns
Sco
pu
sC
itat
ion
sW
oSM
ean
cita
-ti
ons
Au
thor
s
Rel
ease
the
pow
erof
the
pu
bli
cp
urs
e20
06E
ner
gyP
olic
y5
44.
5B
org
N.,
Blu
me
Y.,
Th
omas
S.,
Irre
kW
.,F
anin
ger-
Lu
nd
H.,
Lu
nd
P.,
Pin
dar
A.
Su
stai
nab
lep
art-
ner
ship
sfo
ra
gree
nec
onom
y:
Aca
sest
ud
yof
pu
bli
cp
rocu
rem
ent
for
hom
e-gr
own
sch
ool
feed
ing
2011
Nat
ura
lR
esou
rces
For
um
33
3O
tsu
ki
K.
Con
tinu
edon
nex
tp
age
41
Bibliogra
phyofReviewed
Articles–co
ntinued
from
previouspage
Tit
leY
ear
Jou
rnal
Cit
atio
ns
Sco
pu
sC
itat
ion
sW
oSM
ean
cita
-ti
ons
Au
thor
s
Gre
enp
ub
lic
pro
-cu
rem
ent:
An
aly-
sis
onth
eu
seof
envir
onm
enta
lcr
i-te
ria
inco
ntr
acts
2010
Rev
iew
ofE
uro
pea
nC
omm
un
ity
&In
tern
a-ti
onal
En
-vir
onm
enta
lL
aw
33
Pal
mu
joki,
A,
Par
ikka
-A
lhol
a,K
,E
kro
os,
A
Psy
chol
ogic
alb
ar-
rier
sin
the
road
tosu
stai
nab
led
evel
-op
men
t:E
vid
ence
from
pu
bli
cse
ctor
pro
cure
men
t
2011
Bu
sin
ess
Str
ateg
yan
dth
eE
nvir
on-
men
t
32
2.5
Pre
uss
L.,
Wal
ker
H.
Th
ere
lati
onsh
ipb
etw
een
sust
ain
-ab
lep
rocu
rem
ent
and
e-p
rocu
rem
ent
inth
ep
ub
lic
sect
or
2012
Inte
rnat
ion
alJou
rnal
ofP
rod
uct
ion
Eco
nom
ics
32
2.5
Wal
ker
H.,
Bra
mm
erS
.
Con
tinu
edon
nex
tp
age
42
Bibliogra
phyofReviewed
Articles–co
ntinued
from
previouspage
Tit
leY
ear
Jou
rnal
Cit
atio
ns
Sco
pu
sC
itat
ion
sW
oSM
ean
cita
-ti
ons
Au
thor
s
Bar
rier
sto
add
ress
-in
gsu
stai
nab
leco
nst
ruct
ion
inp
ub
lic
pro
cure
men
tst
rate
gies
2011
Pro
ceed
ings
ofth
eIn
stit
u-
tion
ofC
ivil
En
gin
eers
:E
ngi
nee
rin
gS
ust
ain
abil
ity
31
2S
oura
ni,
A,
Soh
ail,
M
Pu
bli
cp
rocu
re-
men
tin
centi
ves
for
sust
ain
able
des
ign
serv
ices
:S
wed
ish
exp
erie
nce
s
2009
Arc
hit
ectu
ral
engi
nee
rin
gan
dd
esig
nm
anag
emen
t
22
Sp
orro
ng
J.,
Bro
chn
erJ.
Wh
atfa
ctor
sin
flu
-en
ceth
eu
pta
keof
GP
P(g
reen
pu
bli
cp
rocu
rem
ent)
pra
c-ti
ces?
New
evi-
den
cefr
oman
Ital
-ia
nsu
rvey
2012
Eco
logi
cal
Eco
nom
ics
11
1T
esta
F.,
Iral
do
F.,
Fre
yM
.,D
add
iT
.
Con
tinu
edon
nex
tp
age
43
Bibliogra
phyofReviewed
Articles–co
ntinued
from
previouspage
Tit
leY
ear
Jou
rnal
Cit
atio
ns
Sco
pu
sC
itat
ion
sW
oSM
ean
cita
-ti
ons
Au
thor
s
Pro
cure
men
tfo
rsu
stai
nab
lelo
cal
econ
omic
dev
elop
-m
ent
2012
Inte
rnat
ion
alJou
rnal
ofP
ub
lic
Sec
tor
Man
agem
ent
11
Nij
aki
L.K
.,W
orre
lG
.
Su
stai
nab
ilit
yan
dlo
cal
food
pro
cure
-m
ent:
aca
sest
ud
yof
Fin
nis
hp
ub
lic
cate
rin
g
2012
Bri
tish
Food
Jou
rnal
20
1L
ehti
nen
,U
Gre
enp
ub
lic
pro
-cu
rem
ent
inp
rac-
tice
-T
he
case
ofN
orw
ay
2011
Soci
ety
and
Eco
nom
y1
1F
etA
.,M
ich
else
nO
.,B
oer
L.
Con
tinu
edon
nex
tp
age
44
Bibliogra
phyofReviewed
Articles–co
ntinued
from
previouspage
Tit
leY
ear
Jou
rnal
Cit
atio
ns
Sco
pu
sC
itat
ion
sW
oSM
ean
cita
-ti
ons
Au
thor
s
Th
eim
pac
tof
sus-
tain
able
pu
bli
cp
ro-
cure
men
ton
sup
-p
lier
man
agem
ent
-T
he
case
ofF
ren
chp
ub
lic
hos
pit
als
2012
Ind
ust
rial
Mar
keti
ng
Man
agem
ent
20
1O
ruez
abal
aG
.,R
ico
J.-
C.
Envir
onm
enta
lim
pac
tsan
dth
em
ost
econ
omi-
call
yad
vanta
geou
ste
nd
erin
pu
bli
cp
rocu
rem
ent
2012
Jou
rnal
ofP
ub
lic
Pro
-cu
rem
ent
00
Par
ikka
-A
lhol
aK
.,N
issi
nen
A.
Su
stai
nab
lep
ro-
cure
men
tin
hea
lth
and
soci
alca
rein
Nor
ther
nIr
elan
d
2012
Pu
bli
cM
oney
&M
anag
e-m
ent
00
0E
rrid
geA
.,H
enn
igan
S.
Con
tinu
edon
nex
tp
age
45
Bibliogra
phyofReviewed
Articles–co
ntinued
from
previouspage
Tit
leY
ear
Jou
rnal
Cit
atio
ns
Sco
pu
sC
itat
ion
sW
oSM
ean
cita
-ti
ons
Au
thor
s
Are
flec
tion
onth
eD
utc
hS
ust
ain
able
Pu
bli
cP
rocu
re-
men
tP
rogr
amm
e
2012
Jou
rnal
ofIn
tegr
ativ
eE
nvir
onm
en-
tal
Sci
ence
s
00
0M
elis
sen
,F
,R
ein
der
s,H
46
B. Authors
Author Publications
Preuss L. 4Parikka-Alhola K. 3Walker H. 3de Boer L. 2Michelsen, O 2Nissinen A. 2Bala A. 1Blume Y. 1Bolton P. 1Borg N. 1Brammer S. 1Brochner J. 1Daddi T. 1Davis, G 1Davis, K 1Ekroos, A 1Erridge A. 1Faninger-Lund H. 1Fet A. 1Finnveden, G 1Frey M. 1Gunther E. 1Hennigan S. 1Hochschorner, E 1Iraldo F. 1Irrek W. 1Jackson, T 1Kunzlik, P 1Lehtinen, U 1
Continued on next page
47
Table 10Authors – continued from previous page
Author Publications
Loprieno, AD 1Lund P. 1Melissen, F 1Munoz P. 1Nijaki L.K. 1Oosterhuis, F 1Oruezabala G. 1Otsuki K. 1Palmujoki, A 1Pindar A. 1Porta, PL 1Reinders, H 1Rico J.-C. 1Rieradevall J. 1Rita H. 1Scheibe L. 1Socolof, ML 1Sohail, M 1Sourani, A 1Sporrong J. 1Swanson, M 1Tarantini, M 1Testa F. 1Thomas S. 1Thomson, J 1van Asselt, H 1van der Grijp, N 1Weissman, A 1Worrel G. 1Ysern P. 1
Continued on next page
48
Table 10Authors – continued from previous page
Author Publications
B.1. Authorships per interval
Authorship 2001 - 2004 2005 - 2008 2009 - 2012Frequency Percent Frequency Percent Frequency Percent
Single author 1 100.0% 2 20.0% 3 16.7%Two authors 4 40.0% 10 55.6%Three authors 1 10.0% 4 22.2%Four authors 1 10.0% 1 5.6%Five authors 1 10.0%Seven authors 1 10.0%
Total 1 100% 10 100.0% 18 100.0%
C. Journals
Journal Publications Percent
Journal of Cleaner Production 3 10.3%Natural Resources Forum 2 6.9%Ecological Economics 2 6.9%Business Strategy and the Environment 2 6.9%The International Journal of Life Cycle Assess-ment
1 3.4%
Supply Chain Management: An InternationalJournal
1 3.4%
Society and Economy 1 3.4%Review of European Community & Interna-tional Environmental Law
1 3.4%
Public Money & Management 1 3.4%
Continued on next page
49
Table 11 – continued from previous page
Journal Publications Percent
Proceedings of the Institution of Civil Engi-neers: Engineering Sustainability
1 3.4%
Journal of Public Procurement 1 3.4%Journal of Integrative Environmental Sciences 1 3.4%Journal Of Environmental Planning And Man-agement
1 3.4%
Journal of Environmental Management 1 3.4%Journal of Environmental Law 1 3.4%International Journal of Public Sector Manage-ment
1 3.4%
International Journal of Production Economics 1 3.4%Industrial Marketing Management 1 3.4%Energy Policy 1 3.4%Energy 1 3.4%Corporate Social Responsibility and Environ-mental Management
1 3.4%
Climate Policy 1 3.4%British Food Journal 1 3.4%Architectural engineering and design manage-ment
1 3.4%
Total 29 100.0%
50
D. Publishing Countries
D.1. Publishing Countries per Interval
Publishing country 2001 - 2004 2005 - 2008 2009 - 2012Frequency Percent Frequency Percent Frequency Percent
Finland 2 20.0% 9 50.0%France 2 11.1%Germany 4 40.0%Italy 1 10.0% 7 38.9%Japan 1 5.6%Netherlands 3 30.0% 2 11.1%Norway 5 27.8%South Africa 1 10.0%Spain 4 40.0%Sweden 4 40.0% 2 11.1%UK 1 100.0% 5 50.0% 9 50.0%USA 5 50.0% 2 11.1%
Total 1 100.0% 29 290.0% 39 216.7%
E. Government Levels
E.1. Government Levels per Interval
2001 - 2004 2005 - 2008 2009 - 2012Government level Frequency Percent Frequency Percent Frequency Percent
Governmental 1 10.0%Municipal 1 10.0% 6 33.3%Local 4 40.0% 4 22.2%Unspecified 1 100.0% 4 40.0% 9 50.0%
Total 1 100.00% 10 100.0% 19 105.6%
51
E.2. Government Levels per Publishing Country
Publishing country Governmental Municipal Local Unspecified
Finland (11) 27.3% 72.7%France (2) 100.0%Germany (4) 50.0% 50.0%Italy (8) 50.0% 50.0%Japan (1) 100.0%Netherlands (5) 100.0%Norway (5) 40.0% 60.0%South Africa (1) 100.0%Spain (4) 100.0%Sweden (6) 33.3% 66.7%UK (15) 53.3% 46.7%USA (7) 71.4% 28.6% 28.6%
E.3. Government Levels per Paper Impact
Citation category Governmental Municipal Local Unspecified
0 - 10 (20) 25.0% 15.0% 65.0%10.5 - 20 (6) 16.7% 16.7% 50.0% 16.7%20.5 - 30 (3) 33.3% 66.7%
52
F. Product Types
F.1. Studied Product Types
Product type Frequency Percent
Goods 12 41.4%Services 10 34.5%Works 2 6.9%Unspecified 12 41.4%
Total 36 124.1%
F.2. Product Types per Publishing Country
Publishing country Goods Services Works Unspecified
Finland (11) 54.5% 45.5% 18.2% 27.3%France (2) 100.0% 100.0%Germany (4) 50.0% 50.0% 50.0%Italy (8) 100.0% 50.0% 12.5%Japan (1) 100.0%Netherlands (5) 100.0%Norway (5) 100.0%South Africa (1) 100.0% 100.0%Spain (4) 100.0%Sweden (6) 66.7% 33.3% 33.3%UK (15) 13.3% 20.0% 13.3% 66.7%USA (7) 100.0% 28.6%
53
F.3. Product Types per Paper Impact
Citation category Goods Services Works Unspecified
0 - 10 (20) 50.0% 45.0% 10.0% 30.0%10.5 - 20 (6) 33.3% 16.7% 50.0%20.5 - 30 (3) 100.0%
G. Industries and Sectors
G.1. Industries and Sectors per Interval
2001 - 2004 2005 - 2008 2009 - 2012Industry /sector
Frequency Percent Frequency Percent Frequency Percent
Construction 1 10.0% 1 5.6%Defensesector
1 10.0%
Professionalservices
1 5.6%
Services,not speci-fied
1 100.0% 1 10.0% 4 22.2%
Privatesector, notspecified
8 80.0% 12 66.7%
Manufacturing 1 5.6%Catering 2 11.1%Transportation 1 5.6%SMEs 1 10.0%
Total 1 100.0% 12 120.0% 22 122.2%
54
G.2.In
dustriesand
Sectors
perPublish
ingCountry
Pu
bli
shin
gco
untr
yC
onst
ruc-
tion
Def
ense
sect
orP
rofe
s-si
onal
ser-
vic
es
Ser
vic
es,
not
spec
ified
Pri
vate
sect
or,
not
spec
i-fi
ed
Man
ufa
c-tu
rin
gC
ater
ing
Tra
nsp
or-
tati
onS
ME
s
Fin
lan
d(1
1)18
.2%
27.3
%72
.7%
9.1%
18.2
%
Fra
nce
(2)
100.
0%10
0.0%
Ger
man
y(4
)50
.0%
100.
0%
Ital
y(8
)12
.5%
50.0
%62
.5%
37.5
%Jap
an(1
)10
0.0%
Net
her
-la
nd
s(5
)10
0.0%
Nor
way
(5)
100.
0%S
outh
Afr
ica
(1)
100.
0%10
0.0%
Sp
ain
(4)
100.
0%S
wed
en(6
)33
.3%
33.3
%33
.3%
33.3
%U
K(1
5)13
.3%
6.7%
66.7
%13
.3%
US
A(7
)28
.6%
100.
0%
55
G.3.In
dustriesand
Sectors
perPaperIm
pact
Cit
atio
nca
te-
gory
Con
stru
c-ti
onD
efen
sese
ctor
Pro
fes-
sion
alse
rvic
es
Ser
vic
es,
not
spec
i-fied
Pri
vate
sect
or,
not
spec
i-fied
Man
ufa
c-tu
ring
Cat
erin
gT
ransp
or-
tati
onSM
Es
0-
10(2
0)10
.0%
5.0%
5.0%
25.0
%65
.0%
5.0%
10.0
%5.
0%10
.5-
20(6
)16
.7%
83.3
%20
.5-
30(3
)66
.7%
33.3
%
56
H. Studied Countries
Studied country Frequency Percent
UK 9 32.1%Sweden 6 21.4%Finland 5 17.9%USA 4 14.3%Italy 3 10.7%France 2 7.1%Netherlands 2 7.1%Norway 2 7.1%EU 2 7.1%Austria 1 3.6%Brazil 1 3.6%Canada 1 3.6%Denmark 1 3.6%Estonia 1 3.6%Germany 1 3.6%Greece 1 3.6%Hungary 1 3.6%Ireland 1 3.6%Japan 1 3.6%Korea 1 3.6%Slovakia 1 3.6%South Africa 1 3.6%Spain 1 3.6%Poland 1 3.6%Western Europe 1 3.6%Eastern Europe 1 3.6%Scandinavia 1 3.6%Unspecified regions 1 3.6%
Total 54 192.9%
57
H.1. Home Bias
Publishing country Home bias
Finland 81.8%France 100.0%Germany 100.0%Italy 100.0%Japan 0.0%Netherlands 40.0%Norway 100.0%South Africa 100.0%Spain 100.0%Sweden 100.0%UK 93.3%USA 100.0%
I. Methodologies
I.1. Methodologies per Interval
Methodology 2001 - 2004 2005 - 2008 2009 - 2012Frequency Percent Frequency Percent Frequency Percent
Qualitative 1 100.0% 10 100.0% 12 66.7%
Quantitative 7 38.9%
Total 1 100.0% 10 100.0% 19 105.6%
58
I.2. Methodologies per Publishing Country
Publishing country Qualitative Quantitative
Finland (11) 54.5% 45.5%France (2) 100.0%Germany (4) 100.0%Italy (8) 50.0% 50.0%Japan (1) 100.0%Netherlands (5) 100.0%Norway (5) 60.0% 100.0%South Africa (1) 100.0%Spain (4) 100.0%Sweden (6) 66.7% 33.3%UK (15) 86.7% 13.3%USA (7) 100.0%
I.3. Methodologies per Paper Impact
Citation category Qualitative Quantitative
0 - 10 (20) 75.0% 30.0%10.5 - 20 (6) 83.3% 16.7%20.5 - 30 (3) 100.0%
J. Time Dimension
Time dimension Frequency Percent
Cross-sectional 24 82.8%Longitudinal 5 17.2%
Total 29 100.0%
59
J.1. Time Dimension per Interval
Time dimension 2001 - 2004 2005 - 2008 2009 - 2012Frequency Percent Frequency Percent Frequency Percent
Cross-sectional 1 100.0% 9 90.0% 14 77.8%Longitudinal 1 10.0% 4 22.2%
Total 1 100.0% 10 100.0% 18 100.0%
J.2. Time Dimension per Paper Impact
Citation category Cross-sectional Longitudinal
0 - 10 (20) 80.0% 20.0%10.5 - 20 (6) 83.3% 16.7%20.5 - 30 (3) 100.0%
K. Data Collection Methods
Data collection method Frequency Percent
Interview 13 44.8%Literature review, non-academic 13 44.8%Questionnaire 6 20.7%Secondary analysis 4 13.8%Literature review 3 10.3%Observation 1 3.4%Focus group 1 3.4%Unspecified 1 3.4%
Total 42 144.8%
60
K.1. Data Collection Methods per Interval
Data collec-tion method
2001 - 2004 2005 - 2008 2009 - 2012
Frequency Percent Frequency Percent Frequency Percent
Questionnaire 2 20.0% 4 22.2%Interview 6 60.0% 7 38.9%Secondaryanalysis
1 10.0% 3 16.7%
Observation 1 5.6%Literature re-view
2 20.0% 1 5.6%
Literaturereview, non-academic
1 100.0% 6 60.0% 6 33.3%
Focus group 1 5.6%Unspecified 1 5.6%
Total 1 100.0% 10 170.0% 18 133.3%
61
K.2.Data
Collection
Meth
odsperPublish
ingCountry
Publish
ing
countr
yQ
ues
tion
-nai
reIn
terv
iew
Sec
ondar
yan
alysi
sO
bse
rvat
ion
Lit
erat
ure
revie
wL
iter
ature
revie
w,
non
-ac
adem
ic
Focu
sgr
oup
Unsp
ecifi
ed
Fin
land
(11)
27.3
%27
.3%
9.1%
63.6
%F
rance
(2)
100.
0%G
erm
any
(4)
50.0
%10
0.0%
50.0
%50
.0%
Ital
y(8
)62
.5%
37.5
%12
.5%
Jap
an(1
)10
0.0%
Net
her
lands
(5)
100.
0%
Nor
way
(5)
40.0
%40
.0%
60.0
%60
.0%
Sou
thA
fric
a(1
)10
0.0%
Spai
n(4
)10
0.0%
Sw
eden
(6)
33.3
%66
.7%
33.3
%33
.3%
UK
(15)
26.7
%66
.7%
46.7
%13
.3%
USA
(7)
71.4
%28
.6%
28.6
%
62
K.3.Data
Collection
Meth
odsperPaperIm
pact
Cit
atio
nca
te-
gory
Ques
tion
-nai
reIn
terv
iew
Sec
ondar
yan
alysi
sO
bse
rvat
ion
Lit
erat
ure
revie
wL
iter
ature
revie
w,
non
-ac
adem
ic
Focu
sgr
oup
Unsp
ecifi
ed
0-
10(2
0)15
.0%
35.0
%15
.0%
5.0%
10.0
%40
.0%
5.0%
5.0%
10.5
-20
(6)
33.3
%50
.0%
16.7
%50
.0%
20.5
-30
(3)
33.3
%10
0.0%
33.3
%66
.7%
63
L. Research Strategies
Research strategy Frequency Percent
Single case study 12 41.4%Survey research 8 27.6%Multiple case study 5 17.2%Literature study 3 10.3%Meta-study 1 3.4%
Total 29 100.0%
L.1. Research Strategies per Interval
Research strategy 2001 - 2004 2005 - 2008 2009 - 2012Frequency Percent Frequency Percent Frequency Percent
Survey research 1 10.0% 7 38.9%Literature study 1 10.0% 2 11.1%Meta-study 1 5.6%Single case study 1 100.0% 4 40.0% 7 38.9%Multiple case study 4 40.0% 1 5.6%
64
L.2. Research Strategies per Publishing Country
Publishingcountry
Surveyresearch
Literaturestudy
Meta-study
Singlecasestudy
Multiplecasestudy
Finland (11) 27.3% 27.3% 27.3% 18.2%France (2) 100.0%Germany (4) 50.0% 50.0%Italy (8) 50.0% 37.5% 12.5%Japan (1) 100.0%Netherlands(5)
60.0% 40.0%
Norway (5) 40.0% 60.0%South Africa(1)
100.0%
Spain (4) 100.0%Sweden (6) 33.3% 33.3% 33.3%UK (15) 40.0% 33.3% 26.7%USA (7) 28.6% 71.4%
L.3. Research Strategies per Paper Impact
Citation cate-gory
Surveyresearch
Literaturestudy
Meta-study
Singlecasestudy
Multiplecasestudy
0 - 10 (20) 30.0% 15.0% 5.0% 45.0% 5.0%10.5 - 20 (6) 16.7% 50.0% 33.3%20.5 - 30 (3) 33.3% 66.7%
65
M. Topics
M.1. Topics per Paper Impact
Citation cate-gory
Supplierrelations
Selection Contracting Legal as-pects
PP tool
0 - 10 (9) 22.2% 33.3% 11.1% 33.3%10.5 - 20 (2) 50.0% 50.0%
M.2. Government Levels per Topic
Topic Governmental Municipal Local Unspecified
Supplier relations (2) 50.0% 50.0%Selection (3) 66.7% 33.3%Contracting (1) 100.0%Legal aspects (4) 100.0%PP tool (1) 100.0%
M.3. Product Types per Topic
Topic Goods Services Unspecified
Supplier relations (2) 50.0% 100.0%Selection (3) 66.7% 33.3%Contracting (1) 100.0% 100.0%Legal aspects (4) 25.0% 50.0% 50.0%PP tool (1) 100.0%
66
M.4. Industries and Sectors per Topic
Topic Professionalservices
Services,not speci-fied
Privatesector, notspecified
Catering Transpor-tation
Supplier relations (2) 50.0% 50.0% 50.0%Selection (3) 33.3% 33.3% 33.3%Contracting (1) 100.0% 100.0%Legal aspects (4) 50.0% 75.0%PP tool (1) 100.0%
67
M.5.CountriesStu
died
perTopic
Top
icB
razi
lD
enm
ark
Fin
land
Fra
nce
Net
her
lands
Nor
way
Sou
thA
fric
aSw
eden
USA
EU
Supplier
rela
tion
s(2
)50
.0%
50.0
%Sel
ecti
on(3
)33
.3%
33.3
%10
0.0%
Con
trac
ting
(1)
100.
0%10
0.0%
Leg
alas
pec
ts(4
)25
.0%
25.0
%25
.0%
25.0
%25
.0%
PP
tool
(1)
100.
0%
68
M.6. Methodologies per Topic
Topic Qualitative Quantitative
Supplier relations (2) 100.0%Selection (3) 100.0%Contracting (1) 100.0%Legal aspects (4) 100.0% 25.0%PP tool (1) 100.0%
M.7. Time Dimension per Topic
Topic Cross-sectional Longitudinal
Supplier relations (2) 50.0% 50.0%Selection (3) 66.7% 33.3%Contracting (1) 100.0%Legal aspects (4) 100.0%PP tool (1) 100.0%
69
M.8.Data
Collection
Meth
odsperTopic
Top
icQ
ues
tion
nai
reIn
terv
iew
Sec
ondar
yan
alysi
sL
iter
ature
revie
w,
non
-aca
dem
icU
nsp
ecifi
ed
Supplier
rela
tion
s(2
)50
.0%
50.0
%Sel
ecti
on(3
)33
.3%
33.3
%33
.3%
Con
trac
ting
(1)
100.
0%L
egal
asp
ects
(4)
25.0
%10
0.0%
PP
tool
(1)
100.
0%
70
M.9. Research Strategies per Topic
Topic Surveyresearch
Literaturestudy
Meta-study
Singlecasestudy
Supplier rela-tions (2)
50.0% 50.0%
Selection (3) 33.3% 33.3% 33.3%Contracting(1)
100.0%
Legal aspects(4)
100.0%
PP tool (1) 100.0%
71