Mistakes encountered in manuscripts on educationand their effects on journal rejections

Embiya Celik • Nuray Gedik • Guler Karaman • Turgay Demirel •

Yuksel Goktas

Received: 18 June 2013 / Published online: 2 October 2013� Akademiai Kiado, Budapest, Hungary 2013

Abstract The goal of this study was to identify common mistakes made in research study

manuscripts submitted to journals of Education and the effects of these mistakes on rejection

by the journal editors and referees. An online questionnaire was developed for this purpose

with 43 items and five open-ended questions. Common mistakes were identified by admin-

istering the 43 questions, which were to be answered in two stages: first by using 5-point

Likert scale responses, and then by responses arranged according to semantic differential

scale (for the effects of the mistakes on rejections). The online questionnaire was sent to the

editors and referees of Turkish journals of Education indexed in SSCI and ULAKBIM. Data

were then collected from 232 participants and examined. The quantitative data obtained from

the questionnaire items were analyzed, and the mean and standard deviation scores were

presented in tables. The qualitative data gathered from the open-ended questions were ana-

lyzed descriptively. The results show that researchers mostly make mistakes in the discus-

sion, conclusion, and suggestions part of the manuscripts. However, mistakes made in the

methods part are the most significant causes of manuscript rejection.

Keywords Reasons for manuscript rejection � Manuscript review � Mistakes

in education manuscripts � Manuscript review process

Introduction

A scientific paper is a report that defines the results of original research (Day 1997). To be

published, a scientific paper should be the first report of these results and should derive

E. Celik � G. Karaman � T. Demirel � Y. Goktas (&)Department of Computer Education and Instructional Technology, Ataturk University, 25240 Erzurum,Turkeye-mail: yuksel.goktas@hotmail.com

N. GedikDepartment of Computer Education and Instructional Technology, Akdeniz University, 07058 Antalya,Turkeye-mail: ngedik@akdeniz.edu.tr

123

Scientometrics (2014) 98:1837–1853DOI 10.1007/s11192-013-1137-y

from an original study; also, the same results should be able to be replicated if the study is

repeated (Seckin 2003). No matter how original and striking the results of research studies

may be, their general acceptance will be controversial unless they are shared and the

process used to reach them is adequately described, so the results can be scrutinized by

others (Asan 2006). For many years, research studies have generally been presented to the

scientific community via science journals (Day 1997). It is a challenging process for

researchers to prepare such publications. Once a formal report has been written, the

researchers send it to an academic journal that is specific to their field. Following peer-

reviews, studies that are found to be acceptable in this first stage are then sent to the journal

referees, who are asked to review and evaluate these studies in more detail. The reasons for

this process are to guarantee the acceptance of only well and responsibly conducted, and

clearly presented studies; to help shape the further development of weaker studies, so that

they may be revised and resubmitted; and to reject studies that are seriously flawed

(Bornmann and Daniel 2010; Wijnhoven and Dejong 2010).

Although the evaluation of scientific research reports is an important process, the

evaluation criteria have not yet been standardized in all research areas. Many journals have

their own evaluation criteria. Very few of the scientific research studies that are conducted

pass the evaluation criteria of the journals successfully and get published after the first

submission (Summers 2001). Studies that fail the evaluation criteria are either rejected or

reevaluated after the necessary corrections are made.

A review of the literature revealed that there are many mistakes which can cause scientific

research studies to fail typical journal evaluation criteria. Examining such mistakes in two

categories (formal mistakes and content-related mistakes), Arikan (2009) defined formal

mistakes as elements of the presentation that are out of compliance with a journal’s for-

matting standards and norms, and content-related mistakes as those made in such sections of

the study as the introduction, the methods description, and the discussion of the findings.

Formal mistakes can be resolved by small-scale adjustments, like the correction of linguistic

errors, the re-arrangement of tables and graphics, and the re-formatting of the manuscript to

comply with the publication guidelines of the journal. But the content-related mistakes can

require large-scale adjustments, such as improving the introduction, adding further orga-

nization or discussion of the methods, or using new or additional statistical analyses and

providing reinterpretations of the findings (Belcher 2009; Arikan 2009). If these adjustments

are not done properly, the study will be rejected. Sonmez (2005) also asserted that any one

mistake made in scientific research can result in other mistakes. For example, a mistake in

the introduction, specifically in the section on the goals of the study or regarding the

statement of the problem to be investigated, could also lead to mistakes and inconsistencies

in other parts, like the methods section, the sample selection, the discussion of the data

collection instruments, or the conduct of the data analysis. The structure of the study should

be built around the research problem and goal. Price and Maushak (2000) stated that the most

frequent mistakes encountered by journal editors are the manuscript’s lack of suitability for

the target audience of the journal, a weak topic choice, an inappropriate choice of methods,

poor writing, weakness in the quality of the research, and/or ethical violations such as

plagiarism. Sample mistakes that were considered to be the most frequent ones in scientific

research of various fields are summarized in Table 1.

The mistakes that stand out the most in Table 1 are ‘‘difficulties regarding the cohesion

and writing quality of the manuscript’’, ‘‘linguistic, presentation, and abbreviation mistakes

in the writing’’, ‘‘improper/inadequate method of analysis and/or use of statistics’’, and

‘‘inappropriate and outdated review of literature.’’ Due to similar mistakes, many scientific

research studies cannot pass the referees’ review and thus get rejected. Studies on these

1838 Scientometrics (2014) 98:1837–1853

123

Ta

ble

1C

om

mo

nM

ista

kes

inS

cien

tifi

cR

esea

rch

and

Rea

son

sfo

rR

ejec

tio

n

Aja

o(2

00

5)

Au

dis

ioet

al.

(20

09)

Bord

age

(20

01)

Cher

nic

k(2

00

8)

Fis

cher

(20

04)

Gu

pta

etal

.(2

00

6)

Hes

s(2

00

4)

McK

erch

eret

al.

(20

07)

Pie

rso

n(2

00

4)

Ulu

og

lu(2

01

0)

Th

em

anu

scri

pt

con

ten

tis

ou

tsid

eth

esc

op

eo

fth

ejo

urn

al4

44

44

Th

ere

sear

chto

pic

isn

ot

up

-to-d

ate,

inte

rest

ing,

or

ori

gin

alen

ough

44

44

Dif

ficu

ltie

sre

gar

din

gth

eco

hes

ion

and

wri

tin

gq

ual

ity

of

the

man

usc

rip

t4

44

44

4

No

tco

nfo

rmin

gto

the

auth

or

gu

idel

ines

of

the

journ

al4

44

Lin

gu

isti

c,p

rese

nta

tio

n,

and

abb

rev

iati

on

mis

tak

esin

the

wri

tin

g4

44

44

Eth

ical

vio

lati

ons,

such

aspla

gia

rism

,fa

lsifi

cati

on,

and/o

rfa

bri

cati

on

44

Th

eto

pic

and

the

sig

nifi

can

ceo

fth

ere

sear

char

en

ot

exp

lain

edcl

earl

y4

4

Th

ere

sear

chg

oal

,ra

tion

ale,

and

pro

ble

mar

en

ot

exp

lain

edcl

earl

y4

44

4

Inap

pro

pri

ate

and

ou

tdat

edre

vie

wo

fli

tera

ture

44

44

4

Imp

rop

er/f

ault

y/p

oor

rese

arch

des

ign

44

44

4

Inef

fici

ent

pre

cauti

ons

toen

sure

the

val

idit

yan

dre

liab

ilit

yo

fth

ere

sear

ch4

44

Inad

equ

acy

of

the

sam

ple

inn

um

ber

44

Dat

aco

llec

tio

nin

stru

men

ts/d

ata

no

tsu

itab

lefo

rth

ere

sear

chg

oal

44

4

Imp

rop

er/i

nad

equ

ate

met

ho

do

fan

alysi

san

d/

or

use

of

stat

isti

cs4

44

44

4

Un

suit

abil

ity

of

the

stru

ctu

res/

des

crip

tio

ns

of

the

tab

les

and

fig

ure

s4

44

Scientometrics (2014) 98:1837–1853 1839

123

Ta

ble

1co

nti

nued

Aja

o(2

00

5)

Au

dis

ioet

al.

(20

09)

Bord

age

(20

01)

Cher

nic

k(2

00

8)

Fis

cher

(20

04)

Gu

pta

etal

.(2

00

6)

Hes

s(2

00

4)

McK

erch

eret

al.

(20

07)

Pie

rso

n(2

00

4)

Ulu

og

lu(2

01

0)

Rep

etit

ion

of

the

findin

gs/

pre

senta

tion

of

just

the

dat

a4

44

Inte

rpre

tati

ons

indep

enden

to

fth

efi

ndin

gs

44

Ov

er-i

nte

rpre

tati

on,

no

tb

ased

on

rese

arch

fin

din

gs

44

4

1840 Scientometrics (2014) 98:1837–1853

123

problems also indicate that most published articles had been rejected by one or more

journals previously (Bornmann & Daniel 2010; Turcotte et al. 2004). In their research,

Ehara and Takahashi (2007) stated that the rejection rates for manuscripts range from 22.6

to 73.4 %.

There are a limited number of research studies that discuss mistakes in scientific studies

or the reasons for journal rejections. The majority of such studies have been conducted in

the field of Healthcare (e.g. Ann Ibadan Postgrad Med, Cerebrovascular Diseases, Surgical

Oncology, Academic Medicine, Pediatric Pulmonology, American Journal of Roentgen-

ology, Indian Pediatrics, Respiratory Care, Respir Care, Archives of Iranian Medicine,

Canadian Journal of Anaesthesia-Journal Canadien D Anesthesie, Journal of Environ-

mental and Occupational Medicine, British Journal of Surgery).But in the field of Edu-

cation, further study of this topic is still needed. To remedy that situation, the goal of this

study was to determine the common mistakes that are made in research studies that are

submitted to journals of Education, and the effects of these mistakes on decisions con-

cerning the acceptance or rejection of the manuscripts by journal editors and referees. The

Turkish journals of Education were taken into consideration in the study in an effort to

present the current status of a country where the productivity of publishing has recently

increased dramatically (Tseng et al. 2013). This study is significant, because it presents

important points to take into consideration when preparing research studies on Education

for publication. The results also may help researchers in the field to conduct higher quality

research studies.

The following research questions guided this study:

1. From the perspectives of the journal editors and referees,

a. what mistakes are commonly seen in scientific research studies submitted to the

Turkish journals of Education, which are indexed in SSCI and Turkish Academic

Network and Information Center (ULAKBIM)?

b. to what extent do these mistakes lead to the rejection of manuscripts?

2. Is there a significant difference between the viewpoints of journal editors and referees

regarding what constitutes mistakes in scientific research studies that are submitted to

Turkish journals of Education indexed in SSCI and ULAKBIM, or regarding the

effects of these mistakes on rejection?

Methods

A survey was used for this study. The data were collected by means of questionnaires

(Karasar 2005; McMillan & Schumacher 2010). During the analysis of the gathered data,

descriptive statistical methods and an independent t test, which is one of the inferential

statistical methods, were used.

The data collection instrument

A questionnaire was used in the study, because the sample group contained a high number

of participants, who were widely dispersed geographically (McMillan & Schumacher

2010). The questionnaire was developed by the researchers. Firstly, the item pool was

constructed. This was based upon scientific articles in the literature, books on research

methods, and the referee/editor evaluation forms of selected journals. The questionnaire

Scientometrics (2014) 98:1837–1853 1841

123

items were divided into five parts, corresponding to the following typical paper sections:

introduction, methods, findings, and discussion-conclusion-suggestions, with the fifth part

devoted to general reasons for manuscript rejections. Then, a peer-reviewed pilot was

conducted with eight doctoral students. The necessary corrections were made, based on the

feedback gathered, and expert opinions were elicited. The questionnaire was reviewed by

five experts (two of them are measurement and evaluation experts, three of them are

educational technology experts), all of whom have a doctoral degree in Education, and

work as journal referees and editors. The questionnaire was submitted to the experts, one at

a time, and then adjusted, based on their overall feedback. After the necessary corrections

were made, the questionnaire was checked by a Turkish language expert, and was edited to

its final form. In the questionnaire, there are 43 scale response items and 5 open-ended

questions. Two different types of data were collected from each questionnaire item, except

for the five open-ended questions. Firstly, a 5-point Likert scale (1- never, 2- rarely, 3-

sometimes, 4- often, 5- always) was used to collect data regarding how frequently each

type of specified mistake was made. Secondly, a 5-point semantic differential scale

(1 = Not at all, 5 = A great deal) was used with the same set of questions to gather data

about the effects of these specified mistakes on journal rejections.

The participants

The participants included the editors and referees of 32 Turkish Journals of Education

which were indexed in SSCI and ULAKBIM as of February, 2012. ULAKBIM is a

research and development center within The Scientific and Technological Research

Council of Turkey. Since the researchers’ purpose was to reach the whole population, a

sample selection was not used. The included journals were each reviewed by the

researchers, and their editors and referees were listed. A total of 993 editors’ and referees’

e-mail addresses were found using search engines. The questionnaire was transferred to the

web environment, and was sent to these email addresses during a four-week period. The

participants were asked in the email message for their cooperation. The questionnaire then

remained online for three weeks. Data were provided for the questionnaire by 232 of the

addressees, who became the participants in the study. 188 of the participants (81.03 %)

were referee, and 44 of the participants (18.97 %) were editor.

The data analysis

The quantitative data were analyzed using SPSS 20 statistical software, descriptive sta-

tistical methods (percentages, means, and standard deviations), and an independent t-test

based in inferential statistical methods. The data belonging to each of the five parts of the

questionnaire were then arranged for presentation in tables.

Descriptive analysis was used to analyze the data collected from the open-ended

questions. In this process, the researchers made a sincere attempt to be faithful to the

original form of the data (as much as was possible). This data is presented below in the

form of direct quotations taken from the statements made by the participants.

Validity and reliability

Various precautions were taken to ensure the validity and reliability of the study. These

precautions include the peer-review, the expert opinions, and professional inspection of the

1842 Scientometrics (2014) 98:1837–1853

123

language used for the questionnaire. Also, detailed descriptions of the methods and the

roles of the researchers have been provided, to further ensure the validity of the study. A

reliability analysis for the questionnaire was conducted for each part separately. Cron-

bach’s Alpha (a) coefficients for both the mistakes in manuscripts and their effects on

rejection are shown in Table 2.

As seen in Table 2, the reliability coefficients of the questionnaire parts are quite high.

The lowest is 0.77, and the highest is 0.93. These values mean that the developed ques-

tionnaire has a high level of reliability.

The roles of the researchers

The study was conducted by two academics in the field of Instructional Technology, who

are also the referees of many national and international journals, and three doctoral stu-

dents. The researchers all exchanged ideas during the process. The academics asked for

expert opinions, transmitted the questionnaire to the participants, and evaluated the ana-

lysis results. The doctoral students were in charge of constructing the item pool of the

questionnaire, reaching the editors and referees of the journals by email, transferring the

questionnaire to the web environment, administering the questionnaire, and analyzing the

data.

The findings

The common mistakes in scientific research and the effects of these mistakes on the

rejection of manuscripts were investigated under seven headings. The subheadings are: the

whole manuscript; introduction; methods; findings; discussion, conclusion, and sugges-

tions; mean values of all the parts; and comparison of the referees’ and editors’ views. In

the tables created for the first six subheadings, the items were sequenced according to the

mean values of the mistakes, and the mean values ( �X) and standard deviations (SD)

belonging to these items were provided. The results of the independent t-test were pre-

sented in a table under the last subheading. Then, the answers to the open-ended questions

were supplied under each relevant part to support the quantitative data.

General mistakes made in the whole manuscript and their effects on rejection

The analysis results are displayed in Table 3. The most common mistake encountered in

the whole manuscript was ‘‘unoriginality of the research topic’’ ( �X = 3.44), and the least

common mistake was ‘‘existence of ethical violations, such as plagiarism, falsification, and

fabrication or suspicion of these ethical violations’’ ( �X = 2.28).

However, when the data on the effects of the mistakes on rejection were examined, the

sequence changed. The item ‘‘existence of ethical violations, such as plagiarism, falsifi-

cation, and fabrication or suspicion of these ethical violations’’ ( �X = 4.44) turned out to be

the mistake that had the most effect on rejection. The item ‘‘lack of an impressive title

appropriate for the research topic’’ ( �X = 2.18) had the least impact on the rejection of a

manuscript.

In the answers to the open-ended questions, the participants generally preferred to

request corrections rather than definitely rejecting the manuscripts due to the mistakes in

this part. One participant said:

Scientometrics (2014) 98:1837–1853 1843

123

..although some mistakes are important, correction can be [requested to resolve] this

issue; therefore, their effects on rejection have been indicated to be low (e.g., 6, 7,

10, etc.)…

Mistakes made in the introduction part of the manuscript and their effects on rejection

The common mistakes made in the introduction part of the manuscripts are illustrated in

Table 4. The most common mistake was ‘‘weakness of theoretical foundations of the

research’’ ( �X = 3.45), while the least common mistake was ‘‘unsuitability of the research

hypotheses and/or questions for the goal of the study’’ ( �X = 2.95). Regarding the effects of

the mistakes on rejection, the mistake with the greatest effect was the item ‘‘unsuitability of

the research hypotheses and/or questions for the goal of the study’’ ( �X = 3.84), and the

mistake with the least impact on the rejection of manuscripts was ‘‘not indicating the

significance and rationale of the research clearly’’ ( �X = 3.38).

Table 2 Reliability results for the parts of the questionnaire

Parts Number of items Cronbach’s alpha coefficient forinternal consistency (a)

Mistakes Effects on rejection

General reasons part 12 0.77 0.82

Introduction part 5 0.85 0.89

Methods part 15 0.92 0.93

Findings part 5 0.83 0.84

Discussion, conclusion, and suggestions part 7 0.85 0.91

Total 44 0.84 0.88

Table 3 General mistakes made in the whole manuscript and their effects on rejection

Items belonging to general reasons Perceivedmistakefrequency

Effects onrejection

�X SD �X SD

3 Unoriginality of the research topic 3.44 0.83 3.65 1.10

4 Lack of cohesion in the study 3.23 0.82 3.84 1.03

2 Outdated and uninteresting research topic 3.16 0.73 3.30 1.10

7 Existence of many spelling and punctuation errors in the manuscript 3.00 0.91 2.28 1.09

6 Incompatibility of the formal features of the study with the authorguidelines of the journal (in-text citations, reference list, font style, etc.)

2.96 0.96 2.61 1.32

5 Difficulties in reading the manuscript 2.91 0.82 3.31 1.19

10 Lack of an impressive title appropriate for the research topic 2.72 0.83 2.18 1.07

11 Abstract does not reflect the research 2.59 0.94 2.31 1.18

1 The manuscript content is outside the scope of the journal 2.33 0.67 3.66 1.38

9 Not conforming to ethical guidelines (receiving approval from ethicscommittees, getting permission for the voluntary participation of theresearch participants)

2.32 0.98 3.67 1.44

8 Existence of ethical violations, such as plagiarism, falsification, andfabrication or suspicion of these ethical violations

2.28 0.97 4.44 1.27

1844 Scientometrics (2014) 98:1837–1853

123

Regarding the open-ended questions in this part, the participants agreed that the mistakes

mentioned in these items can usually be corrected. However, the participants emphasized that

deficiencies in the literature review are of critical importance. One participant stated:

If the literature review part of the study is inefficient or outdated, I ask for correction/

rewriting, but I do not reject it.

Mistakes made in the methods part of the manuscript and their effects on rejection

The mistakes made in the methods part are shown in Table 5. The mistake that was

encountered most frequently was ‘‘unclear and undetailed research method’’ ( �X = 3.24).

The least common mistake was ‘‘not specifying the population and the sample’’

( �X = 2.65).

Regarding the impacts of these mistakes on rejection, the item ‘‘unsuitability of the

selected method for the goal of the study’’ ( �X = 4.27) had the greatest impact. The mistake

with the least effect on rejection was ‘‘not mentioning the role of the researchers in the

research process’’ ( �X = 2.74).

Regarding the answers for the open-ended questions, ‘‘inadequacy of the sample to

represent the population’’ was emphasized. The participants suggested that the word

‘‘sample’’ be changed to ‘‘study group’’ if the sample did not represent the population.

Another comment focused on mistakes made in the data analysis and discussion parts of

qualitative research studies. The following quotes illustrate these points:

If the sample does not represent the population, I suggest changing [the term to]

study group.

There are a lot of mistakes especially in the data analysis and discussion parts of

qualitative research studies. I have found no qualitative data analysis, presentation, or

discussion section in any manuscript received to date to be adequate.

Mistakes made in the findings part of the manuscript and their effects on rejection

The mistakes made in the findings part of the manuscripts are displayed in Table 6. The

most common mistake was ‘‘inadequacy of findings to answer the research questions’’

( �X = 2.98), and the least common mistake was ‘‘unsuitability of the findings for the

research questions’’ ( �X = 2.82).

Concerning the effects of the mistakes on rejection, the item ‘‘unsuitability of the findings for

the research questions’’ ( �X = 4.01) had the greatest effect on rejection, and the item ‘‘exis-

tence of redundant details, tables, and figures’’ ( �X = 2.67) had the least effect on rejection.

Regarding the open-ended questions, the participants replied that they usually request

corrections for the mistakes made in this part. The most important criticism of this section

was that our questionnaire items focused only on the aspects of quantitative and qualitative

data presented in research studies evaluated by journals; review articles also should have

been included in the questionnaire items for this part. This is a limitation in the present

study. To illustrate, one participant commented:

The questions in this part cover only practices concerning qualitative and quantita-

tive research studies. However, some articles have the characteristics of review

articles. What about our evaluations regarding these articles?

Scientometrics (2014) 98:1837–1853 1845

123

Mistakes made in the discussion, conclusion, and suggestions part of the manuscripts

and their effects on rejection

The results for this part are displayed in Table 7. The most common mistake encountered

in this part of the manuscripts was ‘‘not discussing the topic with reference to the relevant

literature (parallel and opposing views), and/or the discussion is not based on the research

Table 4 Mistakes made in the introduction part of the manuscript and their effects on rejection

Items belonging to the introduction part Perceivedmistakefrequency

Effects onrejection

�X SD �X SD

12 Weakness of the theoretical foundations of the research 3.45 0.74 3.81 1.08

14 Not indicating the significance and rationale of the research clearly 3.19 0.74 3.38 1.12

13 Not indicating the goal of the study clearly 3.11 0.75 3.46 1.11

16 Unsuitability of the literature review for the goal of the study/researchquestions

2.98 0.84 3.63 1.14

15 Unsuitability of the research hypotheses and/or questions for the goal ofthe study

2.95 0.74 3.84 1.16

Table 5 Mistakes made in the methods part of the manuscript and their effects on rejection

Items belonging to the methods part Perceivedmistakefrequency

Effects onrejection

�X SD �X SD

18 Unclear and undetailed research method 3.24 0.76 3.78 1.02

25 Not specifying the features of the data collection instruments (structural,content-related, etc.) in detail

3.22 0.80 3.54 1.07

26 Not taking the measures of validity and reliability for the data collectioninstruments

3.19 0.85 4.06 1.01

20 Unclear and incomprehensible sample selection method 3.13 0.83 3.54 1.03

27 Not explaining the data collection process in detail 3.13 0.81 3.34 1.05

30 Not mentioning the role of the researchers in the research process 3.03 1.06 2.74 1.14

29 Unsuitability of the statistical methods used for the research questions orthe data collected

3.01 0.87 4.21 0.94

22 Inadequacy of the sample to represent the population 2.97 0.86 3.73 1.17

31 Not taking the measures of validity and reliability in the data collectionprocess/for the conditions of the research practices

2.96 0.86 3.54 1.16

17 Unsuitability of the selected method for the goal of the study 2.91 0.8 4.27 0.94

28 Not explaining the method of data analysis 2.91 0.87 3.56 1.14

23 Unclear demographic information about the sample 2.88 0.83 3.05 1.11

24 Unsuitability of the data collection instruments for the goal of the study 2.83 0.95 4.10 1.10

21 Unsuitability of the sample for the goal of the study 2.80 0.9 3.92 1.11

19 Not specifying the sample and the population 2.65 0.93 3.56 1.17

1846 Scientometrics (2014) 98:1837–1853

123

questions and findings’’ ( �X = 3.60), while the least common mistake was ‘‘not specifying

the limitations of the research’’ ( �X = 3.21).

Regarding the effects of the mistakes on rejection, the item ‘‘lack of results and

inferences appropriate for the research findings’’ ( �X = 3.81) was regarded as the mistake

that had the greatest impact on rejection. The item ‘‘lack of suggestions for future

research’’ ( �X = 2.69) was the mistake that had the least effect on rejection.

Regarding the open-ended questions, the participants most frequently commented that a

big problem they had observed was that researchers, in writing the discussion and con-

clusion sections, did not fully grasp the meaning or implications of their own findings. The

answers provided here also indicated that research findings were too often simply repeated

in this part. Concerning this point, one participant said:

The discussion and conclusion often indicate that the findings were not understood.

[Instead,] the findings are [merely] repeated in the discussion and conclusion.

Unfortunately, comparisons, analysis, and interpretations of the findings based on the

literature [often] are not provided.

The mean values for the all of the parts on mistakes made in the manuscripts and their

effects on rejection

According to the findings, the parts in which mistakes are made the most are ‘‘dis-

cussion, conclusion, and suggestions’’ ( �X = 3.38), while the part in which mistakes are

encountered least frequently is ‘‘general reasons’’ ( �X = 2.81). Concerning the effects of

the mistakes on rejection, the mistakes made in the ‘‘methods part’’ ( �X = 3.66) had the

greatest effect on the rejection of the manuscripts, while those made in the ‘‘general

reasons’’ part ( �X = 3.04) had the least impact on rejection. The mean values of the

questionnaire sections are illustrated in Table 8.

Differing editor and referee views regarding the questionnaire items on the mistakes

made in the manuscripts and their effects on rejection

In Table 9, the results of the comparison of the editors’ and the referees’ opinions con-

cerning the mistakes in manuscripts and their impacts on rejection are provided for the

second research question.

Table 6 Mistakes made in the findings part of the manuscript and their effects on rejection

Items belonging to the findings part Perceived mistakefrequency

Effects onrejection

�X SD �X SD

33 Inadequacy of the findings to answer the research questions 2.98 0.84 3.97 0.98

36 Existence of subjective expressions/judgments in the findings 2.89 0.89 2.97 1.19

34 Existence of redundant details, tables, and figures 2.88 0.90 2.67 1.07

35 Unsuitability of the descriptions of the tables and figures 2.88 0.87 2.88 1.13

32 Unsuitability of the findings for the research questions 2.82 0.86 4.01 1.07

Scientometrics (2014) 98:1837–1853 1847

123

The findings indicate that there is a significant difference in only the mean values

belonging to the item ‘‘the manuscript content is outside the scope of the journal,’’ with the

editors rating higher on this point [t(2,21) = -2.199, p \ 0.05]. The views of the editors

and referees have similar mean values in the other items. No significant difference was

found between the questionnaire items in either the mistakes made or the effects of these

mistakes on rejection.

Discussion, conclusion, and suggestions

In light of the gathered data, among the items overall, ‘‘unoriginality of the research topic’’

(Ehara and Takahashi 2007; WeiWei et al. 2009; Bornmann et al. 2010), ‘‘lack of cohesion

in the research’’ (Uluoglu 2010; Chernick 2008; Ajao 2005; Audisio et al. 2009; Fischer

2004), and ‘‘outdated and uninteresting research topic’’ are the most prominent general

mistakes made in education research studies according to Turkish editors and referees.

Table 7 Mistakes made in the discussion, conclusion, and suggestions part of the manuscripts and theireffects on rejection

Items belonging to the discussion, conclusion, and suggestions part Perceivedmistakefrequency

Effects onrejection

�X SD �X SD

39 Not discussing the topic with reference to the relevant literature (paralleland opposing views), and/or the discussion is not based on the researchquestions and findings

3.60 0.84 3.69 1.05

37 Repeating the findings in the discussion 3.50 0.83 3.01 1.19

40 Lack of results and inferences appropriate for the research findings 3.40 0.77 3.81 1.04

38 Comments that are exaggerated and/or independent of the findings 3.36 0.85 3.54 1.16

43 Lack of suggestions about the applicability of the research results 3.35 0.83 2.84 1.09

42 Lack of suggestions for future research 3.22 0.86 2.69 1.15

41 Not specifying the limitations of the research 3.21 0.88 3.06 1.14

Table 8 The mean values for all of the parts on mistakes made in the manuscripts and their effects onrejection

Parts Perceived mistake frequency Effects on rejection

�X SD �X SD

Discussion, conclusion, and suggestions part 3.38 0.14 3.23 0.44

Introduction part 3.14 0.20 3.62 0.21

Methods part 2.99 0.17 3.66 0.42

Findings part 2.89 0.06 3.30 0.64

General reasons 2.81 0.40 3.04 0.91

1848 Scientometrics (2014) 98:1837–1853

123

Although these items are the most common mistakes made in the studies, only ‘‘lack of

cohesion in the research’’ was stated to be affecting the rejection of a study to a great

extent. Notably, Shakiba et al. (2008) asserted that one of the most important criteria for

the publication of manuscripts is good writing. This lack can be aligned to a language

problem since writing the manuscripts in a foreign language (i.e., English is required for

many Turkish journals indexed in SSCI) can be a hindering effect for authors to write

good. However, as Tseng et al. (2013) and Vinluan (2012) pointed out, these journals

accept manuscripts written in mother language (i.e., Turkish) as well.

Correction was found to be generally preferred for the mistakes made due to general

reasons. However, the two items that have the most impact on the rejection of research

studies both involve ethical principles. Today, the most significant problem in the science

world is ethical violations (Bornmann et al. 2008). Such ethical violations as plagiarism,

falsification, and fabrication are mistakes that are not forgiven by journal editors and

referees. Scientific ethics and plagiarism are issues that are increasingly emphasized in the

literature, though they are not emphasized enough in Turkey (Ucak & Birinci 2008). It is

notable that these items are the mistakes made in submitted research studies least fre-

quently. This seems to indicate that researchers are conscious and cautious of the rules of

scientific ethics.

The introduction of an article manuscript is the part that consists of a succinct statement

about the study, the research topic, its significance, the rationale, hypotheses, and ques-

tions. According to the gathered data, the items with the highest value among the common

mistakes made in the introduction part of submitted manuscripts mostly have low impacts

on rejection. Thus, the most common mistakes can usually be corrected without automatic

rejection. But the item ‘‘weakness of the theoretical foundations for the research’’ featured

high values both as one of the most common mistakes made by researchers and for its

effect on rejection. Aside from providing suitable and up-to-date information about the

study and the research problem (which is the point of departure for the study), the intro-

duction should also contain a pertinent and thorough review of relevant literature to pro-

vide the study with a strong theoretical structure and context. The theoretical background

drawn from the literature also supports the argument for the significance of the researcher’s

own study (i.e., the study should fill an unaddressed gap in the literature, or take the next

step by building on previous findings). Conducting the literature review while devising a

study also assists the researcher to map or modify his/her developing study. A poor

literature review has been regarded as a reason for rejection in many studies (Bordage

2001; Fischer 2004; McKercher et al. 2007; Audisio et al. 2009; Uluoglu 2010). Another

item in the introduction part with a high effect on rejection is ‘‘unsuitability of the research

hypotheses and/or questions for the goal of the study.’’ If these points constituting the basis

of the study are not explained clearly and comprehensively, the readers may not be able to

properly understand the study. Each person reading the research questions should infer the

same meaning (Sonmez 2005).

Table 9 Comparison of the editors’ and referees’ views about the mistakes in the manuscripts and theireffects on rejection

‘‘The manuscript content is outside the scope of the journal’’ N �X SD df t p

Referee 188 3.55 1.38 2.21 -2.199 0.029

Editor 44 4.07 1.37

Scientometrics (2014) 98:1837–1853 1849

123

The ratings for the effects of mistakes made in the ‘‘methods part’’ are higher compared to

the other parts. Hence, manuscripts are most frequently rejected due to mistakes in the

methods part. Such mistakes reflect the researchers’ inadequate knowledge of appropriate

research methods (Sayin 2008). McKercher et al. (2007) stated that referees encounter

methods problems in manuscripts most frequently. Turcotte et al. (2004) stated that methods

are the main factor that determines the destiny of research study. The primary mistake among

the common mistakes made in the methods part is ‘‘unclear and undetailed research method.’’

This finding is also supported in the literature (Audisio et al. 2009). A clear and detailed

description of the research methods allows the readers to evaluate their suitability, and the

validity and reliability of the results. It also serves as a guide to other researchers, who may

wish to replicate the relevant research. It is remarkable that mistakes involving the data

collection instruments, one of the most important parts of the methods section, are frequently

encountered. Collecting data with an inappropriate data collection instrument, or with one for

which the ratings of validity and reliability have not been carefully assessed, can lead to the

gathering of irrelevant and contradictory information, and possibly to faulty results. Yet,

though these are the most common mistakes, they are not the mistakes with the highest values

in terms of their effects on rejection. Instead, ‘‘unsuitability of the selected method for the goal

of the study’’ ranks in the first place. The problem statement in the study, in other words the

goal of the study, determines what methods should be used (Sonmez 2005). If attention is not

paid to this point, the problem’s solution, the data collection, and the researcher’s interpre-

tations are all negatively affected (Sonmez 2005). This item also ranks among the top reasons

stated in similar studies on the rejection of manuscripts (Pierson 2004; Ajao 2005; McKercher

et al. 2007; Uluoglu 2010). The item ‘‘unsuitability of the statistical methods used for the

research questions or the data collection’’ follows as the second in importance. Since all the

conclusions and generalizations about the research are made according to the analyzed data,

even a simple mistake in the methods or the handling of the statistical data can result in wrong

results and interpretations. Bordage (2001) stated that unsuitable and incomplete statistics,

and inappropriate or inadequate instruments are among the primary reasons for the rejection

of submitted Health Education research study manuscripts.

The rates of frequency of the mistakes discussed in the findings section are quite similar.

However, when the effects of these mistakes on rejection are analyzed, the most prominent

items are ‘‘unsuitability of the findings for the research questions’’ and ‘‘inadequacy of the

findings for answering the research questions.’’ These are directly related to the data

collection instruments, data analysis techniques, selection of the sample population, and

the particular sample selection in the methods part. Mistakes made in these sections will

directly affect the findings that are obtained.

The discussion, conclusion, and suggestions part is the section in which the mistake rates

are at the highest level. Accordingly, the effects of these mistakes on rejection are high, and

show similarly high rates of frequency. Therefore, the most common mistakes made in this

part are the ones with the greatest effect on rejection. The most remarkable item is ‘‘not

discussing the topic with reference to the relevant literature (parallel and opposing views),

and/or the discussion is not based on the research questions and findings.’’ Explaining the

meanings of the results and supporting the results with other findings (Alexandrov et al. 2009)

constitute the most important function of this part of a manuscript. In fact, the discussion

section directly involves supporting and validating the value of one’s own results by com-

parison with other studies, for the sake of the larger scientific community. This activity

enhance reliability of the research in term of the results attained (Pierson 2004).

When the editors’ and referees’ views on the questionnaire items are compared, only the

item ‘‘the content is outside the scope of the journal’’ is significantly different. This

1850 Scientometrics (2014) 98:1837–1853

123

difference is due to the journal editors’ being the first people who examine the submitted

manuscripts. After reviewing the submitted studies for form and content, the journal

editors decide whether or not to send it to the referees for evaluation (Klingner et al. 2005).

Even in this initial evaluation process, a lot of studies are rejected. One of the most

important reasons for rejection is the research being outside the scope of the journal

(Fischer 2004; Pierson 2004).

When all the parts are taken into account, the views of the journal editors and referees

indicate that mistakes are most frequently made in the discussion, conclusion, and sug-

gestions part and in the introduction part of the manuscripts submitted to Turkish journals

of Education indexed in SSCI and ULAKBIM. Secondly, mistakes made in the methods

and introduction parts have the greatest impact on rejection of the manuscripts.

This study provides guidance for both novice scientific writers, and for the editors and

referees of journals regarding standard criteria for the acceptance of manuscripts on educa-

tion. But the study does have some limitations. In this design, we did not adequately compare

the gathered data to investigate the roles and experiences of the participants. We also were not

able to identify potential mistakes beyond those listed in the data collection instrument (the

questionnaire), because face-to-face meetings could not be arranged with the participants. We

did not collect opinions from the referees and editors regarding submitted review articles. We

provided qualitative data from only 23 of the participants. We did not collect the opinions of

the research study writers, who also are a part of the process of manuscript evaluation. And we

were not able to generalize our results, since the research has not been conducted at inter-

national. In future studies on this topic, a larger sample selection, data analysis that includes

comparing the participants according to their experiences and roles, and data collection from

writers whose manuscripts have been rejected may all provide more extensive results.

In spite of all these limitations, the following suggestions have been made to help

researchers prepare their education research for journal publication.

• The research topic should be original.

• The theoretical foundations of the research should be strong.

• The cohesion of the study should be taken into consideration.

• The research methods should be explained clearly and in detail.

• The features of the data collection instruments (structural, content-related, etc.) should

be specified in detail.

• Conclusions and inferences should be drawn in accordance with the research findings.

• Comments should not be exaggerated and/or independent of the findings.

• Attention should be paid to not merely repeating the findings in the discussion.

• The research topic should be discussed with reference to the relevant literature (parallel

and opposing views), and the discussion should be based on the research questions and

findings.

• Suggestions about applications of the research results should be provided.

Acknowledgments We thank Res. Asst. Omer ARPACIK for his assistance with the transference of thequestionnaire to the web environment.

References

Ajao, O. G. (2005). Some reasons for manuscript rejection by peer-reviewed journals. Annals of IbadanPostgraduate Medicine, 3(2), 9–12.

Scientometrics (2014) 98:1837–1853 1851

123

Alexandrov, A. V., Hennerici, M. G., & Norrving, B. (2009). Suggestions for reviewing manuscripts.Cerebrovascular Diseases, 28(3), 243–246. doi:10.1159/000228588.

Arikan, R. (2009). Tez hazirlama teknikleri: Arastirma tezlerinin reddedilmesi [Thesis preparation tech-niques: Rejection of research thesis (in Turkish)]. Ankara, Detay Yayincilik.

Asan, A. (2006). Bilimsel dergilere yayin sunma ve yayin degerlendirme islemleri [Assessment and pub-lication processes in scientific journals (in Turkish)]. Saglik Bilimlerinde Sureli Yayincilik, 101–106.

Audisio, R. A., Stahel, R. A., Aapro, M. S., Costa, A., Pandey, M., & Pavlidis, N. (2009). Successfulpublishing: how to get your paper accepted. Surgical Oncology, 18(4), 350–356.

Belcher, W. L. (2009). Writing your journal article in 12 weeks: A guide to academic publishing andsuccess. Thousand Oaks, CA: Sage.

Bordage, G. (2001). Reasons reviewers reject and accept manuscripts: the strengths and weaknesses inmedical education reports. Academic Medicine, 76(9), 889–896. doi:10.1097/00001888-200109000-00010.

Bornmann, L., & Daniel, H. D. (2010). The manuscript reviewing process: Empirical research on reviewrequests, review sequences, and decision rules in peer review. Library and Information ScienceResearch, 32(1), 5–12. doi:10.1016/j.lisr.2009.07.010.

Bornmann, L., Nast, I., & Daniel, H. D. (2008). Do editors and referees look for signs of scientificmisconduct when reviewing manuscripts? A quantitative content analysis of studies that examinedreview criteria and reasons for accepting and rejecting manuscripts for publication. Scientometrics,77(3), 415–432. doi:10.1007/s11192-007-1950-2.

Bornmann, L., Weymuth, C., & Daniel, H. D. (2010). A content analysis of referees’ comments: How docomments on manuscripts rejected by a high-impact journal and later published in either a low- orhigh-impact journal differ? Scientometrics, 83(2), 493–506.

Chernick, V. (2008). How to get your paper rejected. Pediatric Pulmonology, 43(3), 220–223.Day, R. A. (1997). Bilimsel bir makale nasil yazilir ve yayimlanir? [How to write and publish a scientific

paper (translated to Turkish by Gulay Askar Altay)]. 2. Baskı. Ankara: TUBITAK.Ehara, S., & Takahashi, K. (2007). Reasons for rejection of manuscripts submitted to AJR by international

authors. American Journal of Roentgenology, 188(2), W113–W116.Fischer, C. C. (2004). Managing your research writing for success: Passing the ‘‘Gate Keepers.’’ http://www.

westga.edu/*bquest/2004/gatekeepers.htm. Accessed 2 Feb 2007.Gupta, P., Kaur, G., Sharma, B., Shah, D., & Choudhury, P. (2006). What is submitted and what gets

accepted in Indian Pediatrics: Analysis of submissions, review process, decision making and criteriafor rejection. Indian Pediatrics, 43(6), 479.

Hess, D. R. (2004). How to write an effective discussion. Respiratory Care, 49(10), 1238–1241.Karasar, N. (2005). Bilimsel Arastirma Yontemi [Research Methods]. Ankara: Nobel Yayınevi.Klingner, J. K., Scanlon, D., & Pressley, M. (2005). How to publish in scholarly journals. Educational

Researcher, 34(8), 14–20.McKercher, B., Law, R., Weber, K., Song, H., & Hsu, C. (2007). Why referees reject manuscripts. Journal

of Hospitality and Tourism Research, 31(4), 455–470.McMillan, J. H., & Schumacher, S. (2010). Research in education: Evidence-based inquiry (7th ed.).

Boston: Pearson.Pierson, D. J. (2004). The top 10 reasons why manuscripts are not accepted for publication. Respiratory

care, 49(10), 1246–1252.Price, R. V., & Maushak, N. J. (2000). Publishing in the field of educational technology: Getting started.

Educational Technology, 40(4), 47–52.Sayin, S. (2008). Bilimsel arastırmalarda bazi istatistiksel ve yontembilimsel hatalar-III: Guvenirlik kesti-

rimlerine yonelik hatalar [Some of the statistical and methodological mistakes made in scientificresearches-III: Mistakes made in reliability estimations]. Mehmet Akif Ersoy Universitesi EgitimFakultesi Dergisi, 17(1), 53–69.

Seckin, D. (2003). Bilimsel bir makale nasil yazilir? [How to write a scientific paper?]. Dermatose, 2,90–94.

Shakiba, B., Salmasian, H., Yousefi-Nooraie, R., & Rohanizadegan, M. (2008). Factors influencing editors’decision on acceptance or rejection of manuscripts: the authors’ perspective. Archives of IranianMedicine, 11(3), 257–262.

Sonmez, V. (2005). Bilimsel arastırmalarda yapilan yanlisliklar [Methodological errors in scientificresearch]. Eurasian Journal of Educational Research, 5(18), 236–252.

Summers, J. (2001). Guidelines for conducting research and publishing in marketing: From conceptuali-zation through the review process. Journal of the Academy of Marketing Science, 29(4), 405–415.

1852 Scientometrics (2014) 98:1837–1853

123

Tseng, Y., Chang, C., Tutwiler, M. S., Lin, M., & Barufaldi, J. P. (2013). A scientometric analysis of theeffectiveness of Taiwan’s educational research projects. Scientometrics, 95(3), 1141–1166. doi:10.1007/s11192-013-0966-z.

Turcotte, C., Drolet, P., & Girard, M. (2004). Study design, originality and overall consistency influenceacceptance or rejection of manuscripts submitted to the Journal. Canadian Journal of Anaesthesia-Journal Canadien D Anesthesie, 51(6), 549–556.

Ucak, O. N., & Birinci, H. G. (2008). Bilimsel etik ve intihal. [Scientific ethics and plagiarism]. TurkKutuphaneciligi, 22(2), 187–204.

Uluoglu, C. (2010). Makalelerin reddedilme nedenleri ve reddedilen makalelerin gelecegi. [Reasons for therejection of manuscripts and future of rejected manuscripts]. Saglik Bilimlerinde Sureli Yayincilik,19–27.

Vinluan, L. R. (2012). Research productivity in education and psychology in the Philippines and comparisonwith ASEAN countries. Scientometrics, 91(1), 277–294.

WeiWei, G., Qi, H., XiaoYu, W., XiaPing, X., & JinYu, D. (2009). Analysis on reasons of manuscriptrejection in Journal of Environmental and Occupational Medicine from 2006 to 2007. Journal ofEnvironmental and Occupational Medicine, 26(6), 620–622.

Wijnhoven, B. P. L., & Dejong, C. H. C. (2010). Fate of manuscripts declined by the British Journal ofSurgery. British Journal of Surgery, 97(3), 450–454. doi:10.1002/bjs.6880.

Scientometrics (2014) 98:1837–1853 1853

123