Mistakes encountered in manuscripts on educationand their effects on journal rejections
Embiya Celik • Nuray Gedik • Guler Karaman • Turgay Demirel •
Yuksel Goktas
Received: 18 June 2013 / Published online: 2 October 2013� Akademiai Kiado, Budapest, Hungary 2013
Abstract The goal of this study was to identify common mistakes made in research study
manuscripts submitted to journals of Education and the effects of these mistakes on rejection
by the journal editors and referees. An online questionnaire was developed for this purpose
with 43 items and five open-ended questions. Common mistakes were identified by admin-
istering the 43 questions, which were to be answered in two stages: first by using 5-point
Likert scale responses, and then by responses arranged according to semantic differential
scale (for the effects of the mistakes on rejections). The online questionnaire was sent to the
editors and referees of Turkish journals of Education indexed in SSCI and ULAKBIM. Data
were then collected from 232 participants and examined. The quantitative data obtained from
the questionnaire items were analyzed, and the mean and standard deviation scores were
presented in tables. The qualitative data gathered from the open-ended questions were ana-
lyzed descriptively. The results show that researchers mostly make mistakes in the discus-
sion, conclusion, and suggestions part of the manuscripts. However, mistakes made in the
methods part are the most significant causes of manuscript rejection.
Keywords Reasons for manuscript rejection � Manuscript review � Mistakes
in education manuscripts � Manuscript review process
Introduction
A scientific paper is a report that defines the results of original research (Day 1997). To be
published, a scientific paper should be the first report of these results and should derive
E. Celik � G. Karaman � T. Demirel � Y. Goktas (&)Department of Computer Education and Instructional Technology, Ataturk University, 25240 Erzurum,Turkeye-mail: [email protected]
N. GedikDepartment of Computer Education and Instructional Technology, Akdeniz University, 07058 Antalya,Turkeye-mail: [email protected]
123
Scientometrics (2014) 98:1837–1853DOI 10.1007/s11192-013-1137-y
from an original study; also, the same results should be able to be replicated if the study is
repeated (Seckin 2003). No matter how original and striking the results of research studies
may be, their general acceptance will be controversial unless they are shared and the
process used to reach them is adequately described, so the results can be scrutinized by
others (Asan 2006). For many years, research studies have generally been presented to the
scientific community via science journals (Day 1997). It is a challenging process for
researchers to prepare such publications. Once a formal report has been written, the
researchers send it to an academic journal that is specific to their field. Following peer-
reviews, studies that are found to be acceptable in this first stage are then sent to the journal
referees, who are asked to review and evaluate these studies in more detail. The reasons for
this process are to guarantee the acceptance of only well and responsibly conducted, and
clearly presented studies; to help shape the further development of weaker studies, so that
they may be revised and resubmitted; and to reject studies that are seriously flawed
(Bornmann and Daniel 2010; Wijnhoven and Dejong 2010).
Although the evaluation of scientific research reports is an important process, the
evaluation criteria have not yet been standardized in all research areas. Many journals have
their own evaluation criteria. Very few of the scientific research studies that are conducted
pass the evaluation criteria of the journals successfully and get published after the first
submission (Summers 2001). Studies that fail the evaluation criteria are either rejected or
reevaluated after the necessary corrections are made.
A review of the literature revealed that there are many mistakes which can cause scientific
research studies to fail typical journal evaluation criteria. Examining such mistakes in two
categories (formal mistakes and content-related mistakes), Arikan (2009) defined formal
mistakes as elements of the presentation that are out of compliance with a journal’s for-
matting standards and norms, and content-related mistakes as those made in such sections of
the study as the introduction, the methods description, and the discussion of the findings.
Formal mistakes can be resolved by small-scale adjustments, like the correction of linguistic
errors, the re-arrangement of tables and graphics, and the re-formatting of the manuscript to
comply with the publication guidelines of the journal. But the content-related mistakes can
require large-scale adjustments, such as improving the introduction, adding further orga-
nization or discussion of the methods, or using new or additional statistical analyses and
providing reinterpretations of the findings (Belcher 2009; Arikan 2009). If these adjustments
are not done properly, the study will be rejected. Sonmez (2005) also asserted that any one
mistake made in scientific research can result in other mistakes. For example, a mistake in
the introduction, specifically in the section on the goals of the study or regarding the
statement of the problem to be investigated, could also lead to mistakes and inconsistencies
in other parts, like the methods section, the sample selection, the discussion of the data
collection instruments, or the conduct of the data analysis. The structure of the study should
be built around the research problem and goal. Price and Maushak (2000) stated that the most
frequent mistakes encountered by journal editors are the manuscript’s lack of suitability for
the target audience of the journal, a weak topic choice, an inappropriate choice of methods,
poor writing, weakness in the quality of the research, and/or ethical violations such as
plagiarism. Sample mistakes that were considered to be the most frequent ones in scientific
research of various fields are summarized in Table 1.
The mistakes that stand out the most in Table 1 are ‘‘difficulties regarding the cohesion
and writing quality of the manuscript’’, ‘‘linguistic, presentation, and abbreviation mistakes
in the writing’’, ‘‘improper/inadequate method of analysis and/or use of statistics’’, and
‘‘inappropriate and outdated review of literature.’’ Due to similar mistakes, many scientific
research studies cannot pass the referees’ review and thus get rejected. Studies on these
1838 Scientometrics (2014) 98:1837–1853
123
Ta
ble
1C
om
mo
nM
ista
kes
inS
cien
tifi
cR
esea
rch
and
Rea
son
sfo
rR
ejec
tio
n
Aja
o(2
00
5)
Au
dis
ioet
al.
(20
09)
Bord
age
(20
01)
Cher
nic
k(2
00
8)
Fis
cher
(20
04)
Gu
pta
etal
.(2
00
6)
Hes
s(2
00
4)
McK
erch
eret
al.
(20
07)
Pie
rso
n(2
00
4)
Ulu
og
lu(2
01
0)
Th
em
anu
scri
pt
con
ten
tis
ou
tsid
eth
esc
op
eo
fth
ejo
urn
al4
44
44
Th
ere
sear
chto
pic
isn
ot
up
-to-d
ate,
inte
rest
ing,
or
ori
gin
alen
ough
44
44
Dif
ficu
ltie
sre
gar
din
gth
eco
hes
ion
and
wri
tin
gq
ual
ity
of
the
man
usc
rip
t4
44
44
4
No
tco
nfo
rmin
gto
the
auth
or
gu
idel
ines
of
the
journ
al4
44
Lin
gu
isti
c,p
rese
nta
tio
n,
and
abb
rev
iati
on
mis
tak
esin
the
wri
tin
g4
44
44
Eth
ical
vio
lati
ons,
such
aspla
gia
rism
,fa
lsifi
cati
on,
and/o
rfa
bri
cati
on
44
Th
eto
pic
and
the
sig
nifi
can
ceo
fth
ere
sear
char
en
ot
exp
lain
edcl
earl
y4
4
Th
ere
sear
chg
oal
,ra
tion
ale,
and
pro
ble
mar
en
ot
exp
lain
edcl
earl
y4
44
4
Inap
pro
pri
ate
and
ou
tdat
edre
vie
wo
fli
tera
ture
44
44
4
Imp
rop
er/f
ault
y/p
oor
rese
arch
des
ign
44
44
4
Inef
fici
ent
pre
cauti
ons
toen
sure
the
val
idit
yan
dre
liab
ilit
yo
fth
ere
sear
ch4
44
Inad
equ
acy
of
the
sam
ple
inn
um
ber
44
Dat
aco
llec
tio
nin
stru
men
ts/d
ata
no
tsu
itab
lefo
rth
ere
sear
chg
oal
44
4
Imp
rop
er/i
nad
equ
ate
met
ho
do
fan
alysi
san
d/
or
use
of
stat
isti
cs4
44
44
4
Un
suit
abil
ity
of
the
stru
ctu
res/
des
crip
tio
ns
of
the
tab
les
and
fig
ure
s4
44
Scientometrics (2014) 98:1837–1853 1839
123
Ta
ble
1co
nti
nued
Aja
o(2
00
5)
Au
dis
ioet
al.
(20
09)
Bord
age
(20
01)
Cher
nic
k(2
00
8)
Fis
cher
(20
04)
Gu
pta
etal
.(2
00
6)
Hes
s(2
00
4)
McK
erch
eret
al.
(20
07)
Pie
rso
n(2
00
4)
Ulu
og
lu(2
01
0)
Rep
etit
ion
of
the
findin
gs/
pre
senta
tion
of
just
the
dat
a4
44
Inte
rpre
tati
ons
indep
enden
to
fth
efi
ndin
gs
44
Ov
er-i
nte
rpre
tati
on,
no
tb
ased
on
rese
arch
fin
din
gs
44
4
1840 Scientometrics (2014) 98:1837–1853
123
problems also indicate that most published articles had been rejected by one or more
journals previously (Bornmann & Daniel 2010; Turcotte et al. 2004). In their research,
Ehara and Takahashi (2007) stated that the rejection rates for manuscripts range from 22.6
to 73.4 %.
There are a limited number of research studies that discuss mistakes in scientific studies
or the reasons for journal rejections. The majority of such studies have been conducted in
the field of Healthcare (e.g. Ann Ibadan Postgrad Med, Cerebrovascular Diseases, Surgical
Oncology, Academic Medicine, Pediatric Pulmonology, American Journal of Roentgen-
ology, Indian Pediatrics, Respiratory Care, Respir Care, Archives of Iranian Medicine,
Canadian Journal of Anaesthesia-Journal Canadien D Anesthesie, Journal of Environ-
mental and Occupational Medicine, British Journal of Surgery).But in the field of Edu-
cation, further study of this topic is still needed. To remedy that situation, the goal of this
study was to determine the common mistakes that are made in research studies that are
submitted to journals of Education, and the effects of these mistakes on decisions con-
cerning the acceptance or rejection of the manuscripts by journal editors and referees. The
Turkish journals of Education were taken into consideration in the study in an effort to
present the current status of a country where the productivity of publishing has recently
increased dramatically (Tseng et al. 2013). This study is significant, because it presents
important points to take into consideration when preparing research studies on Education
for publication. The results also may help researchers in the field to conduct higher quality
research studies.
The following research questions guided this study:
1. From the perspectives of the journal editors and referees,
a. what mistakes are commonly seen in scientific research studies submitted to the
Turkish journals of Education, which are indexed in SSCI and Turkish Academic
Network and Information Center (ULAKBIM)?
b. to what extent do these mistakes lead to the rejection of manuscripts?
2. Is there a significant difference between the viewpoints of journal editors and referees
regarding what constitutes mistakes in scientific research studies that are submitted to
Turkish journals of Education indexed in SSCI and ULAKBIM, or regarding the
effects of these mistakes on rejection?
Methods
A survey was used for this study. The data were collected by means of questionnaires
(Karasar 2005; McMillan & Schumacher 2010). During the analysis of the gathered data,
descriptive statistical methods and an independent t test, which is one of the inferential
statistical methods, were used.
The data collection instrument
A questionnaire was used in the study, because the sample group contained a high number
of participants, who were widely dispersed geographically (McMillan & Schumacher
2010). The questionnaire was developed by the researchers. Firstly, the item pool was
constructed. This was based upon scientific articles in the literature, books on research
methods, and the referee/editor evaluation forms of selected journals. The questionnaire
Scientometrics (2014) 98:1837–1853 1841
123
items were divided into five parts, corresponding to the following typical paper sections:
introduction, methods, findings, and discussion-conclusion-suggestions, with the fifth part
devoted to general reasons for manuscript rejections. Then, a peer-reviewed pilot was
conducted with eight doctoral students. The necessary corrections were made, based on the
feedback gathered, and expert opinions were elicited. The questionnaire was reviewed by
five experts (two of them are measurement and evaluation experts, three of them are
educational technology experts), all of whom have a doctoral degree in Education, and
work as journal referees and editors. The questionnaire was submitted to the experts, one at
a time, and then adjusted, based on their overall feedback. After the necessary corrections
were made, the questionnaire was checked by a Turkish language expert, and was edited to
its final form. In the questionnaire, there are 43 scale response items and 5 open-ended
questions. Two different types of data were collected from each questionnaire item, except
for the five open-ended questions. Firstly, a 5-point Likert scale (1- never, 2- rarely, 3-
sometimes, 4- often, 5- always) was used to collect data regarding how frequently each
type of specified mistake was made. Secondly, a 5-point semantic differential scale
(1 = Not at all, 5 = A great deal) was used with the same set of questions to gather data
about the effects of these specified mistakes on journal rejections.
The participants
The participants included the editors and referees of 32 Turkish Journals of Education
which were indexed in SSCI and ULAKBIM as of February, 2012. ULAKBIM is a
research and development center within The Scientific and Technological Research
Council of Turkey. Since the researchers’ purpose was to reach the whole population, a
sample selection was not used. The included journals were each reviewed by the
researchers, and their editors and referees were listed. A total of 993 editors’ and referees’
e-mail addresses were found using search engines. The questionnaire was transferred to the
web environment, and was sent to these email addresses during a four-week period. The
participants were asked in the email message for their cooperation. The questionnaire then
remained online for three weeks. Data were provided for the questionnaire by 232 of the
addressees, who became the participants in the study. 188 of the participants (81.03 %)
were referee, and 44 of the participants (18.97 %) were editor.
The data analysis
The quantitative data were analyzed using SPSS 20 statistical software, descriptive sta-
tistical methods (percentages, means, and standard deviations), and an independent t-test
based in inferential statistical methods. The data belonging to each of the five parts of the
questionnaire were then arranged for presentation in tables.
Descriptive analysis was used to analyze the data collected from the open-ended
questions. In this process, the researchers made a sincere attempt to be faithful to the
original form of the data (as much as was possible). This data is presented below in the
form of direct quotations taken from the statements made by the participants.
Validity and reliability
Various precautions were taken to ensure the validity and reliability of the study. These
precautions include the peer-review, the expert opinions, and professional inspection of the
1842 Scientometrics (2014) 98:1837–1853
123
language used for the questionnaire. Also, detailed descriptions of the methods and the
roles of the researchers have been provided, to further ensure the validity of the study. A
reliability analysis for the questionnaire was conducted for each part separately. Cron-
bach’s Alpha (a) coefficients for both the mistakes in manuscripts and their effects on
rejection are shown in Table 2.
As seen in Table 2, the reliability coefficients of the questionnaire parts are quite high.
The lowest is 0.77, and the highest is 0.93. These values mean that the developed ques-
tionnaire has a high level of reliability.
The roles of the researchers
The study was conducted by two academics in the field of Instructional Technology, who
are also the referees of many national and international journals, and three doctoral stu-
dents. The researchers all exchanged ideas during the process. The academics asked for
expert opinions, transmitted the questionnaire to the participants, and evaluated the ana-
lysis results. The doctoral students were in charge of constructing the item pool of the
questionnaire, reaching the editors and referees of the journals by email, transferring the
questionnaire to the web environment, administering the questionnaire, and analyzing the
data.
The findings
The common mistakes in scientific research and the effects of these mistakes on the
rejection of manuscripts were investigated under seven headings. The subheadings are: the
whole manuscript; introduction; methods; findings; discussion, conclusion, and sugges-
tions; mean values of all the parts; and comparison of the referees’ and editors’ views. In
the tables created for the first six subheadings, the items were sequenced according to the
mean values of the mistakes, and the mean values ( �X) and standard deviations (SD)
belonging to these items were provided. The results of the independent t-test were pre-
sented in a table under the last subheading. Then, the answers to the open-ended questions
were supplied under each relevant part to support the quantitative data.
General mistakes made in the whole manuscript and their effects on rejection
The analysis results are displayed in Table 3. The most common mistake encountered in
the whole manuscript was ‘‘unoriginality of the research topic’’ ( �X = 3.44), and the least
common mistake was ‘‘existence of ethical violations, such as plagiarism, falsification, and
fabrication or suspicion of these ethical violations’’ ( �X = 2.28).
However, when the data on the effects of the mistakes on rejection were examined, the
sequence changed. The item ‘‘existence of ethical violations, such as plagiarism, falsifi-
cation, and fabrication or suspicion of these ethical violations’’ ( �X = 4.44) turned out to be
the mistake that had the most effect on rejection. The item ‘‘lack of an impressive title
appropriate for the research topic’’ ( �X = 2.18) had the least impact on the rejection of a
manuscript.
In the answers to the open-ended questions, the participants generally preferred to
request corrections rather than definitely rejecting the manuscripts due to the mistakes in
this part. One participant said:
Scientometrics (2014) 98:1837–1853 1843
123
..although some mistakes are important, correction can be [requested to resolve] this
issue; therefore, their effects on rejection have been indicated to be low (e.g., 6, 7,
10, etc.)…
Mistakes made in the introduction part of the manuscript and their effects on rejection
The common mistakes made in the introduction part of the manuscripts are illustrated in
Table 4. The most common mistake was ‘‘weakness of theoretical foundations of the
research’’ ( �X = 3.45), while the least common mistake was ‘‘unsuitability of the research
hypotheses and/or questions for the goal of the study’’ ( �X = 2.95). Regarding the effects of
the mistakes on rejection, the mistake with the greatest effect was the item ‘‘unsuitability of
the research hypotheses and/or questions for the goal of the study’’ ( �X = 3.84), and the
mistake with the least impact on the rejection of manuscripts was ‘‘not indicating the
significance and rationale of the research clearly’’ ( �X = 3.38).
Table 2 Reliability results for the parts of the questionnaire
Parts Number of items Cronbach’s alpha coefficient forinternal consistency (a)
Mistakes Effects on rejection
General reasons part 12 0.77 0.82
Introduction part 5 0.85 0.89
Methods part 15 0.92 0.93
Findings part 5 0.83 0.84
Discussion, conclusion, and suggestions part 7 0.85 0.91
Total 44 0.84 0.88
Table 3 General mistakes made in the whole manuscript and their effects on rejection
Items belonging to general reasons Perceivedmistakefrequency
Effects onrejection
�X SD �X SD
3 Unoriginality of the research topic 3.44 0.83 3.65 1.10
4 Lack of cohesion in the study 3.23 0.82 3.84 1.03
2 Outdated and uninteresting research topic 3.16 0.73 3.30 1.10
7 Existence of many spelling and punctuation errors in the manuscript 3.00 0.91 2.28 1.09
6 Incompatibility of the formal features of the study with the authorguidelines of the journal (in-text citations, reference list, font style, etc.)
2.96 0.96 2.61 1.32
5 Difficulties in reading the manuscript 2.91 0.82 3.31 1.19
10 Lack of an impressive title appropriate for the research topic 2.72 0.83 2.18 1.07
11 Abstract does not reflect the research 2.59 0.94 2.31 1.18
1 The manuscript content is outside the scope of the journal 2.33 0.67 3.66 1.38
9 Not conforming to ethical guidelines (receiving approval from ethicscommittees, getting permission for the voluntary participation of theresearch participants)
2.32 0.98 3.67 1.44
8 Existence of ethical violations, such as plagiarism, falsification, andfabrication or suspicion of these ethical violations
2.28 0.97 4.44 1.27
1844 Scientometrics (2014) 98:1837–1853
123
Regarding the open-ended questions in this part, the participants agreed that the mistakes
mentioned in these items can usually be corrected. However, the participants emphasized that
deficiencies in the literature review are of critical importance. One participant stated:
If the literature review part of the study is inefficient or outdated, I ask for correction/
rewriting, but I do not reject it.
Mistakes made in the methods part of the manuscript and their effects on rejection
The mistakes made in the methods part are shown in Table 5. The mistake that was
encountered most frequently was ‘‘unclear and undetailed research method’’ ( �X = 3.24).
The least common mistake was ‘‘not specifying the population and the sample’’
( �X = 2.65).
Regarding the impacts of these mistakes on rejection, the item ‘‘unsuitability of the
selected method for the goal of the study’’ ( �X = 4.27) had the greatest impact. The mistake
with the least effect on rejection was ‘‘not mentioning the role of the researchers in the
research process’’ ( �X = 2.74).
Regarding the answers for the open-ended questions, ‘‘inadequacy of the sample to
represent the population’’ was emphasized. The participants suggested that the word
‘‘sample’’ be changed to ‘‘study group’’ if the sample did not represent the population.
Another comment focused on mistakes made in the data analysis and discussion parts of
qualitative research studies. The following quotes illustrate these points:
If the sample does not represent the population, I suggest changing [the term to]
study group.
There are a lot of mistakes especially in the data analysis and discussion parts of
qualitative research studies. I have found no qualitative data analysis, presentation, or
discussion section in any manuscript received to date to be adequate.
Mistakes made in the findings part of the manuscript and their effects on rejection
The mistakes made in the findings part of the manuscripts are displayed in Table 6. The
most common mistake was ‘‘inadequacy of findings to answer the research questions’’
( �X = 2.98), and the least common mistake was ‘‘unsuitability of the findings for the
research questions’’ ( �X = 2.82).
Concerning the effects of the mistakes on rejection, the item ‘‘unsuitability of the findings for
the research questions’’ ( �X = 4.01) had the greatest effect on rejection, and the item ‘‘exis-
tence of redundant details, tables, and figures’’ ( �X = 2.67) had the least effect on rejection.
Regarding the open-ended questions, the participants replied that they usually request
corrections for the mistakes made in this part. The most important criticism of this section
was that our questionnaire items focused only on the aspects of quantitative and qualitative
data presented in research studies evaluated by journals; review articles also should have
been included in the questionnaire items for this part. This is a limitation in the present
study. To illustrate, one participant commented:
The questions in this part cover only practices concerning qualitative and quantita-
tive research studies. However, some articles have the characteristics of review
articles. What about our evaluations regarding these articles?
Scientometrics (2014) 98:1837–1853 1845
123
Mistakes made in the discussion, conclusion, and suggestions part of the manuscripts
and their effects on rejection
The results for this part are displayed in Table 7. The most common mistake encountered
in this part of the manuscripts was ‘‘not discussing the topic with reference to the relevant
literature (parallel and opposing views), and/or the discussion is not based on the research
Table 4 Mistakes made in the introduction part of the manuscript and their effects on rejection
Items belonging to the introduction part Perceivedmistakefrequency
Effects onrejection
�X SD �X SD
12 Weakness of the theoretical foundations of the research 3.45 0.74 3.81 1.08
14 Not indicating the significance and rationale of the research clearly 3.19 0.74 3.38 1.12
13 Not indicating the goal of the study clearly 3.11 0.75 3.46 1.11
16 Unsuitability of the literature review for the goal of the study/researchquestions
2.98 0.84 3.63 1.14
15 Unsuitability of the research hypotheses and/or questions for the goal ofthe study
2.95 0.74 3.84 1.16
Table 5 Mistakes made in the methods part of the manuscript and their effects on rejection
Items belonging to the methods part Perceivedmistakefrequency
Effects onrejection
�X SD �X SD
18 Unclear and undetailed research method 3.24 0.76 3.78 1.02
25 Not specifying the features of the data collection instruments (structural,content-related, etc.) in detail
3.22 0.80 3.54 1.07
26 Not taking the measures of validity and reliability for the data collectioninstruments
3.19 0.85 4.06 1.01
20 Unclear and incomprehensible sample selection method 3.13 0.83 3.54 1.03
27 Not explaining the data collection process in detail 3.13 0.81 3.34 1.05
30 Not mentioning the role of the researchers in the research process 3.03 1.06 2.74 1.14
29 Unsuitability of the statistical methods used for the research questions orthe data collected
3.01 0.87 4.21 0.94
22 Inadequacy of the sample to represent the population 2.97 0.86 3.73 1.17
31 Not taking the measures of validity and reliability in the data collectionprocess/for the conditions of the research practices
2.96 0.86 3.54 1.16
17 Unsuitability of the selected method for the goal of the study 2.91 0.8 4.27 0.94
28 Not explaining the method of data analysis 2.91 0.87 3.56 1.14
23 Unclear demographic information about the sample 2.88 0.83 3.05 1.11
24 Unsuitability of the data collection instruments for the goal of the study 2.83 0.95 4.10 1.10
21 Unsuitability of the sample for the goal of the study 2.80 0.9 3.92 1.11
19 Not specifying the sample and the population 2.65 0.93 3.56 1.17
1846 Scientometrics (2014) 98:1837–1853
123
questions and findings’’ ( �X = 3.60), while the least common mistake was ‘‘not specifying
the limitations of the research’’ ( �X = 3.21).
Regarding the effects of the mistakes on rejection, the item ‘‘lack of results and
inferences appropriate for the research findings’’ ( �X = 3.81) was regarded as the mistake
that had the greatest impact on rejection. The item ‘‘lack of suggestions for future
research’’ ( �X = 2.69) was the mistake that had the least effect on rejection.
Regarding the open-ended questions, the participants most frequently commented that a
big problem they had observed was that researchers, in writing the discussion and con-
clusion sections, did not fully grasp the meaning or implications of their own findings. The
answers provided here also indicated that research findings were too often simply repeated
in this part. Concerning this point, one participant said:
The discussion and conclusion often indicate that the findings were not understood.
[Instead,] the findings are [merely] repeated in the discussion and conclusion.
Unfortunately, comparisons, analysis, and interpretations of the findings based on the
literature [often] are not provided.
The mean values for the all of the parts on mistakes made in the manuscripts and their
effects on rejection
According to the findings, the parts in which mistakes are made the most are ‘‘dis-
cussion, conclusion, and suggestions’’ ( �X = 3.38), while the part in which mistakes are
encountered least frequently is ‘‘general reasons’’ ( �X = 2.81). Concerning the effects of
the mistakes on rejection, the mistakes made in the ‘‘methods part’’ ( �X = 3.66) had the
greatest effect on the rejection of the manuscripts, while those made in the ‘‘general
reasons’’ part ( �X = 3.04) had the least impact on rejection. The mean values of the
questionnaire sections are illustrated in Table 8.
Differing editor and referee views regarding the questionnaire items on the mistakes
made in the manuscripts and their effects on rejection
In Table 9, the results of the comparison of the editors’ and the referees’ opinions con-
cerning the mistakes in manuscripts and their impacts on rejection are provided for the
second research question.
Table 6 Mistakes made in the findings part of the manuscript and their effects on rejection
Items belonging to the findings part Perceived mistakefrequency
Effects onrejection
�X SD �X SD
33 Inadequacy of the findings to answer the research questions 2.98 0.84 3.97 0.98
36 Existence of subjective expressions/judgments in the findings 2.89 0.89 2.97 1.19
34 Existence of redundant details, tables, and figures 2.88 0.90 2.67 1.07
35 Unsuitability of the descriptions of the tables and figures 2.88 0.87 2.88 1.13
32 Unsuitability of the findings for the research questions 2.82 0.86 4.01 1.07
Scientometrics (2014) 98:1837–1853 1847
123
The findings indicate that there is a significant difference in only the mean values
belonging to the item ‘‘the manuscript content is outside the scope of the journal,’’ with the
editors rating higher on this point [t(2,21) = -2.199, p \ 0.05]. The views of the editors
and referees have similar mean values in the other items. No significant difference was
found between the questionnaire items in either the mistakes made or the effects of these
mistakes on rejection.
Discussion, conclusion, and suggestions
In light of the gathered data, among the items overall, ‘‘unoriginality of the research topic’’
(Ehara and Takahashi 2007; WeiWei et al. 2009; Bornmann et al. 2010), ‘‘lack of cohesion
in the research’’ (Uluoglu 2010; Chernick 2008; Ajao 2005; Audisio et al. 2009; Fischer
2004), and ‘‘outdated and uninteresting research topic’’ are the most prominent general
mistakes made in education research studies according to Turkish editors and referees.
Table 7 Mistakes made in the discussion, conclusion, and suggestions part of the manuscripts and theireffects on rejection
Items belonging to the discussion, conclusion, and suggestions part Perceivedmistakefrequency
Effects onrejection
�X SD �X SD
39 Not discussing the topic with reference to the relevant literature (paralleland opposing views), and/or the discussion is not based on the researchquestions and findings
3.60 0.84 3.69 1.05
37 Repeating the findings in the discussion 3.50 0.83 3.01 1.19
40 Lack of results and inferences appropriate for the research findings 3.40 0.77 3.81 1.04
38 Comments that are exaggerated and/or independent of the findings 3.36 0.85 3.54 1.16
43 Lack of suggestions about the applicability of the research results 3.35 0.83 2.84 1.09
42 Lack of suggestions for future research 3.22 0.86 2.69 1.15
41 Not specifying the limitations of the research 3.21 0.88 3.06 1.14
Table 8 The mean values for all of the parts on mistakes made in the manuscripts and their effects onrejection
Parts Perceived mistake frequency Effects on rejection
�X SD �X SD
Discussion, conclusion, and suggestions part 3.38 0.14 3.23 0.44
Introduction part 3.14 0.20 3.62 0.21
Methods part 2.99 0.17 3.66 0.42
Findings part 2.89 0.06 3.30 0.64
General reasons 2.81 0.40 3.04 0.91
1848 Scientometrics (2014) 98:1837–1853
123
Although these items are the most common mistakes made in the studies, only ‘‘lack of
cohesion in the research’’ was stated to be affecting the rejection of a study to a great
extent. Notably, Shakiba et al. (2008) asserted that one of the most important criteria for
the publication of manuscripts is good writing. This lack can be aligned to a language
problem since writing the manuscripts in a foreign language (i.e., English is required for
many Turkish journals indexed in SSCI) can be a hindering effect for authors to write
good. However, as Tseng et al. (2013) and Vinluan (2012) pointed out, these journals
accept manuscripts written in mother language (i.e., Turkish) as well.
Correction was found to be generally preferred for the mistakes made due to general
reasons. However, the two items that have the most impact on the rejection of research
studies both involve ethical principles. Today, the most significant problem in the science
world is ethical violations (Bornmann et al. 2008). Such ethical violations as plagiarism,
falsification, and fabrication are mistakes that are not forgiven by journal editors and
referees. Scientific ethics and plagiarism are issues that are increasingly emphasized in the
literature, though they are not emphasized enough in Turkey (Ucak & Birinci 2008). It is
notable that these items are the mistakes made in submitted research studies least fre-
quently. This seems to indicate that researchers are conscious and cautious of the rules of
scientific ethics.
The introduction of an article manuscript is the part that consists of a succinct statement
about the study, the research topic, its significance, the rationale, hypotheses, and ques-
tions. According to the gathered data, the items with the highest value among the common
mistakes made in the introduction part of submitted manuscripts mostly have low impacts
on rejection. Thus, the most common mistakes can usually be corrected without automatic
rejection. But the item ‘‘weakness of the theoretical foundations for the research’’ featured
high values both as one of the most common mistakes made by researchers and for its
effect on rejection. Aside from providing suitable and up-to-date information about the
study and the research problem (which is the point of departure for the study), the intro-
duction should also contain a pertinent and thorough review of relevant literature to pro-
vide the study with a strong theoretical structure and context. The theoretical background
drawn from the literature also supports the argument for the significance of the researcher’s
own study (i.e., the study should fill an unaddressed gap in the literature, or take the next
step by building on previous findings). Conducting the literature review while devising a
study also assists the researcher to map or modify his/her developing study. A poor
literature review has been regarded as a reason for rejection in many studies (Bordage
2001; Fischer 2004; McKercher et al. 2007; Audisio et al. 2009; Uluoglu 2010). Another
item in the introduction part with a high effect on rejection is ‘‘unsuitability of the research
hypotheses and/or questions for the goal of the study.’’ If these points constituting the basis
of the study are not explained clearly and comprehensively, the readers may not be able to
properly understand the study. Each person reading the research questions should infer the
same meaning (Sonmez 2005).
Table 9 Comparison of the editors’ and referees’ views about the mistakes in the manuscripts and theireffects on rejection
‘‘The manuscript content is outside the scope of the journal’’ N �X SD df t p
Referee 188 3.55 1.38 2.21 -2.199 0.029
Editor 44 4.07 1.37
Scientometrics (2014) 98:1837–1853 1849
123
The ratings for the effects of mistakes made in the ‘‘methods part’’ are higher compared to
the other parts. Hence, manuscripts are most frequently rejected due to mistakes in the
methods part. Such mistakes reflect the researchers’ inadequate knowledge of appropriate
research methods (Sayin 2008). McKercher et al. (2007) stated that referees encounter
methods problems in manuscripts most frequently. Turcotte et al. (2004) stated that methods
are the main factor that determines the destiny of research study. The primary mistake among
the common mistakes made in the methods part is ‘‘unclear and undetailed research method.’’
This finding is also supported in the literature (Audisio et al. 2009). A clear and detailed
description of the research methods allows the readers to evaluate their suitability, and the
validity and reliability of the results. It also serves as a guide to other researchers, who may
wish to replicate the relevant research. It is remarkable that mistakes involving the data
collection instruments, one of the most important parts of the methods section, are frequently
encountered. Collecting data with an inappropriate data collection instrument, or with one for
which the ratings of validity and reliability have not been carefully assessed, can lead to the
gathering of irrelevant and contradictory information, and possibly to faulty results. Yet,
though these are the most common mistakes, they are not the mistakes with the highest values
in terms of their effects on rejection. Instead, ‘‘unsuitability of the selected method for the goal
of the study’’ ranks in the first place. The problem statement in the study, in other words the
goal of the study, determines what methods should be used (Sonmez 2005). If attention is not
paid to this point, the problem’s solution, the data collection, and the researcher’s interpre-
tations are all negatively affected (Sonmez 2005). This item also ranks among the top reasons
stated in similar studies on the rejection of manuscripts (Pierson 2004; Ajao 2005; McKercher
et al. 2007; Uluoglu 2010). The item ‘‘unsuitability of the statistical methods used for the
research questions or the data collection’’ follows as the second in importance. Since all the
conclusions and generalizations about the research are made according to the analyzed data,
even a simple mistake in the methods or the handling of the statistical data can result in wrong
results and interpretations. Bordage (2001) stated that unsuitable and incomplete statistics,
and inappropriate or inadequate instruments are among the primary reasons for the rejection
of submitted Health Education research study manuscripts.
The rates of frequency of the mistakes discussed in the findings section are quite similar.
However, when the effects of these mistakes on rejection are analyzed, the most prominent
items are ‘‘unsuitability of the findings for the research questions’’ and ‘‘inadequacy of the
findings for answering the research questions.’’ These are directly related to the data
collection instruments, data analysis techniques, selection of the sample population, and
the particular sample selection in the methods part. Mistakes made in these sections will
directly affect the findings that are obtained.
The discussion, conclusion, and suggestions part is the section in which the mistake rates
are at the highest level. Accordingly, the effects of these mistakes on rejection are high, and
show similarly high rates of frequency. Therefore, the most common mistakes made in this
part are the ones with the greatest effect on rejection. The most remarkable item is ‘‘not
discussing the topic with reference to the relevant literature (parallel and opposing views),
and/or the discussion is not based on the research questions and findings.’’ Explaining the
meanings of the results and supporting the results with other findings (Alexandrov et al. 2009)
constitute the most important function of this part of a manuscript. In fact, the discussion
section directly involves supporting and validating the value of one’s own results by com-
parison with other studies, for the sake of the larger scientific community. This activity
enhance reliability of the research in term of the results attained (Pierson 2004).
When the editors’ and referees’ views on the questionnaire items are compared, only the
item ‘‘the content is outside the scope of the journal’’ is significantly different. This
1850 Scientometrics (2014) 98:1837–1853
123
difference is due to the journal editors’ being the first people who examine the submitted
manuscripts. After reviewing the submitted studies for form and content, the journal
editors decide whether or not to send it to the referees for evaluation (Klingner et al. 2005).
Even in this initial evaluation process, a lot of studies are rejected. One of the most
important reasons for rejection is the research being outside the scope of the journal
(Fischer 2004; Pierson 2004).
When all the parts are taken into account, the views of the journal editors and referees
indicate that mistakes are most frequently made in the discussion, conclusion, and sug-
gestions part and in the introduction part of the manuscripts submitted to Turkish journals
of Education indexed in SSCI and ULAKBIM. Secondly, mistakes made in the methods
and introduction parts have the greatest impact on rejection of the manuscripts.
This study provides guidance for both novice scientific writers, and for the editors and
referees of journals regarding standard criteria for the acceptance of manuscripts on educa-
tion. But the study does have some limitations. In this design, we did not adequately compare
the gathered data to investigate the roles and experiences of the participants. We also were not
able to identify potential mistakes beyond those listed in the data collection instrument (the
questionnaire), because face-to-face meetings could not be arranged with the participants. We
did not collect opinions from the referees and editors regarding submitted review articles. We
provided qualitative data from only 23 of the participants. We did not collect the opinions of
the research study writers, who also are a part of the process of manuscript evaluation. And we
were not able to generalize our results, since the research has not been conducted at inter-
national. In future studies on this topic, a larger sample selection, data analysis that includes
comparing the participants according to their experiences and roles, and data collection from
writers whose manuscripts have been rejected may all provide more extensive results.
In spite of all these limitations, the following suggestions have been made to help
researchers prepare their education research for journal publication.
• The research topic should be original.
• The theoretical foundations of the research should be strong.
• The cohesion of the study should be taken into consideration.
• The research methods should be explained clearly and in detail.
• The features of the data collection instruments (structural, content-related, etc.) should
be specified in detail.
• Conclusions and inferences should be drawn in accordance with the research findings.
• Comments should not be exaggerated and/or independent of the findings.
• Attention should be paid to not merely repeating the findings in the discussion.
• The research topic should be discussed with reference to the relevant literature (parallel
and opposing views), and the discussion should be based on the research questions and
findings.
• Suggestions about applications of the research results should be provided.
Acknowledgments We thank Res. Asst. Omer ARPACIK for his assistance with the transference of thequestionnaire to the web environment.
References
Ajao, O. G. (2005). Some reasons for manuscript rejection by peer-reviewed journals. Annals of IbadanPostgraduate Medicine, 3(2), 9–12.
Scientometrics (2014) 98:1837–1853 1851
123
Alexandrov, A. V., Hennerici, M. G., & Norrving, B. (2009). Suggestions for reviewing manuscripts.Cerebrovascular Diseases, 28(3), 243–246. doi:10.1159/000228588.
Arikan, R. (2009). Tez hazirlama teknikleri: Arastirma tezlerinin reddedilmesi [Thesis preparation tech-niques: Rejection of research thesis (in Turkish)]. Ankara, Detay Yayincilik.
Asan, A. (2006). Bilimsel dergilere yayin sunma ve yayin degerlendirme islemleri [Assessment and pub-lication processes in scientific journals (in Turkish)]. Saglik Bilimlerinde Sureli Yayincilik, 101–106.
Audisio, R. A., Stahel, R. A., Aapro, M. S., Costa, A., Pandey, M., & Pavlidis, N. (2009). Successfulpublishing: how to get your paper accepted. Surgical Oncology, 18(4), 350–356.
Belcher, W. L. (2009). Writing your journal article in 12 weeks: A guide to academic publishing andsuccess. Thousand Oaks, CA: Sage.
Bordage, G. (2001). Reasons reviewers reject and accept manuscripts: the strengths and weaknesses inmedical education reports. Academic Medicine, 76(9), 889–896. doi:10.1097/00001888-200109000-00010.
Bornmann, L., & Daniel, H. D. (2010). The manuscript reviewing process: Empirical research on reviewrequests, review sequences, and decision rules in peer review. Library and Information ScienceResearch, 32(1), 5–12. doi:10.1016/j.lisr.2009.07.010.
Bornmann, L., Nast, I., & Daniel, H. D. (2008). Do editors and referees look for signs of scientificmisconduct when reviewing manuscripts? A quantitative content analysis of studies that examinedreview criteria and reasons for accepting and rejecting manuscripts for publication. Scientometrics,77(3), 415–432. doi:10.1007/s11192-007-1950-2.
Bornmann, L., Weymuth, C., & Daniel, H. D. (2010). A content analysis of referees’ comments: How docomments on manuscripts rejected by a high-impact journal and later published in either a low- orhigh-impact journal differ? Scientometrics, 83(2), 493–506.
Chernick, V. (2008). How to get your paper rejected. Pediatric Pulmonology, 43(3), 220–223.Day, R. A. (1997). Bilimsel bir makale nasil yazilir ve yayimlanir? [How to write and publish a scientific
paper (translated to Turkish by Gulay Askar Altay)]. 2. Baskı. Ankara: TUBITAK.Ehara, S., & Takahashi, K. (2007). Reasons for rejection of manuscripts submitted to AJR by international
authors. American Journal of Roentgenology, 188(2), W113–W116.Fischer, C. C. (2004). Managing your research writing for success: Passing the ‘‘Gate Keepers.’’ http://www.
westga.edu/*bquest/2004/gatekeepers.htm. Accessed 2 Feb 2007.Gupta, P., Kaur, G., Sharma, B., Shah, D., & Choudhury, P. (2006). What is submitted and what gets
accepted in Indian Pediatrics: Analysis of submissions, review process, decision making and criteriafor rejection. Indian Pediatrics, 43(6), 479.
Hess, D. R. (2004). How to write an effective discussion. Respiratory Care, 49(10), 1238–1241.Karasar, N. (2005). Bilimsel Arastirma Yontemi [Research Methods]. Ankara: Nobel Yayınevi.Klingner, J. K., Scanlon, D., & Pressley, M. (2005). How to publish in scholarly journals. Educational
Researcher, 34(8), 14–20.McKercher, B., Law, R., Weber, K., Song, H., & Hsu, C. (2007). Why referees reject manuscripts. Journal
of Hospitality and Tourism Research, 31(4), 455–470.McMillan, J. H., & Schumacher, S. (2010). Research in education: Evidence-based inquiry (7th ed.).
Boston: Pearson.Pierson, D. J. (2004). The top 10 reasons why manuscripts are not accepted for publication. Respiratory
care, 49(10), 1246–1252.Price, R. V., & Maushak, N. J. (2000). Publishing in the field of educational technology: Getting started.
Educational Technology, 40(4), 47–52.Sayin, S. (2008). Bilimsel arastırmalarda bazi istatistiksel ve yontembilimsel hatalar-III: Guvenirlik kesti-
rimlerine yonelik hatalar [Some of the statistical and methodological mistakes made in scientificresearches-III: Mistakes made in reliability estimations]. Mehmet Akif Ersoy Universitesi EgitimFakultesi Dergisi, 17(1), 53–69.
Seckin, D. (2003). Bilimsel bir makale nasil yazilir? [How to write a scientific paper?]. Dermatose, 2,90–94.
Shakiba, B., Salmasian, H., Yousefi-Nooraie, R., & Rohanizadegan, M. (2008). Factors influencing editors’decision on acceptance or rejection of manuscripts: the authors’ perspective. Archives of IranianMedicine, 11(3), 257–262.
Sonmez, V. (2005). Bilimsel arastırmalarda yapilan yanlisliklar [Methodological errors in scientificresearch]. Eurasian Journal of Educational Research, 5(18), 236–252.
Summers, J. (2001). Guidelines for conducting research and publishing in marketing: From conceptuali-zation through the review process. Journal of the Academy of Marketing Science, 29(4), 405–415.
1852 Scientometrics (2014) 98:1837–1853
123
Tseng, Y., Chang, C., Tutwiler, M. S., Lin, M., & Barufaldi, J. P. (2013). A scientometric analysis of theeffectiveness of Taiwan’s educational research projects. Scientometrics, 95(3), 1141–1166. doi:10.1007/s11192-013-0966-z.
Turcotte, C., Drolet, P., & Girard, M. (2004). Study design, originality and overall consistency influenceacceptance or rejection of manuscripts submitted to the Journal. Canadian Journal of Anaesthesia-Journal Canadien D Anesthesie, 51(6), 549–556.
Ucak, O. N., & Birinci, H. G. (2008). Bilimsel etik ve intihal. [Scientific ethics and plagiarism]. TurkKutuphaneciligi, 22(2), 187–204.
Uluoglu, C. (2010). Makalelerin reddedilme nedenleri ve reddedilen makalelerin gelecegi. [Reasons for therejection of manuscripts and future of rejected manuscripts]. Saglik Bilimlerinde Sureli Yayincilik,19–27.
Vinluan, L. R. (2012). Research productivity in education and psychology in the Philippines and comparisonwith ASEAN countries. Scientometrics, 91(1), 277–294.
WeiWei, G., Qi, H., XiaoYu, W., XiaPing, X., & JinYu, D. (2009). Analysis on reasons of manuscriptrejection in Journal of Environmental and Occupational Medicine from 2006 to 2007. Journal ofEnvironmental and Occupational Medicine, 26(6), 620–622.
Wijnhoven, B. P. L., & Dejong, C. H. C. (2010). Fate of manuscripts declined by the British Journal ofSurgery. British Journal of Surgery, 97(3), 450–454. doi:10.1002/bjs.6880.
Scientometrics (2014) 98:1837–1853 1853
123