The Readability of Academic Papers in the Journal of Property Investment & Finance (JPIF)

The Readability of JPIF: ERES Milan 2010

The Readability of Academic Papers in the Journal of Property Investment & Finance (JPIF)

Stephen LeeCass Business School

andNick French

Oxford Brookes University


Introduction:

In the MSc Degree courses I use a number of articles from journals for teaching purposes and refer the students to many others.

In addition, students do a dissertation which usually involves a literature review.

So do the students understand what they are reading?

In other words how readable are Real Estate Journals?


Introduction:

The aim of the JPIF is to keep industry practitioners informed on current thinking and developments in all aspects of real estate research and practice by informing and encourage debate between academics and practising professionals.

To achieve this aim the Journal seeks to:

“publish well-written, readable articles of intellectual rigour with a theoretical and practical relevance to the real estate profession.”

However, nothing is known about the readability of academic papers in real estate journals.


Readability:

In this study readability is defined as the:

“ease of understanding or comprehension based on the style of writing” (Klare, 1963).

It is usually assumed that a particular passage of text is easy to read if it has simple words and short sentences.

Therefore, most readability tests concentrate on these two dimensions of the sampled text to estimate its readability, but differ in how they measure the average simplicity or difficulty of words within a sentence and how they measure the average length of the sentences.

Consequently there are more than 200 methods!


Readability:

We use five educational level readability tests, designed to identify the number of years of education needed to read the text, and average the results across the tests:

1) Flesch-Kincaid Grade Level (FKGL) index,2) Fog index,3) SMOG index,4) Automated Readability (AR) index and5) Coleman-Liau (CL) index.

In addition, we also calculated the Flesch Reading Ease (FRE) index, which varies between 0 and 100, as it is the most widely used, reliable and tested formula and as some readability researchers consider it the most appropriate test for technical documents.


Research Design:

In order to asses the readability of academic papers in the JPIF we followed the approach in previous work and took the text from the introductions of all articles in all the issues of JPIF from Vol. 17 to Vol. 27, which resulted in a total of 297 papers.

We took all samples from the introductory section of each article because sampling from the same section in each article increases the comparability across articles and across time.

In addition, sampling at this point avoids the most technical sections of the articles and so we can assess the authors’ communication skills, without the distraction of the technical aspects of the research methods.


Statistic Mean Q1 Q3 Min Max Number of Words 613.9 355 776 101 2678 Average Grade Level 15.5 13.9 16.6 8.1 25.3 Flesch Reading Ease 30.4 24.1 37.4 -2.0 69.8

Overall Readability Results

The correlation between the Average Grade Level scores and the Flesch Reading Ease scores is -0.91, which indicates the different methods produce similar results and indicate that the introductions to the academic papers in the JPIF are “difficult” and require the reader to have at least a college education, if not a postgraduate degree.

These conclusions can be seen more clearly in the Figures 1 and 2.


Average 15.5 = graduate level

Figure 1: Histogram of Average Grade Scores

0%

5%

10%

15%

20%

25%

30%

35%

40%

10 12 14 16 18 20 22 24 >24


47.4% 50%

Figure 2: Histogram of Flesch Reading Ease Scores

50% 47.4%

99% Difficult and above!

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Very Difficult Difficult Fairly Difficult Standard


Changes over time in the readability of the JPIF?

Grade V17 V18 V19 V20 V21 V22 V23 V24 V25 V26 V27 Average 16.2 14.9 16.0 14.7 15.3 15.9 15.5 15.4 15.4 15.3 15.7 Q1 14.6 13.7 14.5 13.1 14.5 15.0 14.6 14.5 13.5 13.3 13.9 Q2 17.0 15.7 17.4 16.1 16.1 16.4 16.4 16.4 16.9 17.4 16.7 FRE V17 V18 V19 V20 V21 V22 V23 V24 V25 V26 V27 Average 27.1 32.4 27.4 35.2 31.1 30.5 29.7 30.4 30.3 32.6 27.5 Q1 21.5 26.9 22.7 28.9 27.6 25.6 22.4 24.5 21.4 22.1 20.4 Q2 34.2 39.2 33.1 44.0 34.9 35.6 36.3 35.6 39.2 42.9 37.3

There has been very little difference in the readability of the academic papers in the JPIF in either the level of education needed by the reader or its ease of comprehension. These conclusions can be seen more clearly in Figures 3 and 4, which presents Boxplots for Volumes 17 to 27, for the Average Grade Level scores and Flesch Reading Ease scores.


47.4% 50%

Figure 3: Boxplot of Grade Levels: Vol. 17 to Vol. 27

50% 47.4%

7

9

11

13

15

17

19

21

23

25

27

Vol 17 Vol 18 Vol 19 Vol 20 Vol 21 Vol 22 Vol 23 Vol 24 Vol 25 Vol 26 Vol 27

Education Level

Little change over time


47.4% 50%

Figure 4: Boxplot of Flesch Reading Ease Scores: Vol. 17 to Vol. 27

50% 47.4%

-5

5

15

25

35

45

55

65

75

Vol 17 Vol 18 Vol 19 Vol 20 Vol 21 Vol 22 Vol 23 Vol 24 Vol 25 Vol 26 Vol 27

Reading Ease Score

Little change over time


Results: Comparisons with previous work:

Crosier (2004) mean FRE 33.7 with a range of 10.7 to 57.

Loveland et al (1973) average FRE score across the journals was 33.5 with range of 20 to 54.

Bauerly, et al. (2005, 2006) examined the readability of the Journal of Marketing from 1936 to 2001 and found the Journal’s readability level had dropped precipitously in the late 1960s and has remained in the Flesch “very difficult” range ever since.

The mean FRE score of 30.4 for the sample of academic writings in this study places them in a similar range as the papers for other academic journals, i.e. “difficult” to read.


Conclusion:

Unless the sample that generated these findings is unrepresentative in some systematic way, a FRE score of 30.4 and an average reading educational level of 15.5 places the academic papers in the JPIF firmly in the “difficult” range with the reader needing a college educational level in order to understand the text.

Does having a low FRE score, or a high educational reading level, make the JPIF too difficult to read? Not necessarily, because the readability scores used here are but one measure of an article’s readability. Another factor is the education level of the audience.

So should I expect the students to understand the articles?


PS:The FRE score of the introduction to this article is 41.8, which would put it in the lower end of Flesch’s “difficult” range, requiring about a 13th - 14th grade level education, i.e. about first year undergraduate level. A results confirmed by the Average Grade Level score which suggests that the reader only needs at about 12.1 years of schooling (high school senior level) for it to be comprehended. Lastly, the introduction would have been ranked at about the 26th easiest paper to read, of the 297 articles in our analysis.

☺☺☺SEE IT CAN BE DONE


The Readability of Academic Papers in the Journal of Property Investment & Finance

Any Questions?

Documents

The Readability of Academic Papers in the Journal of Property Investment & Finance (JPIF)