1
Exploring Assessment: an Intervention Sonia Shimeld*, Simone Bingham and John Minas
School of Accounting and Corporate Governance, University of Tasmania
*Corresponding author: Sonia Shimeld School of Accounting and Corporate Governance University of Tasmania Private Bag 86 Hobart Tasmania 7001 Australia Email: [email protected]
Abstract
Teaching a complex subject such as taxation law to international students at a professional
university masters level is challenging with high failure rates reflecting the difficulty students
have in learning such a subject. Added to this is the changing face of accounting education in
Australia with the Australian Qualifications Framework enforced by the Tertiary Education
Quality and Standards Agency adding increasing accountability requirements. Using action
research the problem of student learning and poor oral communication skills was addressed
by designing interviews as a form of assessment. It was recognised that assessment could be
a substantial driver for learning. Results of surveys conducted both pre‐ and post‐
intervention with subsequent follow up discussions were very positive with lower failure
rates also indicative of the benefit to students.
Introduction
Taxation law in Australia is constantly changing which makes student learning difficult, as
students cannot rely on past solutions or even past textbooks. With international students
dominating the cohort in the Masters of Professional Accounting (MPA), their challenge is
not only to understand the content of the subject but also to be able to demonstrate their
level of understanding through a high level of communication skills (in English). This is
reflective of the current challenges faced by accounting education in Australia in dealing
with large cohorts of international students (De Lange and Watty, 2010).
2
As with all university subjects, taxation has to meet the needs of all stakeholders,
particularly accounting professional bodies and the local employers, and it must address the
accountability requirements as set by the Australian Qualifications Framework (AQF) (2011).
This is particularly problematic for the MPA, which was considered a conversion course for
postgraduate students requiring accounting expertise, but now must meet minimum
Masters level competencies (including oral communication) as set by the AQF (Evans et al,
2010; Freeman and Hancock, 2011).
Assessment is well recognised as being influential to student learning (Scouller, 1998) so this
avenue was taken to raise the oral communication level and enhance student learning in the
subject. It was hoped that assessment could be used to motivate students to take a more
active role in their learning as by having to explain a case to a professional they would need
to understand the material at a deeper level. An assessment tool was created (a mini‐viva,
or interview) that required as its base the preparation of a “real” tax return so that students
could understand the relevance of the interview.
This paper follows the action research carried out in creating and assessing this tool. After a
brief background about the student cohort, the action research process of planning, taking
action and evaluating this new initiative in assessment is described. That is, identifying the
problem, designing a solution and evaluating the impact of the solution all formed part of
the research procedures. The problem was defined through the experience of the academics
who taught the subject. The solution was identified through discussion with other
academics both from within the discipline and from other disciplines, researching the
literature and developing and conducting a new form of assessment. Evaluation of the
success of the innovation was through student surveys, both pre‐ and post‐intervention,
interviews with students after they had received their final marks, (through a comparison
with results from the previous semester), and by reflections from the academics involved.
3
Background
At our university, taxation consistently has the highest failure rate of all the MPA subjects.
Although a student‐centred approach has always driven the teaching and assessment in the
subject, with criterion‐based assessment through rubrics, peer‐based assessment and forms
of group work; failure rates continued to be high. Many students relied on previous students’
notes, which were not always beneficial with tax law in a constant state of flux. With around
90‐95% of the 80‐100 students who undertake the subject per semester being international
students, it was considered that a better understanding of how the student cohort perceives
assessment and feedback would provide an insight into how assessment could be used to
support their learning.
In taxation, student numbers, timetabling and academic workload dictates that traditional
two‐hour lectures and one‐hour tutorials be the dominant paradigm. Tutorials are limited to
a maximum of 20 students. As there was only one subject of taxation, all the basic
information such as defining assessable income and allowable deductions and the
application of these concepts to different business entities had to be covered. Capital gains
tax, GST and FBT as well as international tax were also examined. Extensive use of the
legislation was critical for students in explaining their understanding, with the tax legislation
text permitted in the test and exam.
The exam mark weighting for the subject remained constant at 60%, however the other
assessment requirements and weightings were often redesigned or refined in response to
student and academic feedback. In the semester of this innovation, 94 students studied the
subject with the same two teaching staff who had consistently taught the subject.
Assessment consisted of a peer‐reviewed test weighted at 10% (the same as the previous
semester) with tutorials and the new interview assessment weighted at 15% each. The
tutorials were marked on participation with the marks awarded for group discussion and
class presentation on weekly sight‐unseen problems.
4
Identifying the problem
The lead academic in the teaching of the taxation subject within the MPA program had
taught it since the inception of the MPA at the university in 2005. This has informed
continual reflection and enhancement. Recognition of consistent issues in teaching the
subject led to the distillation of a major problem with student learning.
A central issue was that despite continual refining of the assessment, many students
continued to have difficulty in applying sources of tax law to ambiguous situations. They
demanded solutions which they then rote learnt and applied to anything that looked
vaguely similar. This surface approach to learning would not suffice when the aspiration was
that students would research to find the latest updates for tax law and apply this to complex
“real life” cases. Changing this approach to learning was a major hurdle. Duff and McKinstry
(2007) have identified it to be the critical link to improving accounting education.
Changes to the subject materials and assessment to achieve any tangible benefits were
becoming more difficult. Alignment of teaching material has been carefully developed, over
the years so that each assessment, whether formative or summative, scaffolded the next
assessment. Use of peer feedback in the subject was constantly redesigned, with peer
review of the mid‐semester test (with instant feedback) proving to be a powerful learning
tool for the final exam. Underpinning all refinement was the understanding that students
needed to be encouraged to actively engage in their learning and this could be best
achieved through the use of assessment that engaged and stimulated them (Duff and
McKinstry, 2007).
One of the difficulties lay in the assignment (whether it was a case study or tax return and
essay) as it was often plagiarised, with the academics often unclear on how much of the
assignment was the individual student’s own work. The assignments were written to
replicate an authentic case study, so there was a lot of ambiguity inherent in the
information. The student’s interpretation of the information was often lost in the written
form so that a full assessment of their level of understanding was questionable. Although a
great deal of time was spent by the academics in providing feedback in marking the
5
assignment, some students often did not even bother to collect their marked assignments,
reflecting their focus on marks and not learning.
In previous semesters, students who failed the subject would often request that an
academic review their exam paper with them, and it was found that although some of these
students did not write the information down correctly, they were able to explain it clearly
verbally. It was then recognised that although written communication was assessed through
the test and the exam, there was very little assessment based on oral communication. This
is common for MPA programmes as noted by Yong et al (2011), and oral communication is
an area that is generally recognised as weak for accounting students (Grace and Gilsdorf,
2004).
That accounting graduates need effective communication skills cannot be disputed and
there are both national and international studies that consistently agree that both
accounting practitioners and professional groups rank communication skills as a key skill for
graduates to possess upon graduation and entry into the workforce (Gray, 2010; Grace &
Gilsdorf, 2004; Hancock et al., 2009; Yong et al., 2011). It is understandable that being able
to have discussions with both peers and professionals is a minimum requirement of the AQF
(2011). This would also enable the academic to more completely assess the student’s
understanding in complex case studies. The problem was therefore the creation of a
practical assessment tool that would be able to assess oral communication and provide
enough evidence to assess student understanding, whilst encouraging students to learn by
assisting in changing their approach to learning.
Designing a solution
The first stage in designing a solution to the problem was to ascertain whether it was
practically possible to assess students through an interview and whether it would achieve
the desired outcomes. Viva voces of the type that are used for PhD students were not
considered appropriate for MPA students. It was therefore necessary to design a smaller
version appropriate to the assessment task which would not increase marking workload.
Another consideration was how reliable it would be as an assessment tool, particularly with
6
international students. Viva voces are used primarily as assessment of learning, however the
focus of our assessment was to use it for learning. Although Gibbs and Simpson (2004‐05)
argued that we should design assessment that supports learning and “worry about reliability
later” (p. 3), it was a major concern. To address these issues the literature was explored and
with little found on the use of vivas in tax teaching, academics from different disciplines
were questioned on their use of vivas.
The use of different forms of assessment has been shown to influence students’ learning
approaches and the content that students learn (Scouller, 1998). Of particular interest is
that Scouller identified the significance of the perceptions of the students to the assessment
task. If students viewed the learning task as assessing low level cognitive processing (factual
recall commonly associated with multiple‐choice assessment), then they would employ
surface approaches to learning. However where students perceived that quality rather than
quantity of learning was being assessed (such as in an assignment), they would be more
likely to adopt deep learning approaches.
Duff and McKinstry (2007) identify a need for accounting educators to encourage lifelong
learning, and to develop the analytical and conceptual thinking skills of students, partly in
response to criticisms that accounting education has a narrow focus with an emphasis on
mechanical procedural approaches. They differentiate between three distinct approaches to
student learning: the deep approach, the surface approach and the strategic approach. It is
clear that the surface approach to student learning should be discouraged as much as
possible in a postgraduate taxation unit. The interview based assessment was partly
intended to encourage a deep learning approach by providing students with the opportunity
to communicate verbally their understanding of the completion of a challenging tax return
problem. The assessment gave students an insight into the how the knowledge acquired in
the subject could be applied in something close to a “real life” taxation scenario.
According to Duff and McKinstry (2007), one of the ways that the deep approach to learning
can be differentiated from the strategic approach is that in the former there is an intention
to understand, whereas in the latter there is the intention to obtain the highest possible
grades. On this description, it would appear that there are many students who adopt the
7
strategic approach to learning. The imperative of students to obtain the highest possible
grade is something that cannot be overlooked from an educator’s perspective. It is arguable
that even where an assessment for learning is implemented, it must be for marks in order
for it to be effective. That is, awarding marks should not, of itself, conflict with an
assessment being formative, as the experience with the interview based assessment would
suggest.
The increasingly diverse student cohort was argued by Brown (2004‐05) to be the driver for
change in teaching from a tutor‐led approach to focusing on learning outcomes. Brown also
argued that employability of graduates must be a factor driving their education, and
therefore assessments should be designed to be practice‐orientated, with authentic
methods used: “assessing what they claim to assess, not just what is easy to assess” (p. 83).
The validity of assessment was also a point questioned by Brown, and was argued to be met
when the process was transparent and the evidence matched the criteria. However
Torrance (2007) questioned the value of such transparency and argued that it encouraged
instrumentalism, which reduced learning. Torrance argued that to avoid such an outcome,
communities of practice should be developed so that the students would be able to
“explore and interrogate criteria, rather than accept them as given” (p. 292). Using
communities of practice to empower students to take charge of their own learning was a
central argument of Elwood and Klenowski (2002) and O’Donovan, Price and Rust (2008).
The use of assessment and its outcomes has generated great debate in the literature, which
is understandable when some argue that it is the assessment and not the teaching that has
the greatest influence on student learning (Gibbs and Simpson, 2004‐05). Boud (1990)
argues that there are two main purposes of student assessment; firstly, improving the
quality of student learning and secondly, formalising the accredidation of knowledge.
According to Boud (1990), the former is achieved through formative assessment or
assessment for learning and the latter is achieved through summative assessment or
assessment for the record. Research such as that done by Craddock and Mathias (2009) has
revealed that formative assessments can have a greater impact on the learning process than
summative assessments. In reviewing the history of assessment, Loaker, Cromwell and
O’Brien (1985) define assessment as a “multidimensional process of judging the individual in
8
action” (p. 4.). They explicitly differentiate assessment from testing and measurement,
which is commonly considered assessment of learning. They argue assessment is assessing
the ability of an individual judged against criteria, rather than their knowledge, with limits to
marked answers.
Prior research has recognised that mere assessment of learning is too narrow, as
assessment has an important role for learning and as learning. Boud and Falchikov (2005)
advocate the significance of understanding the multiple roles that are fulfilled by
assessment, with assessment as learning representing learning that lasts. Stiggins (2002)
explains that assessment for learning is more than formative assessment (as recognised by
others such as Wiliam et al, 2004), since it requires students to be involved in the process
and encourages students to keep learning. This view is shared by Elwood and Klenowski
(2002) by emphasising the role of effective feedback in student understanding of their
learning. Torrance (2007) argues that the emphasis on assessment of learning has shifted to
assessment for learning with the latest version being assessment as learning with criteria
compliance replacing learning. Further, similar to Fook and Sidhu (2010), Taras (2010) notes
that higher education remains focused on grades rather than on learning and questions the
ability to produce “confident, independent and autonomous learners” (p. 502).
From this literature it was ascertained that the primary purpose of our assessment should
be in providing feedback that is effective for student learning rather than weighting it with
additional marks. The focus should be on using the assessment as a learning tool rather than
a grading tool. With this in mind, the use of oral assessment as a form of learning was
explored in the literature. Joughin (1998) notes that whilst oral assessment is common in
medicine, law and architecture education, it is less common in other discipline areas and it is
often thought as a form of alternative assessment. He notes that oral assessment may be
preferred if there is an imperative to ensure that the responses are actually the student’s,
hence it is good for addressing any plagiarism issues.
In analysing results from the use of oral assessment, Singh (2008) found that students
reported that they studied a lot more for their oral assessment in comparison to their
written assessment and that whereas they may have memorised for the latter, they read to
9
understand for the former. On the point of learning during the assessment, Singh found that
students perceived the oral assessments as a tool, which bridged the gap between their
learning environment and the real world. Pearce and Lee (2009) describe a viva as a means
of interactive dialogue between the student and the assessor which allows the student to
demonstrate their strengths and which allows the assessors to differentiate between
superficial and deeper knowledge by way of in‐depth questioning. However they question
whether the interview process is value free.
Joughin (1998) explains that although validity may be considered one of the strengths of
oral assessments, there may be questions about the reliability of oral assessment; for
example, there is the possibility that if two different assessors were to examine a candidate
they may not reach consistent conclusions. Munoz and Alvarez (2003) also address the issue
of inter‐assessor reliability in oral assessments by way of their study in which they found
some inconsistencies between assessors. One of the conclusions of their study is that
uniformity of assessment criteria will be enhanced when there is a frequent and systematic
examination of inter‐assessor reliability.
A mini‐viva used solely for discussion and feedback was discussed by Carless (2002) in
teaching the Bachelor of Education programme in Hong Kong. An assignment was given to a
group of three students (17 groups in total) and after they had submitted their assignment
they were required (as a group) for 15‐20 minutes to justify or clarify their approach. It was
the written assignment that was marked. Carless noted that overall the assignment and
mini‐viva elicited a positive response from students and they were able to consolidate
learning, however the academic and researchers involved held some reservations. These
reservations included having more than one academic involved in the subject (as it was
difficult educating other academics in the process), the size of the cohort (a small cohort
was necessary), and gaining consent of students to participate in an innovative assessment
method. However the biggest issue identified was the increase in workload for the academic.
In using mini‐vivas, Sayce (2007) reflected on some of the issues identified by Carless (2002)
in applying it to a large cohort of international MA business students in research methods.
Sayce found that in defending their written work, students were encouraged to deepen
10
their research and this more than made up for their increase in anxiety. Students submitted
a written proposal for their dissertation and two weeks later had a ten‐minute mini‐viva
with their tutor on what they had written. In reporting on students’ perceptions of a mini‐
viva, those who saw it more as an interview rather than a spoken exam were less focused on
the marks and more on the experience. Although still nervous these students were less
anxious. Huxham et al. (2012) noted the potential for an oral assessment to induce more
anxiety amongst students than a written assessment; however, they also noted that this, of
itself, was not necessarily negative as it may contribute to a better than average
performance. This view is consistent with the assertion of Gabriel and Griffiths (2002) that
some degree of anxiety is necessary for learning.
In analysing this literature, oral assessment appeared to be a useful way to measure
individual student’s understanding. With the use of the term “viva” generating a focus on
marks rather than learning, the term “interview” would be preferable. Student anxiety may
be an issue, but it may also improve performance.
The literature review revealed that the usefulness of interview assessments in higher
education subjects is a topic of interest and debate among the international higher
education community. The research is considered original since it is specifically concerned
with an interview assessment in a university postgraduate taxation subject. This appears to
be an area in which there has been a limited amount of research to date.
Just how practical vivas, or interviews, were as an assessment tool, particularly in the view
of workload, was an aspect that was not well documented in the literature. To address the
practical aspects and increase the confidence in our use of such an assessment tool,
academics from different disciplines within the university (Pharmacy and Asian Languages)
were questioned. In Pharmacy, vivas were common across the degree, so students were
scaffolded in the use of such an assessment. These students would pass or fail the whole
degree based on their final viva and achieving competencies. Some of the practical issues
suggested by the academics were: not to interview for more than five hours per day and to
11
take many breaks, allow 15 minutes per interview, minimise formality so that the student
felt relaxed and to record the interview for quality assurance and appeals purposes.
The intervention (solution)
Before the new assessment was undertaken it was important to assess this particular
cohort’s approach to learning, so a survey was conducted in the very first lecture to gather a
basic understanding. It was found that over half (52%), of the students perceived that
passing tax would be difficult, predominantly based on what they had heard from previous
students. This would influence their approach to learning from the outset. As one student
wrote:
This course determines whether I can graduate in time and I heard it is very difficult. I’m a
little afraid.
The same student did not like multiple choice questions or assessed log books – finding them
“meaningless”. This implies that the student would appreciate assessment that encourages deep
learning. This expectation of an increase in effort due to the difficulty of the unit was quite
consistent:
I heard from some seniors…that this unit is difficult, not easy to pass and (I would) need to
put more effort and time on it.
However the majority of students (20%) preferred multiple choice questions and group written
assignments (17%) as a form of assessment, with short answer test and exams (5% each) the least
preferred. The reason provided was primarily the ease of obtaining marks, as compared to the
difficulty in writing in a limited time. Interestingly, one quarter of the explanations regarding their
least preferred assessment was on group work and the problem with “free riders”.
I don’t really like working with mates with no contribution but also get high mark. It is not
fair for other students in the group.
A 15 minute interview‐based assessment was created, with 10 minutes of interview and 5
minutes of immediate feedback. Marks were awarded to students on the basis of a criteria‐
based rubric. It is the view of Munoz and Alvarez (2003) that rubrics in oral assessments
provide for increased consistency in grading students as well as enhancing their
understanding by providing a set of expectations about what is being assessed and the
12
standards required. However, although rubrics use explicit criteria and descriptors to
explain what is being assessed and how, it is the view of Price and O’Donovan (2006) that
they may be insufficient in providing a precise articulation of assessment standards as some
of the terms used to articulate these standards may be best understood in relative rather
than absolute terms. They explain, by way of example, that what is ‘highly evaluative’ or
‘reasonably coherent’ is influenced by the assessor’s knowledge of the context. Therefore it
was important to give the rubric to students at the beginning of the semester, and for the
academics to jointly mark the first two students and moderate recorded interviews.
The interview assessment was hoped to make students more accountable for their own
learning, which, in turn would encourage a deeper level of engagement with the material.
Immediate feedback was considered to assist with this end. The interview‐based assessment
was also expected to assist students in developing their communications skills, an area that
has long been recognised as a key deficiency in accounting graduates. These communication
skills are required to be at a high enough level to engage in conversation with professional
peers – both “specialist and non‐specialist audiences” AQF (p. 17, 2011).
From the literature, assessment for learning and as learning developed a higher level of
understanding for the students of their own learning. With this in mind, the marks allocated
for the interview‐based assessment were only 15%, so although it was not purely a
formative assessment it was hoped that the focus would be on learning rather than marks. It
was recognised that if no marks were awarded, the effectiveness of the interview as
assessment for learning may have been compromised since many students would not have
considered it as important as their marked assessment tasks. This is consistent with
literature such as Boud (1990) which states that students prefer graded assessment since it
provides a type of currency indicator as to what teachers value. An additional benefit of the
interview assessment was that by awarding marks, marking workload was reduced.1
The interview‐based assessments required students to complete a tax return based on a
complex, “real life” case study. The tax return was used as the tool for the interview and
1 Marking was reduced from 30 minutes per written assignment to 15 minutes per interview.
13
was not assessed separately. Students were required to sign‐up to their preferred interview
time and they were also allowed to choose between two teaching staff as their interviewer,
subject to their availability. Interviews began the day after the assignment was submitted.
Each student was interviewed individually by one of two teaching staff. Assessments were
also recorded to allow for moderation and to address re‐mark requests. At the end of the
interview, the student interviewee assisted the academic in marking their interview
performance using the rubric, so that feedback was immediate.
The interviewers asked questions of varying levels of difficulty with the view that the
responses would provide an indication of the student’s depth of understanding of the topics
covered in the subject. Whilst students were not required to sign a confidentiality form
before their interview, every effort was made to ensure that no two interviews asked
exactly the same questions as the case study was complex and ambiguous.
Evaluating the impact
Students enrolled in the subject were invited to participate in two separate surveys, the first
survey was conducted at the start of the semester before any teaching in the subject had
taken place and the second after the interview assessment had been completed. Students
also completed the university student evaluation of teaching and learning survey (SETL) at
the end of the teaching period and some students agreed to be involved in further
interviews to reflect on the assessment after their final results were released.
Surveys and discussions after the interview reflect the deeper level of learning:
Before I attended this unit I had heard lots of horrible stories about tax. That is why I
put a lot of effort in at the very beginning. Now I am very confident because I did well
in my interview and peer test. I love the interview and style of tutorials.
I think it (the interview‐based assessment) was really awesome, before that I did not
know how I can do that, everyone said interview was horrible and tax law is horrible
14
but I realised I can do this. After the interview I got a great compliment from (the
lecturer) and I realise I can do this, it spurred me on for the whole semester.
Best thing is it helps to practice interview skills and before the interview it makes
students prepare. The worst thing is the workload is too much. During the weeks I
prepared for the assignment I didn’t do anything else for any other units….
Surprisingly, some students even understood the value of ambiguous material with no
standard answer. The academics considered this level of insight from students to be a
significant step forward.
No standard answer for each question (means that) students are encouraged to
argue based on their understanding.
Many students recognised the value of the interview in assessing individual understanding
and therefore eliminating plagiarism.
The interview method is one useful method to check understanding (of) the subject
by the student.
I like this method. Actually, there are some student(s) (who) cheat and copy other
students’ (work). Interview is the best way to adjust (for this)…
I think it can test whether a student does the task on his/her own or not…
The alignment of the interview with the rest of the assessment in the subject was important
and it was also understood by students.
(The) interview helped me and encouraged me to prepare for the exam.
Even other students recognised that many students only focused on marks and how that
was detrimental to deep learning:
15
Most of them just want to get good marks, they do not want to learn, they just want
to pass, then D, then HD. They do not really want to remember, not even anything
from last semester. They are interested in the mark only; this is the problem I see.
Questions about reliability, fairness and insufficient time were raised, especially when the
hard copy of the assignment was not marked:
Assessment is too subjective. Some students just talked to interviewer and they got
high marks. Actually they know nothing about tax law. Other students prepared so
hard with their assignment but lecturer never looked carefully at their assignment. So,
in conclusion, it’s unfair to give marks (that) only depend on (a) 15 minute interview.
Other students held the view that the interview assessment was fair as it removed
plagiarism and assessed their learning. Although the issue of reliability in marking was
addressed by creating a marking rubric and distributing it to students at the beginning of the
semester, jointly marking the first couple of students and re‐assessing recorded interviews,
some students’ perceptions of the lack of reliability was of concern. Munoz and Alvarez
(2003) refer to some advantages of rubrics including that they allow for objective and
consistent assessment and promote student awareness of the criteria used in assessments.
It appeared that some students were dissatisfied with the lack of transparency of how each
performed in the interview.
Perceptions of unfairness are not exclusive to interview‐based assessments. For example,
Falchikov and Boud (2007) assert that assessment is not experienced as a rational process,
from the perspective of the person being assessed, and that emotional responses to being
assessed are universal. They explain that the ways assessment are experienced are a
function of what the person being assessed brings to the event as well as the event itself,
with problems occurring where the teacher objectifies their judgments and the student
subjectifies theirs.
16
Time limitations were a perceived issue, especially for students who had difficulty in
conversing in English. However as oral communication was a critical part of the assessment,
and there was only one week to complete all the interviews, the amount of time per
interview could not be extended. Notwithstanding this, it was usually quite clear within the
ten minutes whether a student could apply tax law or if they lacked understanding.
As the intervention was a new assessment method and students could not rely on past
students for prior knowledge of what was required, this added to their anxiety in some
cases. However, as the interviews were conducted in a friendly manner, usually once the
interview commenced the nerves calmed and students reported that they were more able
to answer questions effectively. With the majority of students in the subject having English
as their second language, this may also have impacted on their anxiety levels:
Yes they would feel very nervous, but that is very common in China. The education
system in China does not focus on opening mind, we can see some of the Chinese
students do not really like to communicate, they do not like to communicate with
anyone they just do their work and go away, they do not realise the importance of
socialising, so but they will change, we come here to learn, we learn we become
better, it might be a cultural difference, but students can change…. I think that is
might be a different way but I agree that an interview method is a fantastic method.
Students also provided advice for future students on the interview:
If they want to be quite serious about taxation I would tell them to practice (their)
speaking skills before they attend the interview because it is very useful, they will be
very nervous so I suggest (to) them to practice English, practice before they attend.
Responses from the university‐wide student evaluation (SETL) of the subject were very
positive. Compared to the previous semester, the mean score for each of the questions had
either remained the same or increased. The largest increase was in response to the question
17
“I gained a good understanding of the subject matter” from 4.1 to 4.42, and an increase
from 4.2 to 4.4 to the question “In this unit I was encouraged to think”.
The student feedback indicated that although the interviews were a good assessment
method overall, they could be improved by allowing more time and by making it clear that it
was only the interview and not the written tax return that was being assessed. Some
students thought some mark should be given for the written component.
The fact that, in the second survey, over one in four students indicated that individual
interviews were one of their preferred assessment methods can be considered a good result
for the first semester of the interview based assessment. This is in the context of a
significant proportion of students who preferred easier forms of assessment such as
multiple choice tests, as per the initial survey responses. Furthermore, there were some
students who appeared to have some strong objections to the written assignment not being
assessed which may have been based, in part, on their expectations about assessment
generally. These expectations may have been informed by how these students were
assessed in other subjects. The academics who taught the subject considered that such
preconceived ideas about assessment may have been quite difficult to change. Despite this,
there appears to have been some success overall in changing students’ perception of
assessment methods which may be indicative of changes in their approaches to learning.
The extent of any such change, however, is difficult to quantify. It also noted that there
were some students who provided positive feedback about the interview assessment, but
did not nominate it as one of their most preferred assessment methods.
Implications
It is difficult to assess whether student perceptions about learning were altered, although
the general consensus was that the interview process was a good form of learning. The
overall total results for the subject improved overall by 5%, which may reflect a deeper
understanding by the students. The increase in marks was a result of improvement in the
final exam with the average assignment mark similar to the previous semester. The failure
rate in the subject compared with the previous semester dropped by 5%, the high
2 SETLs had a 5‐point scale with 5 being Strongly Agree, and 1 being Strongly Disagree, with 3 being Neutral.
18
distinctions increased by 4% and distinctions increased by 7%. This suggests that there were
several benefits of the interview assessment. As well as encouraging students to view
assessment as helpful for learning, most students demonstrated that their oral
communication skills were at the required level and overall student performance in the
subject increased.
The academic workload was not increased from the addition of the interview‐based
assessment, as the written assignment was not marked. The assessors believed that the
marks awarded to the students were a much better indicator of student understanding of
the subject than those from the previous semester’s written assignment. The assessors did
not consider plagiarism to be an issue.
It is difficult to completely address the fairness of assessment issue identified by some
students. This has been identified by Struyven, Dolchy and Janssen (2005) as an important
issue for students with clear assessment criteria and feedback important in this perception.
As the students all had the interview assessment criteria in the subject outline and because
feedback was provided immediately and discussed with the student using the criteria‐based
rubric, it appears that the mere perception of differing marks between students and
assessors was the issue. As the students were assessed on their interviews and they were
not able to hear each other, the marking of the interviews was not as transparent to
students compared to a written assessment. This meant that fairness is likely to have been a
perception issue, 3 especially as students had not experienced an interview‐based
assessment before.
Conclusion
Part of the motivation for the interview‐based assessment was the changing requirements
for MPA graduates in light of the University’s prescribed graduate attributes for
communication skills. In the changing higher education environment, with large
international student numbers and recognition that a mastery of oral communication is
required, innovations such as this interview‐based assessment should be considered.
3 The difference between assessors’ marks was 0.2 out of 15 marks.
19
Through using assessment not only as assessment of learning, but also for learning, as well
as the integration of marks and the provision of detailed and immediate feedback, student
learning is encouraged and the workload for academics can also be reduced. With workload
another factor that is subject to constant change, new innovations in assessment and
teaching may help address such change. The research is considered to be part of an ongoing
research project where the usefulness of the intervention as an assessment for learning will
be evaluated over consecutive semesters. This paper sets the direction for future research
on the topic and reports on the research findings for the first semester the interview
assessment was in operation.
It can be concluded that there was a significantly greater amount of positive feedback than
negative feedback from students on the interview assessment. One of the most common
items of negative feedback was to do with perceived unfairness. Whilst it is acknowledged
that there may be some degree of subjectivity in this type of assessment, overall however,
the assessment is not considered any less rigorous than the written assignment that it
replaced. An incomplete understanding, by some students, of what was being assessed may
partly explain some of the negative feedback on the interview assessment.
A second aspect of the interview‐based assessment which was unpopular with some
students was the limited amount of time for each interview. This type of criticism may be
based on the fact that some students perceived that there was a mismatch between the
long amount of time they spent working on the written component and the short amount of
time that they were assessed by interview. The limited time for the interview, however, is
not considered to detract from its effectiveness as an assessment tool.
The increase in the number of students who attained overall distinction and high distinction
marks as well as the decrease in the number who failed the subject are indicative of the
effectiveness of the interview‐based assessment as form of learning for assessment. Trends
in results over several semesters will be monitored as it may be too early to draw
conclusions on correlations between interview marks and overall marks. There is also the
possibility that the first group of students in the subject who underwent the interview
assessment may have attained similar final marks regardless of the type of assessment used.
20
After one semester our overall conclusion on the interview‐based assessment is that it is a
superior form of assessment, compared to the previous written assessment that it replaced,
in achieving assessment for learning. In this sense, it is considered to have met its
overarching goal, subject to the qualifications previously discussed. Given the lack of
experience in using this type of assessment previously in the MPA program, changes may
need to be introduced over time to increase its effectiveness in response to the
shortcomings identified by the lecturers as well as feedback from the students. An example
of a possible change to the format of the interview‐based assessment is for two lecturers to
interview a group of two or three students. This may decrease the amount of negative
feedback about perceived unfairness as each student would be assessed by two lecturers
and there would be more transparency about how marks are awarded to other students.
Furthermore, the anxiety of students in an interview may be decreased when their peers are
present.
21
References
Australian Qualifications Framework (AQF) 2011, Australian Qualifications Council, South Australia Boud, D. 1990, Assessment and the promotion of academic values, Studies in Higher Education, 15:1, 101‐111. Boud, D. & Falchikov, N. 2005, Redesigning assessment for learning beyond higher education, HERDSA Conference, 34:41 Brown, S. 2004‐05, Assessment for learning, Learning and Teaching in Higher Education, 1, 81:89 Carless, D.R. 2002, The ‘mini‐viva’ as a tool to enhance assessment for learning, Assessment & Evaluation in Higher Education, 27:4, 353‐363
Craddock, D. & Mathias, H. 2009, Assessment options in higher education, Assessment & Evaluation in Higher Education, 34:2, 127‐140.
De Lange, P. & Watty, K. 2010, Accounting education at a crossroad in 2010 and challenges facing accounting education in Australia, Accounting Education: An International Journal, 6, 625‐630
Duff A. & McKinstry S. 2007, Students’ approaches to learning, Issues in Accounting Education, 22:2, 183‐214
Elwood, J. & Klenowski, V. 2002, Creating Communities of shared practice: the challenges of assessment use in learning and teaching, Assessment & Evaluation in Higher Education, 27:3, 243‐256
Evans, E., Burritt, R., Guthrie, J. 2010, Accounting Education at a Crossroad in 2010, Institute of Chartered Accountants in Australia, Sydney
Falchikov, F., & Boud., D. 2007, Assessment and emotion, the impact of being assessed, in Boud., D & Falchikov, F., (eds), Rethinking Assessment in Higher Education: Learning for the Longer Term. Fook, C. Y.& Sidhu, G. K. 2010, Authentic assessment and pedagogical strategies in higher education, Journal of Social Sciences, 6:2, 153‐161
Freeman, M. & Hancock, P. 2011, A brave new world: Australian learning outcomes in accounting education, Accounting Education: An International Journal, 20:3, 265‐273
Gabriel, Y. & Griffiths, D.S. 2002, Emotion learning and organizing, Learning Organisation, 9: 5, 214‐221.
Gibbs, G. and Simpson, C. 2004‐05, Conditions under which assessment supports students’ learning, Learning and Teaching in Higher Education, 1, 3‐31
Grace, D.M. & Gilsdorf, J.W. 2004, Classroom strategies for improving students' oral communication skills, Journal of Accounting Education, 22:2, 165‐172 Gray, E. 2010, Specific oral communication skills desired in new accountancy graduates, Business Communication Quarterly, 73:40
Hancock, P., Howieson, B., Kavanagh, M., Kent, J., Tempone, I., Segal, N. 2009, Accounting for the Future: More than Numbers. A collaborative investigation into the changing skill set for professional accounting graduates over the next ten years and strategies for embedding such skills into professional accounting programs, Australian Learning and Teaching Council, Sydney
22
Huxham, M., Campbell, F., Westwood, J. 2012, Oral versus written assessments: a test of student performance and attitudes, Assessment & Evaluation in Higher Education, 37:1, 125‐136 Joughin G. 1998, Dimensions of Oral Assessment, Assessment & Evaluation in Higher Education, 23:4, 367‐378
Loacker, G., Cromwell, L., O’Brien, K. 1985, Assessment in higher education: to serve the learner, Conference paper for Assessment in Higher Education, Columbia, Oct 13‐15
Munoz, A.P. & Alvarez, M.E. 2003, Estimating the validity and reliability of an oral assessment instrument, REVISTA Universidad EAFIT, 39:132, 65‐75. Pearce G. & Lee G. 2009, Viva voce (oral examination) as an assessment method: Insights from marketing students, Journal of Marketing Education 31:120, 120‐129.
Price, M. & O’Donovan, B. 2006, Improving performance through enhancing student understanding of criteria and feedback, in Bryan, C. & Clegg, K (eds), Innovative Assessment in Higher Education.
Sayce, S. 2007, Managing the fear factor (or how a mini‐viva assessment can improve the process of leaning for international students), The Electronic Journal of Business Research Methods, 5:2, 83‐9.
Scouller, K. 1998, The influence of assessment method on students’ learning approaches: Multiple choice question examination versus assignment essay, Higher Education, 35:4, 453‐472 Singh, P. 2008, Reflection‐in‐and‐on‐Action in Participatory Action Research: Toward Assessment for Learning, Systemic Practice and Action Research, 21, 241‐251 Stiggins, R.J. 2002, Assessment crisis: The absence of assessment for learning, The Phi Delta Kappan, 83:10, 758‐765
Taras, M. 2010, Using assessment for learning and learning from assessment, Assessment & Evaluation in Higher Education, 27:6, 501‐510
Torrance, H. 2007, Assessment as learning? How the use of explicit learning objectives, assessment criteria and feedback in post‐secondary education and training can come to dominate learning, Assessment in Education, 14:3, 281‐294. Wiliam, D., Lee, C., Harrison, C., Black, P. 2004, Teachers developing assessment for learning: impact on student achievement, Assessment in Education, 11:1, 49‐65 Yong, J., Ryan, S., Yap, C., Neelam, G. 2011, A case study in the failure of graduate attributes in accounting education, Conference paper for HERDSA, Gold Coast, Australia