Inclusive teaching design to better support all students ...mams.rmit.edu.au/mc0b2wexcrylz.pdf · Inclusive teaching design to better support all students: Toward a best practice

Inclusive teaching design to better support all students: Toward a best

practice model for RMIT

Strategic objective(s) addressed:

• To support the participation, retention and/or success of low Socio-Economic Status (SES) higher education students in undergraduate studies

Project leader: Associate Professor Andrea Chester Contact details: [email protected] 9925 3150 Project team members:

• Mr Tom Coverdale, Study and Learning Centre • Professor Peter Wilson, School of Health Sciences • Dr Berenice Nyland, School of Education • Mr Shahin Rahimigolkhandan, Project Assistant • Professor Ken Greenwood, School of Health Sciences

Learning and Teaching Investment Fund 2011

Final Project Report

LTIF Final Report 2011 Page 2

Contents Executive summary ......................................................................................................................3

Outcomes ....................................................................................................................................3

Project outcomes and impacts ......................................................................................................4

Project One: Staff Survey of Inclusive Teaching Attitudes and Behaviour.......................................4

Project Aims ......................................................................................................... 4

Participants .......................................................................................................... 4

Measures............................................................................................................. 5

Procedure ............................................................................................................ 5

Results ............................................................................................................... 5

Recommendations ................................................................................................. 6

Project Two: Evaluation of Inclusive Teaching Practices................................................................7

Project Aim .......................................................................................................... 7

Participants .......................................................................................................... 7

Measures............................................................................................................. 9

Procedure .......................................................................................................... 10

Results ............................................................................................................. 10

Recommendations ............................................................................................... 14

References.................................................................................................................................15

Dissemination strategies and outputs..........................................................................................15

Evaluation of project outcomes ...................................................................................................15

Budget report .............................................................................................................................17

Certification....................................................................................Error! Bookmark not defined.

Appendix A: Inclusive Teaching Discussion Paper.......................................................................19

Appendix B: Staff Survey Results ...............................................................................................37

Appendix C: Inclusive Teaching Practice Training Outlines.........................................................43

Appendix D: Evaluation of Inclusive Teaching Practices Results.................................................45

Appendix E: Financial Statement ...................................................Error! Bookmark not defined.


Executive summary Inclusive teaching is an attempt to teach in a way that embraces as many forms of difference as possible across the student population, including low SES, differing language abilities and learning preferences, as well as disability (Grace & Gravestock, 2009). Inclusive teaching is shifting from relying on post hoc adjustments to accommodate marginalised students to a “universal design” approach, which aims to include diverse student cohorts pre-emptively, thus improving the learning experience for all students. A range of teaching strategies have been proposed as “inclusive”, however, few studies have examined their efficacy in higher education settings. This inclusive learning and teaching project had two aims:

• To describe the inclusive teaching strategies currently practised, as well as attitudes towards and knowledge of inclusive teaching amongst RMIT staff.

• To pilot three low-cost, high impact strategies (note-taking and note-making, concept mapping, and argument mapping) in undergraduate higher education curriculum at RMIT across all three Colleges, including students at RMIT Vietnam. A mixed method design, using a quasi-experimental approach, complemented by focus group data, determined efficacy and acceptability of these strategies.

Results from the staff survey indicated that less than one quarter of respondents were familiar with the concept of universal design, however, staff beliefs, goals and expectations, delivery modes, as well as feedback and assessment practices were largely consistent with inclusive teaching principles. In general staff expressed interest in learning more about specific inclusive teaching practices. Results of the student efficacy study support the value of note-taking and note-making as a useful intervention. Little empirical support was noted for concept and argument mapping, although the results need to be read with caution given the limitations of the project. A series of recommendations are proposed, including the value of inclusive teaching professional development for staff and embedding training in note-taking and note-making into the first year curriculum.

Outcomes Outcomes of the project include:

• Collation of information about current inclusive teaching practices and attitudes amongst academic staff at RMIT, to support institutional change.

• A rigorously designed mixed-methods evaluation of three low-cost, high-yield strategies to support the development of evidence-based practice with recommendations for inclusive teaching practices, which can be implemented to better support all students.

• Proposed development of online resources to support staff engaged in inclusive teaching and showcase good practice.

• One journal article in review: Nyland, B., Coverdale, T., Chester, A., Rahimigolkhandan, S., & Wilson, P. (in review). Inclusive teaching: Using computer software mapping tools to assist higher education students in their studies. Studies in Higher Education.

• Two journal articles in preparation, including Nyland, B., Coverdale, T., Chester, A., Rahimigolkhandan, S., & Wilson, P. (in preparation). Pedagogy of access: Powerpoint, inclusive practice and the construction of knowledge.


Project outcomes and impacts The project had four main aims:

• To collate information about current inclusive teaching practices across the university to

facilitate sharing of existing strategies. • To build an understanding of existing inclusive teaching attitudes and knowledge to

support institutional change. • To evaluate low-cost, high-yield strategies to support the development of evidence-based

practice and lead to recommendations for inclusive teaching practices, which can be implemented to better support all students.

• To provide online resources to support staff engaged in inclusive teaching and showcase good practice.

To meet these aims, two separate projects were designed: (i) a staff survey, examining attitudes towards and behaviour consistent with inclusive teaching principles and (ii) an evaluation of the efficacy of a range of inclusive teaching practices. These two projects and their outcomes are described in detail below.

Project One: Staff Survey of Inclusive Teaching Attitudes and Behaviour

Project Aims

• To collate information about current inclusive teaching practices across the university to facilitate sharing of existing strategies.

• To build an understanding of existing inclusive teaching attitudes and knowledge to support institutional change.

Although inclusive teaching practices underpin good teaching at RMIT, little is known about how well staff understand inclusive teaching. In addition, we do not currently have reliable information on staff attitudes towards inclusive teaching or their practice at RMIT. Anecdotal evidence collected during workshops run by the Inclusive Teaching Working Group suggest a relatively low level of interest in inclusive teaching amongst HE staff and a wide range of misconceptions. This project was therefore designed to describe the knowledge, attitudes and teaching practices of RMIT academic staff, providing information on inclusive teaching that can be used to guide future initiatives.

Participants

Of the 93 staff who responded to the survey, 51.6% were female. Staff from all three Colleges completed the survey, although representation was not even, with 62.6% of respondents coming from SEH, 29.7% from DSC and the remaining 7.7% from Business. Staff varied widely in their teaching experience. Respondents included staff in their first year of teaching through to staff who had been teaching for more than 40 years. The mean number of years of teaching practice was 18.9 (SD = 11.7).


Measures

The survey questions were developed from the Inclusive Teaching Discussion Paper prepared by Tom Coverdale (Appendix A), supplemented by a review of the literature conducted for the project. In addition to some brief demographic questions, the survey covered four aspects of inclusive teaching: beliefs; goals and expectations; delivery modes; and feedback and assessment practices. The final section of the survey assessed staff experience in working with students from a range of equity groups. The internal reliability of the sub-scales of the measure is reported in Table 1 (Appendix B). Results indicated that all sub-scales were internally coherent, with the exception of the beliefs sub-scale, which initially returned an alpha level of .53. One item was removed from this sub-scale (“Presenting assignments in different ways gives some students an unfair advantage”) to improve the reliability of this sub-scale to an acceptable level.

Procedure

EduTAG developed the survey for online administration. Following approval from the SEH College Human Ethics Advisory Network (CHEAN), the survey, open to all academic and teaching staff, was advertised via RMIT Update from October 24 to November 30 via the website http://emedia.rmit.edu.au/survey/inclusive-teaching. In addition, emails were sent to staff via the Academic Development Groups in each College.

Results

Frequency of responses to the five sub-scales (beliefs, goals and expectations, delivery modes, feedback and assessment and awareness of equity groups) are presented in Appendix B and summarised below. Although less than one quarter of respondents (23.7%) were familiar with the concept of universal design, results suggested a reasonable knowledge of inclusive teaching principles amongst RMIT staff. For example 75.3% of staff endorsed the belief that inclusive teaching generally benefits the whole class. In addition, goals and expectations were typically well aligned with inclusive teaching approaches, although more attention could be given to welcoming students from diverse backgrounds; 26.9% of staff either never or rarely did this. Staff reports of their delivery of material were also largely consistent with inclusive teaching principles. For example, 90.4% of staff chunked information and 76.3% of staff provided overviews and summaries. Areas receiving less attention include previewing activities to activate prior knowledge, incorporating diversity content into the curriculum and tailoring specific instructional techniques to specific instructional goals. Staff typically used a range of teaching modes consistent with inclusive teaching principles. Of these peer mentoring was the least well used, with 39.8% of staff noting they rarely or never use this approach. In terms of assessment and feedback, the respondents typically reported behaviour consistent with inclusive teaching principles, with frequent provision of feedback and use of detailed assessment criteria. Areas less well attended to include: (i) self and peer assessment as a strategy to increase formative feedback; 39.8% of staff noted that they rarely or never use these approaches and (ii) use of rubrics, which 25.9% of staff noted they rarely or never use. Equity groups extend beyond low SES to include cultural and linguistic diversity, membership of religious minorities, disability and sexual orientation. The final question on the staff survey asked staff about their experience teaching students from a range of equity groups. Results suggested that nearly all staff were aware they had taught students from various equity groups in the last five years, mostly notably CALD students. Staff were also aware of religious and socio-economic


diversity in their classes and of having taught students returning to study after a long break and those with a disability. Staff were least likely to know whether their students were survivors of torture, were gay or lesbian and whether they were refugees. Responses to this question raise the issues of the extent to which staff ought to seek out information about the equity characteristics of the cohorts they teach, the ethics of doing this and, indeed, how such information might be obtained. Regardless of the answers to this question, the results obtained suggest that staff are aware of considerable diversity within their classes. The relatively small number of respondents, particularly from Business, prevented an analysis of the results by College, however, the impact of years of teaching experience was tested. Modest, but significant correlations were noted between years of teaching experience and only a small number of items. More experienced staff were more likely to agree that it is important to address the challenges faced by students from lower socio-economic backgrounds (r = .22) and those with lower English proficiency (r = .25). Years of experience appears to have little impact on inclusive teaching attitudes or behaviour. The results of the survey need to be read with some caution. The small sample size and large proportion of SEH staff relative to the other two Colleges raise questions about representativeness. Although we ensured that advertisements recruiting staff invited all staff to respond regardless of knowledge about inclusive teaching, it is possible that staff who completed the survey were those with already favourable attitudes towards inclusive teaching and those already reasonably well informed. It is therefore possible that the results are an over-estimate of inclusive teaching knowledge and behaviour amongst RMIT teaching staff. Further, it is always possible that self-reported estimates of behaviour may be inaccurate. Staff may have, either consciously or unconsciously, over-estimated descriptions of their good teaching. Observational techniques, including peer partnerships, may help provide additional corroborating data in this area and facilitate insight for individuals into areas for change. The small sample size may reflect the time of year in which the survey was conducted, however, it may also reflect a low priority and/or lack of understanding about inclusive teaching amongst academic staff at RMIT.

Recommendations

The survey indicated several areas for potential attention and professional development. If, as suggested above, the results represent an over-estimate of inclusive teaching knowledge and behaviour, then providing accessible information for staff on the following areas might be considered a priority:

• welcoming students from diverse backgrounds • previewing activities to activate prior knowledge • incorporating diversity-related content into the curriculum • tailoring specific instructional techniques to specific instructional goals • establishing peer mentoring • designing and incorporating self and peer assessment as a strategy to increase formative

feedback • developing and using rubrics

It is recommended that peer partnerships be implemented to provide staff with feedback and opportunities to reflect on their inclusive teaching practices. In general staff demonstrated an interest in enhancing skills in this area, with 68.9% of staff expressing interest in learning more about specific inclusive teaching practices.


Project Two: Evaluation of Inclusive Teaching Practices

Project Aim

• To evaluate low-cost, high-yield strategies to support the development of evidence-based practice and lead to recommendations for inclusive teaching practices, which can be implemented to better support all students.

The inclusive teaching literature is prone to claims that inclusive teaching strategies will benefit all students, often without empirical evidence. When carefully evaluated, some of the popular strategies do not appear to produce strong effect sizes. For example, meta-analyses suggest that e-learning initiatives, while popular, tend to have below-average effects on student achievement (Hattie, 1992; Schmid et al., 2009; US Education Department 2009). In contrast, relatively simple and low-cost approaches, such as teaching students to make visual representations of information (Marzano, 1998), and note-taking skill development (Marzano, 1998) appear to be “high yield”, that is, to produce an impact well above the average impact of educational interventions. This second project sought to evaluate their efficacy in higher education courses at RMIT. To address this aim, the project identified three low-cost, high-yield inclusive teaching strategies:

• note-taking and note-making. Note-taking refers to the process of take notes when reading material or attending class. The aim is to take down the key information so it can be used later. In contast note-making takes place when the student returns to these notes to annotate, paraphrase, summarise, or transform the notes into a diagram.

• concept mapping. A concept map is a visual representation of relationships between concepts. Concept maps can be used to represent the relationship between elements within a topic, relationships between topics in a course, or even courses in a program.

• argument mapping. An argument map is a visual representation of the structure of an argument. Compared to concept mapping, argument mapping focuses on the logical, evidence-based and inferential relationships between propositions.

These three strategies were identified as accessible to most students, likely to produce an impact, supporting different learning styles and not requiring great expense or changes in practice to introduce. The impact of these strategies was evaluated in a range of higher education courses using a mixed-method design, incorporating a quasi-experimental approach complemented by focus group data.

Participants

Higher education courses taught on the RMIT Melbourne and Vietnam campuses in 2011 were targeted. Table 1 shows the courses selected for the project and the strategies identified for implementation in each course. At the start of the project 10 large courses were identified with sufficient numbers of low SES students and course coordinators in these courses were recruited. Despite commitment from course coordinators, the number of student volunteers in three courses was so low that it was not possible to conduct the trials. In another two cases, the study design, as it evolved, did not meet the needs of the course coordinators involved. The final study design therefore included fewer trials than was originally planned. In total 3 courses from SEH, 2 courses from Business and 1 course from DSC were included. The original plan to test three inclusive practice strategies was retained. The 6 courses in the final project are highlighted in Table 1.


Table 1. Inclusive teaching strategies and courses in the project Course Course

code College Campus Sem Students

enrolled Strategy

Biomedical and Physical Sciences 1

BIOL2062 SEH Bundoora 1 180 Concept mapping

Principles of Psychology BESC1123 SEH City 2 370 Note-taking and note-making

Cognitive Psychology BESC1178 SEH City 2 150 Concept mapping Human Physiology 2: Body Systems

BIOL2044 SEH Bundoora 2 250 Argument mapping

Business Information Technology

INTE2043 Business City 2 31 Note-taking and note-making

Introduction to Organisational Behaviour

BUSM2301 Business Vietnam 1 NA Concept mapping

Contemporary Australian Politics POLI1025 DSC City 1 540 Argument mapping Designing Research HUSO2166 DSC City 1 149 Concept mapping Numeracy 1: Introduction to School Mathematics and Numeracy

TCHE2117 DSC Bundoora 2 143 Note-taking and note-making

Issues and Contexts in Education

TCHE2326 DSC Bundoora 2 29 Argument mapping

Table 2 shows the SES statistics provided by Student Services for each of the 6 courses in the project. Most notable across the 6 courses and 3 campuses is the similarity in percentage of low SES students, varying only from 15% to 18%1. Table 2. SES characteristics of each course in the project SES group (%)

Course Low Medium High

Biomedical and Physical Sciences 1 16 55 29 Principles of Psychology 17 46 36 Cognitive Psychology 15 39 46 Numeracy 1 18 55 27 Business Information Technology 18 46 36 Introduction to Organisational Behaviour (Vietnam) 15 39 46 A total of 166 (121 females, 45 males) students volunteered for the project. However, only 106 (73 females and 33 males) completed both the pre and post-test surveys, providing complete data sets, indicating a retention rate of 64%. Of these 106 participants, 38 volunteered for the treatment groups and were trained in one of the three strategies. The remaining 68 students completed the surveys only and were allocated to the control group. Participant numbers in each trial are summarised in Table 3. A total of 16 students attended the end of semester focus groups, 12 from the note-taking trials, 1 from concept mapping and 3 from argument mapping.

1 It is noted that SES data for Vietnam may not be reliable.


Table 3 Participant Demographics at Pre- and Post-Test

Course Strategy Pre-test N (female/male)

Pre-test age Mean (SD)

Post-test N (female/male)

Post-test age Mean (SD)

Training/Control

Principles of Psych

Note-taking and note-making

75 (56/19) 22.40 (0.69) 58 (43/15) 22.46 (0.77) 10/48

Numeracy 1 Note-taking and note-making

13 (13/0) 19.33 (0.44) 8 (8/0) 19.12 (0.29) 8/0

Business Info Tech

Note-taking and note-making

8 (0/8) 20.75 (1.68) 7 (0/7) 19.43 (0.48) 4/3

Cognitive Psych

Concept mapping

15 (11/4) 23.73 (1.38) 12 (8/4) 23.33 (1.66) 5/7

Introduction to Org Behaviour

Argument mapping

19 (15/4) 19.74 (0.47) 9 (6/3) 19.44 (0.17) 5/4

Biomed and Physical Sci 1

Concept mapping

36 (26/10) 22.58 (0.90) 12 (8/4) 21.25 (0.96) 6/6

Measures

The online survey, completed at the start and end of semester, included demographic details and the following two questionnaires.

The Motivated Strategies for Learning Questionnaire (MSLQ; Pintrich, Smith, Garcia, & McKeachie, 1993) is a self-report questionnaire designed to assess the motivation of university students and their use of various learning strategies. The items are scored on a 7-point Likert scale ranging from 1 = “Not at all true of me” to 7 = “Very true of me”; high scores indicate higher level of motivation and well-developed learning strategies. A total of 61 MSLQ items were used in the project, covering 11 sub-scales, 5 motivation sub-scales and 6 learning strategy sub-scales. The five motivation sub-scales measured intrinsic and extrinsic goal orientation, task value, control of learning beliefs, and self-efficacy for learning and performance). The learning strategies examined included rehearsal, elaboration, organisation, and critical thinking, meta-cognitive strategies and effort regulation. The internal reliabilities of sub-scales used in the survey are robust, ranging from .62 (extrinsic goal orientation) to .93 (self-efficacy). The MSLQ has reasonable predictive validity in terms of the actual course performance of students (Pintrich et al., 1993). The Revised Study Process Questionnaire (R-SPQ-2F; Biggs, Kember, & Leung, 2001) is a self-report questionnaire designed to assess learning approaches of students. The items are scored on a 7-point Likert scale ranging from 1 = “Not at all true of me” to 7 = “Very true of me”; high scores indicate higher level of agreement with either deep or surface approaches. Half of the 20 items on the R-SPQ-2F assess deep learning approaches, while the other half represent surface approaches to learning. Both scales are made up of two sub-scales (motive and strategy). According to Biggs et al. (2001), the internal reliability of these sub-scales ranges from .57 (surface strategy) to .72 (surface motive). The pre-test survey included the MSLQ and R-SPQ-2F. In addition to these two measures, the post-test survey for students in the treatment groups included a set of 12 to 18 items, developed specifically to assess experiences of and engagement and satisfaction with the particular learning


strategy and software. These items were scored on a 4-point Likert scale ranging from 1 = “Strongly agree” to 4 = “Strongly disagree”; low scores indicate better experience with the particular technique/software. Students in the control group responded only to the MSLQ and R-SPQ-2F. In addition, we sought permission from all participants to access their grades in the course and notified students that we would be evaluating the CES data for the course. The original intention was to include academic performance as well as CES data in the analysis, but this plan was changed due to the low participation rates in each course.

Procedure

Following approval from the SEH College Human Ethics Advisory Network (CHEAN), undergraduate students in the 6 courses identified above were invited to participate in this study. The announcement was made at the start of one of their lectures and on their course myRMIT Studies site. A plain language statement describing the rights of participants and the nature of the study was given to all students. Participants gave their informed consent by ticking a box at the start of the survey. The lecturers notified students of the details of three training sessions required for each learning strategy. Students who only responded to the survey, but did not attend training made up the control group. Although both note-taking/making and concept mapping were taught to students from different courses, each course had its own training sessions. Argument mapping skills were taught using RationaleTM software, while concept mapping was taught using InspirationTM software. The content of the training is outlined in Appendix C. Students in the concept and argument mapping training had access to the software through selected computers at the university and a take home trial for thirty days. Following the completion of training sessions, participants in both training and control group were invited, via e-mail, to complete the post-training survey. Finally, focus groups were offered within each course for all students who participated in the training to discuss their experiences. Focus group participants were asked to contribute to discussions and to respond to both closed- and open-ended questions including questions on level of engagement, perceptions on acquired skills and knowledge, weaknesses of the approach and suggestions for improvement.

Results

Due to the small sample sizes in each course, quantitative data for each learning technique were analysed together. The focus group recordings were transcribed and examined for emerging themes. To protect participant confidentiality no personally identifiable information is reported. Summarised below are the main significant outcomes. A series of 2 x 2 mixed-factorial ANOVAs was used to assess the group (training vs. control) x phase (pre-intervention vs. post-intervention) interactions on both the sub-scales of the MSLQ and R-SPQ for the note-taking/making and concept mapping trials. Significant outcomes are summarised below. Results of the student evaluation of the strategies are presented in Appendix D and summarised below. Inclusive Teaching Practice Efficacy Note-taking and note-making Summary statistics for each sub-scale of the MSLQ, together with the ANOVA results are presented in Table 4 below. Significant interactions are highlighted, suggesting significant increases on four sub-scales for the note-taking group compared to the control. Rehearsal strategies are simple strategies including repetition and mnemonics. These approaches are best applied to simple tasks and do not help support connections between new and existing information. The act of note-making involves rehearsal of course content. In contrast, elaboration strategies were targeted by the note-taking training. These strategies, including paraphrasing,


summarising and describing the material in your own words and linking topics to build connections both between new material and between new material and prior knowledge. Similarly, organisation was an important facet of the note-taking training. These strategies help learners select and structure information to show connections. Examples of organisation include clustering, outlining, selecting key ideas. The effort required to organise information makes it an active task. The final sub-scale showing improvement after note-taking training was metacognitive self-regulation. The metacognitive processes tested in this sub-scale include planning (goal setting and task analysis which facilitate organisation), monitoring (including self-testing and questioning) and regulating (adjustments to fine-tune cognitive approaches). Table 4 Pre and post-test scores on the MSLQ sub-scales for note-taking Control

Group Experimental Group

Pre-test Post-test Pre-test Post-test F p η2 MSLQ sub-scales

M SD M SD M SD M SD

Intrinsic goal orientation 20.65 3.61 19. 63 3.49 20.71 4.57 20.67 3.71 1.24 .27 .02

Extrinsic goal orientation 22.29 3.97 20.73 5.55 20.25 5.97 18.75 5.59 .003 .96 0

Task value 35.65 4.34 32.59 7.46 34.23 5.86 32.32 5.79 0.99 .51 .007

Control of learning beliefs 22.02 3.76 21.08 4.16 22.23 3.48 22.91 3.28 3.02 .09 .04

Self-efficacy for learning and performance

41.65 8.72 36.13 10.93 44.10 7.68 42.67 6.30 3.45 .07 .05

Rehearsal 18.27 5.09 18.05 5.09 18.20 4.70 20.35 4.25 4.13 .046 .06

Elaboration 31.56 6.15 30.76 7.10 31.05 5.43 33.57 4.27 4.10 .047 .06

Organisation 14.91 3.99 14.27 3.83 14.57 3.85 16.38 2.87 4.91 .03 .07

Critical thinking 21.85 5.89 22.00 5.59 21.55 5.62 22.80 5.14 0.85 .36 .01

Metacognitive self-regulation

54.50

10.46 53.11 10.65 53.16 9.53 60.53 7.98 15.39 .001 .20

Effort regulation 15.71 3.10 16.27 2.99 14.67 2.15 16.00 2.51 0.91 .34 .01

Note-taking/making was associated with significant change on the deep strategy sub-scale of the R-SPQ (Wilks’ Λ = .93, F (1, 61) = 4.35, p = .04, η2 = .07). This sub-scale is characterised by the intention to seek understanding. Deep learning typically involves the learner making connections between what is already known and new knowledge. Deep learning involves organising and structuring information in coherent ways that create meaning for the student. These are all facets targeted by the note-taking and making training. No changes on the deep motive sub-scale (which measures intrinsic interest and commitment to work) were noted. Students’ perceptions of the value of note-taking and making were consistent with the changes noted in MSLQ scores. For example, note-taking was seen to help identify and summarise the important aspects taught in class, help concentration on their reading and helped deconstruct complex relationships and processes. Students rated note-taking as useful and time-saving. Note-making was perceived to be a useful strategy for revising class notes and enhancing memory. Like note-taking, note-making was useful in clarifying and simplifying relationships between ideas. Students agreed that they would have liked more time to practise this skill. Students trained in note-taking and making were asked what they found most useful about the approach. Students described the benefit of the technique in focusing their attention in class. For example, “The main benefit I found with note-taking was that it helped keep my attention on what was going on in the lecture/tutorial. It helped stop my attention from drifting elsewhere”. These students also noted the value of note-making to help conceptualise issues and store information, commenting that the approach helped, “solidify the information in my memory. Enhanced the effects of note-taking”. It is worth noting that two participants commented that these skills had already been taught in VCE.


Concept mapping Compared to the changes discussed for note-taking and making, concept mapping produced few changes in either the MSLQ or R-SPQ. Significant difference between the control and concept map group was noted on only one sub-scale of the MSLQ: task value (Wilks’ Λ = .83, F (1, 19) = 3.89, p = .06, η2 = .17). Task value refers to how important, useful and interesting the student perceives the task to be. High task value should lead to engagement in learning. Tests of simple main effects showed that the significant interaction noted here was the result of task value increasing slightly for the training group and decreasing significantly for the control group (p = .04). No changes on the R-SPQ were noted. Despite the lack of quantitative change, students trained in concept mapping noted the value of the approach.

I found this experience very enlightening. I already had this software on my laptop from when I was in high school, but had never really seen the benefit of using it. I now see how useful it can be. key concepts were able to be linked, rather than presented and learned as a list i was able to use one piece of information to prompt other information which i think will be great when under exam conditions, if i remember the basic map, i can use what i do remember to cue more information and fill in the blanks.

All the students strongly agreed that the Inspiration software was a useful tool for concept mapping. Argument mapping Parametric techniques were not used to analyse the results of argument mapping technique, mainly due to the small number of participants in the control (n = 4) and training (n =5) groups. Instead, confidence intervals (95%) around each participant’s scores on pre- and post-training were calculated using standard error of measurement (SEM) for each sub-scale of the questionnaires, and then compared to infer the statistical significance of change. Empty cells represent those instances where no statistically significant change was detected. Results are presented in Table X. Table X Significant change for control and treatment group participants on the MSLQ and R-SPQ sub-scales for the argument mapping trial

Note. IGO = intrinsic goal orientation, EGO = extrinsic goal orientation, TV = task value, CLB= control of learning beliefs, SE = self-efficacy, RL = rehearsal, EN = elaboration, ON = organisation,

Group Participant IGO EGO TV CLB SE RL EN ON CT MCSRER DM DS SM SS

CONTROL 1 + + + + +

CONTROL 2 - + + + + + + +

CONTROL 3 - - - - - + + + +

CONTROL 4 - - - -

TRAINING 5 + - +

TRAINING 6 - + + - - - +

TRAINING 7 + - + - + + + + +

TRAINING 8 - -

TRAINING 9 - - - -

MSLQ R-SPQ


CT = critical thinking, MCS = meta-cognitive self-regulation, ER = effort regulation, DM = deep motive, DS = deep strategy, SM = surface motive, SS = surface strategy Results suggest few patterns of improvement attributable to the argument mapping training. In fact more positive change appears to be apparent in the control group. Despite the lack of change evidenced in the quantitative data, students reported argument mapping helped them understand and express arguments. The approach was seen to clarify and understand logical relationships between ideas and was seen to be a useful tool for learning, however, the majority of students wanted more time to practise using the approach and more information on how to apply it. Some complexities in the student responses were noted. For example, almost half the participants (4 out of 9) described the software as “not user-friendly”, however, two-thirds also described is as “easy to use”. Further research may be needed to explore usability of the software in more detail. Focus group feedback Participants in these focus groups consisted of 1 undergraduate psychology student who had used concept mapping, 3 business students who had been trained to use the argument mapping software and 12 students trained in note-taking and making. Note-taking and note-making Two focus groups were held with students who had been trained in note-taking and note-making. Similar themes emerged in both focus groups.

The training aimed to teach students how to extract the hierarchy from information delivered verbally by listening for auditory cues such as voice emphasis and repetition. This is useful in situations where no visual cues are provided (as in many tutorials). A number commented on how this helped them produce more organised notes. One student commented, “I always write lots of notes and usually it’s just all over the place, I get notes everywhere, and after doing this I found I grouped them a lot better and like the ideas like with the subheadings and headings, so then I could like, when I went back to look at it and to revise I could actually get the main points out of my notes rather than having to filter through everything. So it made my revision a lot quicker by getting the important points.”

A second goal of the training was to promote methods like the Cornell method for use in lectures and other classes because such methods facilitate a five minute preview of material before the lecture, and a 5-15 minute review of notes after the lecture. Such methods can be used whether the electronic notes are provided in advance or not. (Where the lecturer’s notes are provided in electronic form, students can print them out with a wide left or right margin to serve as the note review column). One Business student said of such methods, “I used some of it for the […] test that was up-coming, which helped me quite a bit.” Another goal of the training was to teach students to annotate or summarise their class notes systematically, or, even better, turn them into diagrams such as mind maps, in order to get the benefits of rehearsal, re-organisation, and re-thinking of noted material, enhanced by the extra processing power of the visual system. Diagrammatic methods were most popular with trainees. One student said of diagrammatic note-making, “I found I was retaining a lot more. Like when I went to revise for the test I was like, I can remember the weeks where I did mind maps[.] I can remember the stuff more easily.” Another said, “Like the week before [a test] or a week or two leading up to it, I’d go over each week’s stuff and make small mind maps of them and then from those small mind maps make a bigger mind map of them all together and then also with like, if we had like readings and stuff I’d make sure that I did like more highlighting, in like different colours for like different kind of topics or different things I was looking for.” A number of participants felt the note-taking and note-making strategies had helped in revising for tests and had a positive impact on their marks.


It is worth noting that students are using electronic notes in inventive ways that have not been captured by the mainstream literature on note-taking in higher education. For instance, one student reported that he makes his own notes in the notes pane of the lecturer’s Powerpoint slides (provided in advance) while the lecturer is talking. The Cornell method could easily be adapted to allow the notes pane to be used for the review step (for summarising notes with key words and phrases), and this could be incorporated into future training. Concept and argument mapping The focus groups were held with students who had either used a concept mapping program (the psychology student) and another where the students had been introduced to argument mapping software (the business group). The findings are presented separately because concept mapping and argument mapping software differ. However the emerging themes were easily combined suggesting student study interests and concerns are similar. The dominant themes in the focus groups were:

• general approval of the software • most students were familiar with the idea of concept or argument mapping and had

previous experience • essays were the assignment task for which the mapping tools were assumed to be most

useful • comments about future use that included a range of assignment tasks and issues of

availability Findings from the focus groups suggest that these students mainly considered concept and argument mapping as useful for essay writing, all were enthusiastic about having the software made available and there were no strong views as to the relationship between the use of mapping tools and assessment results.

Recommendations

This mixed-methods project, examining the efficacy of three inclusive teaching practices, provided strong support for one of the three inclusive teaching strategies tested. Results indicated the value of note-taking and note-making as a cost-effective, easily taught technique to improve elaboration, organisation, metacognitive self-regulation and deep learning. This technique was trialled in courses in all three Colleges. Unfortunately, due to the small numbers in each course we were unable to test for discipline differences. Although concept and argument mapping were well received by the students, there were few changes to learning strategies or study process as a result of training. Failure to find significant change on either the MSLQ or R-SPQ may be a product of several design factors including the small sample sizes. The following recommendations arise from the results of this project:

• Embed training in note-taking and making into the first year curriculum across the university. Developing students’ skill in this area is likely to also reduce the incidence of “accidental” plagiarism due to poor note-taking.

• While a member of the project team from the SLC ran the note-taking and making training delivered in these pilots, this would not be a sustainable approach. Rather it is recommended that the university support the development of online resources for staff and students in this area. The SLC could work with colleges to develop appropriately contextualised resources.


• Continue to evaluate the usefulness of concept and argument mapping in a range of courses, but embedding the techniques into curricula rather than offering extra-curricular training. Students should be given adequate time to learn and practise these approaches.

• If the university were to continue trials of Inspiration and Rationale, it would be preferable if students could also access the software at home.

References Biggs, J.B., Kember, D., & Leung, D.Y.P. (2001) The Revised Two Factor Study Process Questionnaire:

R-SPQ-2F. British Journal of Educational Psychology. 71, 133-149. Grace, S., & Gravestock, P. (2009). Inclusion and diversity: meeting the needs of all students.

Routledge, New York. Hattie, J 2009, Visible learning: a synthesis of over 800 meta-analyses relating to achievement, Routledge, New York. Marzano, R 1998, A theory-based meta-analysis of research on instruction, Department of

Education. Viewed 14 December 2009, <http://graddiped2007.files.wordpress.com/2007/06/marzarno-instructionmeta_analysis.pdf>

Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1991). A manual for the use of the

Motivated Strategies for Learning Questionnaire (MSLQ). Ann Arbor, MI: University of Michigan.

Dissemination strategies and outputs Outcomes of the study have been written up for broad dissemination within the education community. The proposal stated two publications would result from the grant, one focusing on the quantitative data on student achievement and satisfaction produced by the trials; the other focusing on the qualitative and action research data generated by focus groups, interviews, and the process of conducting the trials. The second of these articles has already been submitted for publication and the first is in preparation. The quantitative paper will focus on note-taking and concept mapping. The smaller sample from the argument mapping trial will be written up as a case study for a third paper. In addition, dissemination within RMIT is planned. As part of this project the SLC made connections with staff in a range of disciplines interested in developing the use of these strategies and this process will continue in 2012. We look forward to sharing the outcomes of the projects both online and via face-to-face seminars throughout 2012.

Evaluation of project outcomes In evaluating the project we used a series of key evaluation questions from the Office for Learning and Teaching.

• What were the observable short-term outcomes?

The outcomes of the project were: (i) a quantitative review of the understanding of inclusive teaching attitudes and behaviours amongst academic staff ; (ii) a series of recommendations for improving staff knowledge of inclusive teaching principles and practices; (ii) quantitative and qualitative data to support the wide-spread implementation of note-taking and making in the first year curriculum.


• Were there any variations from the processes that were initially proposed, and if so, why?

The project was characterised by several substantial challenges that lead to revisions of the original proposal. The first was the issue of identification of students characterised by low SES. Early in the project the DVC Students recommended that we not identify individual students as being low SES. This left us with the dilemma of how to determine effectiveness of the interventions for these equity students. The approach we were advised to adopt was to identify the SES characteristics of the courses included in the trials and make general claims about the efficacy assuming that low SES students participated in the trials. We took this approach, but note that this design means it is not possible to draw specific conclusions about the efficacy of these approaches for low SES students. In addition, we originally proposed a randomised controlled trial (RCT) in which participants would volunteer for the study and then be randomly assigned to either the training or control group. Given the very low numbers of students who indicated their interest this was not possible and we modified our design to a quasi-experimental design, with students self-selecting to be in the training or control group. In some cases after recruiting for the training we had to return to the class to recruit for the control. Compared to an RCT, this design has some inherent disadvantages, in particular the possibility that the two groups differ in ways that may impact on the results of the study. Reassuringly comparisons of pre-test data for the training and control groups did not reveal any significant differences. As noted about a further challenge was the considerable and unpredicted difficulty in recruiting participants for the intervention groups. In our largest cohort of 400 students (BESC1123), only 13 students volunteered and this pattern was repeated in other courses. We had originally hoped to test 6 different inclusive teaching strategies, however, given the difficulties we faced during recruitment, this was consolidated to three strategies, with the following strategies untested in the current project: reciprocal teaching, metacognitive control, and enhanced formative feedback. As a result, part of the budget, unspent on these trials, was reallocated to incentives for student participation. The small sample sizes also had implications for our analyses. Our original plans, to compare academic performance and CES scores, had to be revised. The low response rate was not something we had encountered before, despite our experience as educational researchers and we have reflected at length on the reasons for it and how we might have been able to design the project differently. These issues are discussed in more detail below.

• What factors helped and hindered in the achievement of the outcomes?

The project employed a project manager, Tom Coverdale, whose permanent position is in the SLC. Employing a project manager who was familiar with RMIT and had existing relationships with staff was an advantage for our project. The transfer of money for this position, however, took several months to organise. Similarly the payment of the staff member in Vietnam took considerable time to organise. Perhaps training for project leaders, to ensure better knowledge of RMIT systems, could streamline this process. Added complications arose because the School did not have direct access to the budget for this account, making the job of keeping track of expenditure more time consuming than normal.

• What lessons have been learned from this project and how might these be of assistance to others?

The difficulty recruiting students to the second project was unanticipated and raised questions about the viability of some educational research. As a result we have searched for reasons for the low response rate and alternatives that might avoid these problems in the future.


The particularly low response rate might be explained in two ways. First, students may simply not be interested in engaging in additional work to enhance their performance. While this might describe some students, anecdotal evidence suggests that other students make considerable efforts to improve their learning and grades. A more likely explanation is that the training sessions provided for each intervention may not have been run at a convenient time. Should we have implemented the training within the curriculum to all students and compared outcomes with the previous year? To some extent this approach makes sense. Inclusive teaching focuses on changes embedded in the curriculum, so that they are not perceived to be add on, do not require extra time or commitment and are readily available to all students. It is our expectation that the note-taking approach will now be implemented by courses in exactly this way as part of a universal design. Our attempt to design a more rigorous evaluation of the strategies comparing pre and post-test change with a control group ultimately compromised the power of the study and limited our capacity to draw conclusions.

Budget report • What was the amount of funds approved? $55,000 • What was the final amount of funds acquitted? $48,659

Please attach a financial statement.

A financial statement, signed by the team leader and SHS Finance Manager of acquittal of funds is attached in Appendix E. Below is the expenditure itemised against the original budget. As indicated in the report some savings were made in salaries due the project being implemented in 6 rather than 10 courses and funds were redirected to incentives to encourage student participation.

Name of team member /

Service area (e.g. EduTAG,

Library) / Type of material

required

Specify salary level *

/ material cost etc

Direct

Cost

Oncosts

(15.88%)

Total Expenditure

Andrea Chester replacement costs – 10 hours

Casual lecturer @$146.22/hour

$1462

$233

$1,695

$1,695

Peter Wilson replacement costs – 10 hours


$1462

$233

$1,695

$1,083

Berenice Nyland replacement costs – 10 hours


$1462

$233

$1,695

$1,695

Pilot study staff x 8 salary replacement costs - 8 hours


$9358 $1486

$10,844

$5,252

Martin Kelly – 14 hours HEW7@$43.81

$613

$98

$711

-

Alison Brown – 14 hours HEW7@$43.81

$613

$98

$711

-

EduTAG – 35 hours @ $48/hour $1,680

- $1,680

$1,680

Research Assistant – 350 hours

RA1-3 @$34.17/hour

$11,960

$1,899

$13,859

$13,541

Project manager – 350 hours

[email protected]

$15,333

$2,435

$17,768

$16,883


Catering for focus groups

$200 - $200 $38

Incentives for student participation in focus groups

$200 - $200 $1,713

Transcription of focus groups

$600 - $600 $834

Argument mapping software (Rationale)

$2400 (60 licences)

- $2400 $4,245

Printing costs $500 - $500 - TOTAL

$54,558

$48,659


Appendix A: Inclusive Teaching Discussion Paper

Inclusive teaching discussion paper

Tom Coverdale

Study and Learning Centre, RMIT University

Disclaimer: This discussion paper was prepared to get discussion started within RMIT’s newly-established Inclusive Teaching Working Group. The paper’s suggestions should be regarded as tentative, since they are based on only a preliminary overview of the field, not an in-depth study. While this paper favours one approach to inclusive teaching, there are many other approaches. Feedback should be directed to the Inclusive Teaching Working Group, not the author.


Q. What is inclusive teaching and which inclusive teaching strategies would benefit the rest of the class? Inclusive teachers make numerous adjustments to prevent individuals or particular groups of students from being marginalised, and many of these strategies might prove beneficial for teaching students in general. By the same token, many might not. The wider suitability of particular inclusive teaching strategies can be tested using the research that has already been conducted on successful educational interventions. The literature on inclusive teaching has generated a lot of empirical data supporting the use of particular adjustments with particular groups, such as students with disabilities or lower ESL proficiency, but relatively little empirical data to support using the same strategies with all students. However, it is often the case that the same or similar strategies have been used elsewhere by educators, so consulting the broader educational literature (beyond the inclusive teaching literature) provides a way of working out which of these strategies are ‘just good teaching’. For the purposes of discussion, the various principles put forward by the literature on inclusive teaching are boiled down to six:

1. student-centred philosophy 2. multiple means of representation, expression and engagement 3. simple and transparent tasks and materials 4. inclusive spaces 5. a community of learners 6. inclusive climate.

This discussion paper aims to describe what inclusive teaching is, to introduce a quantitative method used in the broader educational literature to measure the success of educational interventions, and to find strategies which are supported by empirical data for use not only with particular marginalised groups but with all students. Ultimately, it argues for a form of universal design built around strategies which have been proven for general use. From deficit to universal design The idea of inclusive teaching has been shaped by political and legal forces (UNESCO 1994; Topping and Maloney 2005; Wright et al 2006; Grace & Gravestock 2009), and by research that replaces old dichotomies (able/disabled, native/non-native speaker, etc) with a continuum of learning abilities and preferences. To teach inclusively, in a broad sense, is to teach to visible and invisible differences within the learner group, including but not limited to:

• religious practices • cultural and linguistic diversity • ESL • indigenous culture • health conditions • medical conditions • disabilities • age, gender, sexuality • lower socio-economic status • young people who left school early


• older learners returning to study • single parents returning to study • part time students • refugees • survivors of trauma (Training Design 2007)

(In the inclusive teaching literature, most has been written about how to include students with disabilities, students with low ESL proficiency, those from different cultures, girls, and those of lower socio-economic status.)

In an even broader sense, ‘inclusive teaching’ can be interpreted not just as a way of making ‘adjustments’ for students from minority groups, of assimilating them into an unchanged system, but as an attempt to teach in a way that embraces as many forms of difference as possible across the student population, including different learning preferences (Grace & Gravestock 2009). For example, some strategies used to include students with disabilities might also benefit all other students in the class. So the philosophical shift here is from teaching to include marginalised students to trying to embracing all forms of difference. In tertiary education, the idea of making special adjustments for students with different learning requirements, such as those with disabilities, has become politically unpalatable, since it perpetuates a ‘deficit’ model of difference. For example, students with disabilities at most tertiary institutions are required to identify themselves with a medical label if they expect additional support. Support takes many forms, for example, students who are physically unable to write or type quickly, or those with dyslexia, might be permitted a scribe and extra time on the exam. Or students with dyslexia might be taught how to use software which converts text to speech, bypassing the difficulty they have with processing words on the page, or to make mind maps and concept maps, which gives them another non-textual way of processing and structuring information. Some of the above learning strategies might benefit the rest of the class as well, but at present they are usually only offered as adjustments to those with declared disabilities.

The increasing number of students registering a disability (Waterfield et al 2006) and the growing number of students from other marginalised groups is making the ‘adjustments’ approach unwieldy and expensive. Consequently, the idea of ‘universal design for learning’ and other approaches that include pre-emptively have gained traction in the inclusive teaching literature. Universal design and allied concepts such as the ‘embedding’ of learning skills are best employed as part of a whole-of-institution approach which attempts to make special adjustments redundant where possible (Scott, McGuire and Foley 2003; Kift 2005; Higbee and Goff 2009; Kift 2009).

An institution based on universal design attempts to create a climate, a community and teaching spaces which promote diversity and collaboration (Scott, McGuire and Foley 2003; Higbee and Goff 2009) and a curriculum which offers all students ‘multiple means of representation, expression and engagement’ (National Centre on Universal Design for Learning 2009). This kind of curriculum is facilitated partly by educational technology (particularly useful for students with disabilities), but largely by ‘embedding’ academic skills into the curriculum in a way that is integrated with the content (Skillen et al 1998; Kift 2005). To take a different minority group as an example, offering international students expensive ‘bolted-on’ English language support is often less useful in resolving their difficulties in a new academic culture than teaching academic skills and literacies using their actual course materials, and in class time. Teaching such skills as writing structure, paragraphing, synthesis, paraphrasing, referencing, hypothesis testing, argumentation and critical reading in an embedded way is not as expensive as extra-curricular workshops, it is not based on a deficit model, since


the whole class benefits, and it is more sustainable, since changes made to the curriculum will benefit future intakes of students. Thus, an inclusive teaching strategy (the ‘embedding’ of academic skills) at once targets lower-proficiency ESL learners and benefits the rest of the class. Which inclusive teaching strategies are ‘just good teaching’? The inclusive teaching literature is prone to claiming that inclusive teaching strategies are ‘just good teaching’, that they will be of benefit to all students, but often without producing empirical evidence to support these claims. In fact, teachers employ a great many diverse and sometimes conflicting strategies for teaching to various marginalised groups. When these strategies are tested against the broader educational literature, it is evident that some are not suitable for general use and these should be reserved for students who really need them. Ultimately, inclusive teaching, when applied in its broadest sense of embracing all students, should employ strategies with empirical support.

So, how do we tell which strategies are suitable for general use? When asking this question of the broad (empirical) educational literature, meta-analyses are a good place to start. Meta-analysis is a quantitative method widely used in the physical and social sciences to remove the bias inherent in the research design of any single study by abstracting from a multitude of studies the educational ‘effects’ being tested. This provides a rough way of ‘comparing apples and apples’. The broad categories used in most meta-analyses inevitably compare apples and oranges to some extent, but they are useful if treated like a geological survey map, as giving an indication of where to dig. To illustrate, Table 1 ranks a number of educational interventions which crop up in the educational literature. An effect size above 0.4 represents a significant positive impact on students’ achievement (typically measured by grades or standard test scores)2.

2 Hattie (1992) explains that:

An effect-size of 1.0 [is] typically associated with advancing children's achievement by one year, improving the rate of

learning by 50%, or a correlation between some variable (e.g. amount of homework) and achievement of approximately

.50. When implementing a new program, an effect-size of 1.0 would mean that approximately 95% of outcomes positively

enhance achievement, or average students receiving that treatment would exceed 84% of students not receiving that

treatment (p 5-6).

Note that in education, the effect sizes represent the impacts of the various educational ‘interventions’ (experiments conducted using

actual classes in a school or university) on which the studies were based. For example a class might be divided into an experimental

and a control group, and their relative achievement measured at the end of the year.


Intervention Effect size

Note-taking (teaching students how to take notes in class and from reading) Teaching critical thinking to uni. students through argument mapping (turning arguments into diagrams) Study skills programs embedded in tertiary courses Effect of reciprocal teaching on comprehension (students take roles within small group: teacher, questioner, commentator) Implementations that emphasise feedback Metacognitive strategies Teaching study skills Cooperative v individualistic learning Mastery learning (student can’t proceed to subsequent skill until current skill is mastered, e.g. competency-based assessment) Concept mapping Socioeconomic status Peer tutoring (tuition by fellow student) Computer based technology providing cognitive support (e.g. simulations) in higher education Average effect of educational interventions

0.99 (Marzano 1998; n=36) 0.78 (Van Gelder et al 2003, n=7) 0.77 (Hattie, Biggs & Purdie 1996; n=108)

0.74 (Hattie 2009; n= 38) 0.73 (Hattie 2009; n=1287) 0.69 (Hattie 2009; n=63) 0.59 (Hattie 2009 ; n=668) 0.59 (Hattie 2009 ; n=774) 0.58 (Hattie 2009 ; n=377) 0.57 (Hattie 2009; n=287) 0.57 (Hattie 2009; n=499) 0.55 (Hattie 2009; n=767) 0.40 (Schmid et al 2009 ; k=112) 0.40 (Hattie 2009; n=52637)

Computer-based instruction in higher education Computer-based instruction in higher education Computer-assisted instruction in science education Online/blended learning (teaching wholly or partly via computer or other ICT) Web-based learning Individualisation (agreeing with every student in the class on a personalised learning plan) Reducing class size Web-based learning Problem-based learning Gender (in favour of males) Television (impact of)

0.38 (Hattie 2009; k=745) 0.28 (Schmid et al 2009; n=231) 0.27 (Bayraktar 2002; n=42) 0.24 (US Education Dept 2009; n=99) 0.24 (Olson and Wisher 2002; n=15) 0.23 (Hattie 2009; n=600) 0.21 (Hattie 2009 ; n=96) 0.18 (Hattie 2009; n=45) 0.15 (Hattie 2009; n= 0.12 (Hattie 2009; n=2926) -0.18 (Hattie 2009 ; n=37)

Table 1 (adapted from Hattie 2009): effect sizes for different types of educational

intervention, where n=number of studies analysed and k= number of effects analysed

(where k is provided, but not n)


Hattie rightly argues that, given scarce educational resources, it is wise to first explore types of intervention which have an effect on achievement that is higher than the average effect (0.4). It is important to note that this average, like most of the effect sizes in the above table, is not based entirely on tertiary education studies, since meta-analyses specific to tertiary education do not exist for most of these interventions3. But such a ‘league table’ gives an indication of which strategies are worth trialling or doing more of with tertiary cohorts. The grey section of Table 1 shows some of the interventions which have had a below-average effect on student learning. This indicates that the (sometimes considerable) expenditure of resources on some of these interventions is not warranted, or that they might need to be implemented in different ways.

It is evident that many of the most successful strategies for teaching students in general are highly compatible with those suggested by the inclusive teaching literature, while others are not. For example, formative feedback, graphic representations of knowledge (mind maps and concept maps), reciprocal teaching and the teaching of meta-cognition have been shown to be of great benefit to students with many kinds of disability and students with lower ESL proficiency.

By way of contrast, individualised learning programs (‘individualisation’) are recommended for general use by some inclusive teaching researchers, because they have been successful when used with special needs students. But there is a lack of empirical support for the general use of individualisation. Since the 1970s, the average effect of giving all students in a class an individualised learning program, across six hundred studies (0.21), has been very small. This is not to say that certain individualised learning interventions have not been highly successful. For example, Kulik (1981) points out the successful use of one system of individualisation in college mathematics instruction, and Hattie concedes that the effects of such systems dramatically increase when individualisation is combined with sufficient feedback, which might explain the success of mastery learning (0.58). Hattie suggests that most forms of individualisation fail because teachers don’t have time to give sufficient feedback on personalised study programs. In tertiary education, teachers are increasingly short of time, so if individualisation were pursued in general classes, the extra feedback would generally need to come from some source other than the teacher, such as a peer tutor. It may be that individualised learning programs have the same susceptibility as web-based learning (0.18), namely ‘garbage in, garbage out’.

Conceding that meta-analyses give only a general overview, that certain types of intervention work best for certain types of instruction, and that almost any kind of intervention can work under certain conditions, the following list of inclusive teaching principles ignores those with low average effect sizes, and uses examples which pluck the low-hanging fruit from the quantitative literature on successful teaching strategies. Inclusive teaching principles There is no definitive set of inclusive teaching principles, though a number of researchers have attempted to make umbrella headings to cover the horde of

3 The Hattie (2009) and Marzano (1998) meta-analyses employ data from across the educational

sectors, but the bulk of these data come from primary and secondary schools. The 1996 Hattie,

Biggs and Purdie meta-analysis of learning skills interventions, and the 2003 meta-analysis on

argument mapping (Van Gelder et al) were conducted entirely using post-secondary studies and

the 2009 US Education Department meta-analysis on e-learning used around 90% tertiary studies.


strategies. A set of principles provides a useful checklist for teaching practice, however. The list below is included mainly as a starting point for discussion. It has been compiled using a number of previous schemata, most of them from the literature on ‘universal design for learning’. The principles are not ranked in order of priority, though it might be useful to attempt that. A few larger-scale examples are given to illustrate each principle, and areas of controversy are indicated. 1. Student-centred philosophy Inclusive teaching is often contrasted with the traditional didactic or ‘transmission’ model of university teaching, now widely discredited, in which the professor, an authority on the content of his subject, simply transmits information at a large audience of passive students in a difference-blind manner. The limitations of that particular teacher-centred model has made attractive a more ‘student-centred’ approach to teaching, not just in the inclusive teaching literature, but more generally.

While it is true that students have traditionally been somewhat neglected by the academy, a more student-centred approach should not lead us to disregard the importance of the teacher. Allowing students to construct their own learning, the teacher acting as merely a ‘facilitator’ of learning, as in problem-based learning (0.15), web-based learning (0.18), and inquiry-based teaching (0.31), needs to be balanced with direct teaching, since such methods are by conventional measures less effective than instruction in which the teacher is an ‘activator’ and guide of learning.

Hattie’s 2009 meta-analysis of 800 earlier meta-analyses, comprising around 52 637 studies found that active, guided instruction is generally far more effective, by conventional measures of achievement, than less guided, facilitative instruction (Table 2). Examples of ‘activator’ approaches include direct instruction (0.59), reciprocal teaching4 (0.74) and teaching meta-cognitive strategies (0.67). Note that direct instruction is not the same thing as ‘transmission model’ didacticism, since it uses feedback and guided practice. A combination of guided and less guided activities is generally recommended. For example, problem-based learning should not be used to teach students the necessary basic knowledge, since it is generally a poor way of teaching students facts (Hattie 2009), but it works very well to promote deep learning and a firm grasp of underlying principles if students have already been taught the necessary surface knowledge, for example, through direct instruction.

So the practical implementation of a student-centred approach to teaching is likely to be most effective when it employs guided instruction mixed with ‘active learning’ strategies, which get students talking, doing and reflecting, rather than sitting passively in lectures and tutorials.

4 Reciprocal teaching is a small group approach to reading and analysing a text, in which students practise steps that have first been modelled by the teacher.


Teacher as activator

ES Teacher as facilitator ES

Reciprocal teaching Feedback Teaching students self-verbalisation Meta-cognition strategies Direct instruction Mastery learning Average activator

0.74 0.72 0.67 0.67 0.59 0.57 0.60

Simulations and gaming Inquiry-based teaching Smaller class sizes Individualised instruction Problem-based learning Web-based learning Average facilitator

0.32 0.31 0.21 0.20 0.15 0.09 0.17

Table 2: Effect sizes (ES) for teacher as activator and teacher as facilitator (adapted from

Hattie 2009)

2. Simple and transparent tasks and materials This vitally important principle relates to many areas of teaching, but most interestingly to assessment. It calls for simple and detailed course guides and assessment rubrics that spell out exactly what will be taught and how it will be assessed. Assessment is generally regarded as ‘the Achilles’ heel of quality’ (Knight 2002, p 107) in university teaching (Knight 2002; Rust 2009), and current assessment practices, particularly in universities, have been widely criticised for not assessing ‘genuine learning’ (Boud & Falchikov 2007). At the same time, it is widely acknowledged that, for students, the assessment is the curriculum: students are primarily driven by grades. The literature on ‘embedding’ argues that, given the high stakes, the least we can do is teach the things we are assessing. The argument is that you cannot expect students, particularly those from other cultures, to pick up the norms of western academic culture by osmosis. If you want to ask students to submit a particular form of assignment (e.g. a report), then you need to teach the skills and ‘literacies’ involved (e.g. structure, expected patterns and style of academic discourse in English, paraphrasing, synthesising, integrating references, etc), and not assume that students already know them. That is, it is not enough to name these skills in the assignment rubric, you need to teach them alongside course content in a way that is scaffolded, and test for them using formative assessment tasks before you ask students to demonstrate such skills in a high-stakes assignment.

In this connection, Hattie, Biggs and Purdie’s 1996 meta-analysis shows that if academic skills and literacies programs are embedded into the curriculum and taught in a meta-cognitive way, they have a large effect (0.77) on student achievement. Such interventions are typically collaborative efforts between teaching staff and Academic Language and Learning (ALL) staff.

The principle of ‘simple and transparent’ can be applied to most areas of teaching. For instance, to include students with lower English proficiency, jargon, long words and colloquial language can always be explained or simplified. This tends to help other students in the class besides ESL students.


3. Multiple means of representation, expression and engagement This is an important principle for day-to-day teaching and curriculum renewal, but it should be applied with care. The National Centre on Universal Design for Learning’s oft-cited principles of Universal Design for Learning (2009) call for:

1. Multiple means of representation, to give learners various ways of acquiring information and knowledge.

2. Multiple means of action and expression, to provide learners alternatives for demonstrating what they know.

3. Multiple means of engagement, to tap into learners' interests, offer appropriate challenges, and increase motivation.

Multiple means of representation might involve such strategies as using (and teaching students to make) visual representations of knowledge, which has been shown to have large positive effects on student achievement. It might also mean offering course materials in electronic form, so that students can use adaptive technology to access those materials in their preferred ways. Assessment Most controversially, perhaps, the principle of ‘multiple means’ (specifically, ‘multiple means of expression’) calls for a range of assessment alternatives for every assessment task. Many teachers fear that this would mean a lot of extra work (Waterfield et al 2006). Universal design advocates contend that most of this extra work and cost comes at curriculum renewal stage, and that this cost and workload diminishes in subsequent years. They argue, moreover, that the cost of routinely offering a choice of assessment methods to all students must be weighed against the rising cost of providing individual ‘adjustments’ to the growing number of students with special needs.

For example, in Australia, universities well know that they are required by law to make ‘reasonable adjustments’ for people with disabilities, according to the Disability Discrimination Act (1992) and the Disability Standards for Education (2005). In this connection, the number of tertiary students who identify themselves as having a disability has reached eight percent in the UK (Department for Innovation, Universities and Skills 2009, p 11). This rate is rising by roughly one per cent a year, and Australia seems to be following the same trend, largely due to an increase in diagnoses of dyslexia and mental illnesses. All of these students have legal grounds for demanding ‘adjustments’ and/or a more inclusive curriculum. Less well known is that Australian universities will soon be audited for compliance with a new set of standards requiring them to develop the ESL proficiency of international students throughout their degrees, and assessment will likely come under scrutiny from this angle, as well.

Given the lack of success which characterises most attempts at individualisation, it might be wise to offer students a narrow, rather than a wide choice of assessment options. An example would be offering all students the choice of doing a major project or taking an exam. Learning ‘styles’ The principle of offering ‘multiple means’ is frequently recommended as a way of addressing different ‘learning styles’ within the group, but the use of ‘learning styles’ questionnaires in designing instruction is not supported by a strong


evidence base, and there are better reasons to offer ‘multiple means’. A large analysis of the learning styles literature by Coffield et al (2004), could not find any empirical grounds for choosing one learning styles model over the hundred-odd others. They characterise the research on learning styles as confused and confusing. For example, the following list is just a sample of the ways in which teachers might categorise their students:

� convergers versus divergers � verbalisers versus imagers � holists versus serialists � deep versus surface learning � activists versus reflectors � pragmatists versus theorists � adaptors versus innovators � assimilators versus explorers � field dependent versus field independent � globalists versus analysts � assimilators versus accommodators � imaginative versus analytic learners � non-committers versus plungers � common-sense versus dynamic learners � concrete versus abstract learners � random versus sequential learners � initiators versus reasoners � intuitionists versus analysts � extroverts versus introverts � sensing versus intuition � thinking versus feeling � judging versus perceiving � left brainers versus right brainers � meaning-directed versus undirected � theorists versus humanitarians � activists versus theorists � pragmatists versus reflectors � organisers versus innovators � lefts/analytics/inductives/successive processors � versus rights/globals/deductives/ � simultaneous processors � executive, hierarchic, conservative versus legislative � anarchic, liberal. (Coffield et al 2004, p 136)

While finding no model or inventory clearly superior, Coffield et al do recommend that some learning styles models be ‘discontinued for educational use’. One of these was Honey and Mumford’s ‘Theorist Activist Pragmatist Reflector’ model, which is still taught on Australia’s Certificate IV in Training and Assessment.

A more recent review by Pashler et al (2008) went further, finding that there is less empirical support for the argument that students learn best when taught according to their diagnosed learning style than there is for the view that students learn better against their learning preferences. They also point out that humans are, overwhelmingly, visual learners, and other preferences are far less significant.

The questions of whether there are learning styles, and whether they have any educational significance are vigorously contended, but offering multiple means is good practice for other reasons. One reason is that offering multiple means is a more equitable way to teach, since it engages students who might otherwise be marginalised. Secondly, there is mounting evidence that it helps most students to encode information in more than one modality (e.g. visual, verbal, textual), since


forcing the brain to make multiple associations with the same material gives that information more neural scaffolding (Marzano 1998; Carey 2010; Davies 2011). That is, it benefits most students to manipulate information using more than one sense. Thirdly, the large effect sizes for meta-cognition and study skills support for providing students with a variety of tools and teaching them the self-awareness to use these tools in some learning situations but not in others. e-learning Much of the research on providing multiple means has focused on computer-mediated means of representation, expression and engagement. In general, computers have made tertiary education far more accessible to students with disabilities, by circumventing many physical and sensory barriers, but computers are not an educational panacaea.

Assistive technology which is useful to students with particular disabilities, such as screen-reading, speech recognition and diagrammatic mapping software can also benefit students without diagnosed disabilities. And it is generally beneficial for students to be able to access information in different electronic formats, such as audio, Powerpoint (with or without voiceover) and video.

But computer-mediated instruction has generally had a below-average effect on student achievement. This is illustrated by the big-picture data on e-learning. In a 2002 meta-analysis, Bayraktar recorded a small effect size of 0.273 for the use of e-learning in college science education, and concluded that e-learning in science ‘may not be highly effective at this point’. Seven years later, Schmid et al (2009) found a figure in almost precise agreement with Bayraktar’s for the use of technology in higher education (0.28), indicating that little progress had been made. Interestingly, they found that students on campuses with a moderate level of technological saturation tend to achieve better than those from campuses with a low or high level of technological saturation. More promising, and only slightly below average, is Hattie’s 2009 effect size of 0.38 for computer-based instruction in higher education, which seems to have been boosted by a few large effect sizes, including the use of computers in group work.

Less generally, Schmid et al recorded low effect sizes for the ‘presentational’ uses of technology (0.10), and roughly average effect sizes for the uses of technology providing ‘cognitve support’ (0.41), giving the example of a computer simulation as an application which provides cognitive support. Specific applications which depend on proven educational strategies tend to be more successful. For example, argument mapping and concept mapping are successful teaching strategies, and the use of computer software designed for these purposes tends to have large positive impacts in trials. For example, the seven trials of computer-aided argument mapping software yielded a large effect size of 0.78. Conversely, applications which lack some essential educational ingredient generally don’t succeed. For example, web-based learning seems to be a lagging area: Olson and Wisher measured an effect of 0.24 across 15 studies in 2002, and there was no improvement by 2009 when Hattie recorded an effect of 0.18 across 45 studies. Hattie (2009) suggests this is because web-based learning tends not to provide enough of that crucial ingredient, feedback.

The above findings lend support to the oft-repeated observation that, in e-learning, technology has preceded pedagogy. Advocates of educational technology tend to agree that it is the instructional design, the pedagogy, that makes the use of technology in education succeed or fail (Schmid et al 2009).


The United States’ National Centre on Universal Design for Learning, which emerged out of the movement to include students with disabilities, has produced exhaustive lists of strategies that provide multiple means of representation, expression and engagement, for all kinds of students (see http://www.udlcenter.org/aboutudl/udlguidelines). 4. Inclusive spaces There are two aspects to this principle. The first is that classrooms themselves should be fit for inclusive teaching, and the second is that there should not be too many people in the room.

On the plus side, recent decades have seen ergonomic provisions such as access ramps and lower lab benches made for students with disabilities, as mandated by law. Moreover, adjustable furniture has become more widely used, preventing some of the injuries associated with one-size-fits-all furniture. Attention has been paid to lighting, sometimes to air quality, and to ensuring that there are still classrooms with flat floors and moveable furniture as well as lecture theatres.

On the minus side, the trend is to keep squeezing more people into classrooms. The traditional ‘transmission’ model of university teaching has been transformed by commercial imperatives into a ‘stack em deep and teach em cheap’ approach, in which hundreds of students are crammed into lecture theatres and dozens into tutorials, which is not conducive to the small group-based, collaborative practices of inclusive teaching (Cornell 2002; DeGuire North 2002; Graetz and Goliber 2002; Van Note Chism 2002).

The debate over the impact of class size on achievement has raged for decades and the meta-analytic data is inconclusive. Hattie (2009) found an effect size of only 0.21 across 96 studies for reducing class size. While this suggests that increasing class sizes is not a good idea, it provides little incentive to reduce them. At the same time, this number doesn’t really tell us much, since it is an average taken across all educational levels, and thus compares lectures to 80 or more students with classes of fifteen doing group work. The reality is that different-sized classes get different kinds of teaching, and the standards for what constitutes an excellent lecture (in the traditional lecture mode) are not the same as the standards for excellence in more student-centred teaching, which relies more on group work and smaller class sizes (Hattie 2009).

Furthermore, students in large classes are more likely to be assessed in ways that, it has been argued, emphasise surface and strategic learning, such as multiple choice tests, instead of ways which emphasise deep learning, such as essays and reflective journals. Thus, increasing class size may have little apparent impact on ‘achievement’, but it might have strong implications for depth of learning.

With respect to higher education in particular, there is a small body of recent research indicating a strong negative relationship between class size and student performance in higher education. Cuseo (2007) provides a narrative synthesis of the recent empirical research on class size in higher education and finds that increasing class size has a deleterious effect on educational outcomes for students in general and first year students in particular, judged by the following measures:

1. Students’ active involvement in the learning process 2. Interaction between staff and students 3. Feedback to students 4. Depth of thinking inside the classroom


5. Breadth and depth of course objectives and assignments 6. Course-related learning outside the classroom 7. Students academic achievement and performance (learning and grades) 8. Course satisfaction

Bedard and Kuhn (2008) and Westerlund (2008) also found a negative relationship between class size and students’ evaluation of the course.

Kokkelenberg et al (2008) argue persuasively that the ‘diseconomies of scale’ associated with poorer student outcomes cannot always be measured financially, but they are measurable, and increasingly well-documented. Their own longitudinal study of class size at one public university in the United States concludes that ‘class size has a negative relationship to grades and that while the value of the class size coefficient differs across different departments and subsets of data, it is negative in all cases’ (p 229).

Accepting the reality of large classes in post-secondary education, many of the principles under ‘multiple means’ can be applied successfully to large groups. And there are many responses teachers of large classes can make to the needs of various marginalised groups or individuals which benefit all students (see Doyle and Robson 2002; Wright et al 2006; Grace and Gravestock 2009, National Centre on Universal Design for Learning 2009). For example, it is quite possible to do group work in a lecture theatre, though few teachers seem to attempt it. 5. A community of learners This principle spans the curricular and extra-curricular aspects of tertiary education, and recognises the importance of the latter for student retention and achievement.

Scott, McGuire and Foley (2003) call for structures that promote contact between students and staff, between students outside class, and for more personalised interactions between staff and students in the classroom. Discussion groups, chat rooms and compulsory meetings between students and teachers are offered as means to achieving a robust ‘community of learners’ (p46). Such relationships influence ‘students’ disposition to learn’, for which Hattie (1992) recorded a large effect size of 0.61.

More democratic, personalised relationships between students and staff in class, and, ideally, out of it, are characteristic of an inclusive classroom, and some of the interactions called for by Scott, McGuire and Foley might flow naturally from the sort of changes to learning spaces and class sizes called for under the principle of ‘inclusive spaces’ (Cornell 2002; DeGuire North 2002; Graetz and Goliber 2002; Van Note Chism 2002).

Another model for building a community of learners is peer tutoring. Peer-support programs are gaining traction in many western universities as a scale-able model for extending academic and social mentoring networks throughout the university. Besides the social benefits, the academic benefits of peer tutoring are considerable (0.55). 6. Inclusive climate The importance of an inclusive climate, the research suggests, is most obvious on campuses that don’t have one. Perhaps this is why much of the campus climate research has been focussed on bridging the racial divide across some US campuses, in order to improve retention rates and grades among marginalised groups.

Scott, McGuire and Foley (2003) call for a ‘climate of respect for diversity’ which can be ‘modeled by the faculty member through awareness and


considerate treatment of all students.’ (p 46). Practical suggestions for doing this include ‘putting a statement in the class syllabus affirming the need for class members to respect diversity’ (p 46), surveying students to monitor their comfort level in class discussions, and incorporating diversity content into the curriculum, for example, by highlighting the work of thinkers from diverse backgrounds.

There is a body of qualitative research suggesting ways to promote a more tolerant attitude towards diversity on campus, which include academic support initiatives to retain students from minority backgrounds, various curricular and co-curricular initiatives, ‘safe space’ initiatives, and community outreach initiatives undertaken by institutions (Hurtado et al 2005). Hurtado et al concede that climate research would be strengthened if the behavioural and psychological data gathered by the welter of different climate surveys (they studied 90) were linked more explicitly to educational outcomes. Conclusion The term ‘inclusive teaching’ covers a broad and at times conflicting range of teaching strategies. The major shift in recent years has been to extend ‘inclusion’ to all students, with all their diverse and shifting strengths and learning preferences. This shift has been caused and accompanied by a growing dissatisfaction with the culture of making ‘adjustments’ for exceptional cases, while leaving the system unchanged. Research that calls for a whole-of-institution approach to inclusion, including that on ‘universal design for learning’, has grown out of this dissatisfaction. The main concern of this paper is to suggest that where inclusive teaching principles are applied to the general student population, it is wise to employ strategies which are supported for general use by evidence from the broader educational literature. This is perfectly compatible with the prevalent system of making special-needs-based adjustments for disabilities; it merely argues that some of these adjustments should be offered to the rest of the class as well, while adjustments that offer little general benefit should be reserved for those who really need them. Of course, the argument and the inclusive teaching principles set out in this paper are not intended to be the last word, but to get discussion started.


Works cited Bayraktar, S 2002, ‘A meta-analysis of the effectiveness of computer-assisted instruction in science education, Journal of research on technology in education, vol 34, no 2, pp 173-188, viewed 10 December 2011, ERIC database. Bedard, K & Kuhn, P 2008, ‘Where class size really matters: class size and student ratings of instructor effectiveness, Economics of education review, vol 27 pp 253-265, viewed 15 November 2010, ERIC database. Boud, D and Falchikov, N 2007 ‘Reframing assessment as if learning were important’, in D Boud and N Falchikov (eds) Rethinking assessment in higher education, Routledge, Abingdon. Carey, B 2010, ‘Forget what you know about good study habits’, New York Times, 6 September, viewed 11 April 2011 < http://www.nytimes.com/2010/09/07/health/views/07mind.html?_r=1>. Coffield, F, Moseley, D, Hall, E & Ecclestone, K 2004, Learning styles and pedagogy in post-16 learning: a systematic and critical review, Learning and Skills Research Centre, <http://www.hull.ac.uk/php/edskas/learning%20styles.pdf>, viewed 30 September 2009. Cornell, P 2002, ‘The impact of changes in teaching and learning on furniture and the learning environment’, in N Van Note Chism and D Bickford (eds) The importance of physical space in creating supportive learning environments, Jossey Bass, San Francisco, pp 33-42. Cuseo, J 2007, ‘The empirical case against large class size: adverse effects on the teaching, learning, and retention of first-year students’, Journal of faculty development, vol. 21 no. 1, pp. 5-21, viewed 28 April 2010, ERIC (CSA) database. Davies, M 2011, 'Concept mapping, mind mapping and argument mapping: what are the differences and do they matter?' Higher education, forthcoming. DeGuire North, J 2002, ‘Put your money where your mouth is: a case study’, in N Van Note Chism and D Bickford (eds) The importance of physical space in creating supportive learning environments, Jossey Bass, San Francisco, pp 73-80. Department for Innovation, Universities and Skills 2009, Disabled Students and Higher Education: Higher Educational Analysis, Department for Innovation, Universities and Skills (UK), viewed 22 September 2009. Doyle, C & Robson K, 2002, Accessible curricula: good practice for all, University of Wales Institute, Cardiff.

Exley, K & Dennick, R 2004, Small group teaching, RoutledgeFalmer, London. Fry, H, Ketteridge, S, & Marshall, S 2003, Teaching and learning in higher education, Kogan Page, London.


Grace, S & Gravestock, P 2009, Inclusion and diversity: meeting the needs of all students, Routledge, New York. Graetz, K and Goliber, M 2002, ‘Designing collaborative learning places: psychological foundations and new frontiers’, in N Van Note Chism and D Bickford (eds) The importance of physical space in creating supportive learning environments, Jossey Bass, San Francisco, pp 13-22. Hattie, J 1992, ‘Measuring the effects of schooling’, Australian Journal of Education, Vol. 36, No. 1, pp 5-13, viewed ERIC (CSA) database 2 October 2009. Hattie, J, Biggs, J & Purdie, N 1996, ‘Effects of learning skills intervention on student learning: a meta-analysis’, International Journal of Educational Research, 11, 187–212. Hattie, J 2009, Visible learning: a synthesis of over 800 meta-analyses relating to achievement, Routledge, New York. Higbee, JL and Goff, E 2008, Pedagogy and student services for institutional transformation: implementing universal design in higher education, Center for Research on Developmental Education and Urban Literacy, accessed 24 March 2011, < http://www.cehd.umn.edu/passit/docs/PASS-IT-Book.pdf>. Hurtado, S, Griffin, K, Arellano, L and Cuellar, M 2008, ‘Assessing the Value of Climate Assessments: Progress and Future Directions’, Journal of Diversity in Higher Education, Vol. 1, No. 4, pp 204–221. Kift, S & Nelson, K 2005, ‘Beyond curriculum reform: embedding the transition experience’, Higher education in a changing world, Sydney, 3-6 July 2005, Higher Education Research and Development Society of Australasia Conference, viewed 7 April 2011, <http://www.herdsa.org.au/wp-content/uploads/conference/2005/papers/kift.pdf>. Kift, S 2009, Keynote address to ninth biennial national conference of the Association for Academic Language and Learning, 26 & 27 November 2009. Knight, PT 2002, The Achilles’ hell of quality: the assessment of student learning, Quality in higher education, Vol 27, no.3, pp 107-115, viewed ERIC (CSA) database 16 February 2010. Kokkelenberg, EC, Dillon, M, Christy, SM, 2008 ‘The effects of class size on student grades at a public university’, Economics of education review, vol 27, p 221-233, viewed 30 March 2011, ERIC database. Kulik, J 1981, ‘Integrating findings from different levels of instruction’. Symposium paper presented at the annual meeting of the American Educational Research Association, Los Angeles, April 1981. Cited in Bangert, R, Kulik, J and Kulik, C 1983, ‘Individualized Systems of Instruction in Secondary Schools’, Review of Educational Research, Vol. 53, No. 2 (Summer, 1983), pp. 143-158, viewed ERIC (CSA) database 14 December 2009.


Marzano, R 1998, A theory-based meta-analysis of research on instruction, Department of Education. Viewed 14 December 2009, <http://graddiped2007.files.wordpress.com/2007/06/marzarno-instructionmeta_analysis.pdf>. Marzano, RJ 2007, The art and science of teaching: a comprehensive framework for effective instruction, ASCD, Alexandria, Virginia. National Centre on Universal Design for Learning <http://www.udlcenter.org/aboutudl/udlguidelines>, viewed 14 December 2009. Olson, T & Wisher, RA 2002, ‘The effectiveness of web-based instruction: an initial inquiry’, The international review of research in open and distance learning, vol 3, no 2, viewed 11 April 2011, < http://www.irrodl.org/index.php/irrodl/article/view/103/182>. Pashler, H, McDaniel, M, Rohrer, D & Bjork, R 2008, Learning styles: concepts and evidence, Psychological science in the public interest, vol 9, no 3, viewed December 2008, <http://www.psychologicalscience.org/journals/pspi/PSPI_9_3.pdf>. Rust, C 2009, ‘Assessment practice: a manifesto for change’, Keynote address to ATN assessment conference, RMIT University, Melbourne, 19-20 November 2009. Schmid, RF, Bernard, RM, Borokhovski, E, Tamim, R, Abrami, PC, Wade, CA, Surkes, MA, Lowerison, G 2009, ‘Technology’s effect on achievement in higher education: a Stage I meta-analysis of classroom applications’, Journal of computing in higher education, vol 21, pp 95-109, viewed 11 November 2009, ERIC database. Scott, S, McGuire, J, and Foley, T 2003, ‘Universal design for instruction: a framework for anticipating and responding to disability and other diverse learning needs in the college classroom’, Equity & excellence in education, vol 36, no 1, pp 40-49,viewed ERIC (CSA) database 14 September 2009. Skillen, J, Merten, M, Trivett, N, & Percy, A, 1998, The IDEALL approach to learning development: a model for fostering improved literacy and learning outcomes for students, viewed ERIC database 14 December 2009. Topping, K & Maloney, S (eds) 2005, The RoutledgeFalmer reader in inclusive education, RoutledgeFalmer, London. Training Design 2007, ‘One and all: a facilitator’s guide to assist in the delivery of TAAENV402A foster and promote an inclusive learning culture’, Wodonga Institute of TAFE. UNESCO 1994, The Salamanca statement and framework for action on special needs education, viewed 20 April 2011, <http://www.unesco.org/education/pdf/SALAMA_E.PDF>.


Van Note Chism, N 2002, ‘A tale of two classrooms’, in N Van Note Chism and D Bickford (eds) The importance of physical space in creating supportive learning environments, Jossey Bass, San Francisco, pp 5-12. Waterfield, J, West, B & Parker, M 2006, ‘Supporting inclusive practice: developing an assessment toolkit’ in M Adams & S Brown (eds) Towards inclusive learning in higher education: developing curricula for disabled students, Routledge, Abingdon. Westerlund, J 2008, ‘Class size and student evaluations in Sweden’, Education economics, vol 16, no 1, pp 19-28, viewed ERIC database, 4 November 2011. Wright, A, Abdi-Jama, S, Colquhoun, S, Speare, J & Partridge, T 2006, F.E. Lecturer’s guide to diversity and inclusion, Continuum, London.

Appendix B: Staff Survey Results Table 1. Internal Reliability Scores for each Sub-Scale of the Staff Survey

Sub-scale Cronbach’s alpha

Beliefs .64

Goals .73

Feedback and assessment .72

Modes of delivery .75

Teaching .80

Table 2. Percentage of Staff Responses - Inclusive Teaching Beliefs (N=93)

Question Strongly agree

Agree Neutral Disagree Strongly disagree

All students can learn, given an appropriate educational environment. 38.7 57 0 4.3 0

All students can be successful in my class. 32.3 41.9 9.7 12.9 3.2

I can ensure that all students experience success by adapting the curriculum. 22.6 38.7 14.0 21.5 3.2


Students with difficulties hold back students who are not as challenged. 0 21.5 19.4 46.2 12.9

When I help some students make links and build on previous knowledge, I am encouraging success in learning.

60.2 36.6 3.2 0 0

It’s important to address the challenges faced by:

• students from lower socio-economic backgrounds 53.8 40.9 3.2 2.2 0

• students with disabilities 53.8 46.2 0 0 0

• students with lower English proficiency 48.4 46.2 3.2 2.2 0

• students from different cultures 48.4 46.2 2.2 3.2 0

A student’s success may be dependent on activating prior knowledge. 33.3 57.0 6.5 3.2 0

Inclusive teaching is about addressing the deficits of individual students (e.g. lack of English proficiency, disability).

19.4 46.2 17.2 14.0 3.2

Making adjustments for individual students (e.g. with physical or learning disabilities) is onerous. 9.7 33.3 19.4 28.0 9.7

I am familiar with the idea of ‘universal design for learning’. 8.6 15.1 19.4 47.3 9.7

I am interested in exploring the use of inclusive teaching strategies (e.g. concept mapping, argument mapping, reciprocal teaching).

28.0 45.2 20.4 3.2 3.2

Inclusive teaching often benefits the whole class. 34.4 40.9 22.6 2.2 0


Table 3. Percentage of Staff Responses - Inclusive Teaching Goals and Expectations (N=93)

In the last year, I… Frequently Often Sometimes Rarely Not at all NA

set learning goals (or ‘outcomes’) that were expressed as statements of what students would know or be able to do at the end of the unit.

60.2 29.0 6.5 1.1 0 3.2

welcomed students from diverse backgrounds, either verbally or in print (e.g. in the course guide). 35.5 20.4 12.9 14.0 12.9 4.3

established positive emotions in the classroom (e.g. through praise and calling on students of various abilities and backgrounds to answer questions, then giving them equal time to answer).

49.5 24.7 16.1 5.4 2.2 2.2

conveyed high expectations to all students in my classes, giving the impression that I believe every student can succeed.

60.2 31.2 7.5 1.1 0 0

Table 4. Percentage of Staff Responses - Inclusive Teaching Delivery (N=93)


provided students with an overview of material to be covered at the start of each class, and at the end of the class, a summary of material covered.

50.5 25.8 16.1 3.2 3.2 1.1

used specific instructional techniques for specific instructional goals (e.g. if the goal was critical thinking, then argument mapping might be used).

25.8 29.0 23.7 10.8 5.4 5.4

engaged students in some kind of previewing activity before the actual presentation of content (e.g. via teacher-prepared questions or notes), to activate prior knowledge.

33.3 28.0 26.9 7.5 4.3 0

delivered new material in small chunks, proceeding from less complex to more complex. 45.2 45.2 7.5 1.1 0 1.1

allowed time for teacher-guided practice of skills to allow students (alone, with other students, or with 34.4 36.6 19.4 5.4 1.1 3.2


teacher) to absorb and get feedback on their practice of particular skills.

taught, and encouraged students to make, visual representations of knowledge, e.g. diagrams, mind-maps, to help them manipulate and structure information.

24.7 22.6 20.4 15.1 16.1 1.1

avoided slang and colloquial language, or explained it, for the benefit of students from different cultures. 39.8 30.1 18.3 6.5 2.2 3.2

incorporated diversity content into the curriculum, e.g. by highlighting the work of female thinkers or intellectuals from historically marginalised cultures.

20.4 12.9 26.9 12.9 8.6 18.3

used cues or ‘signposts’ to link new content and skills to content and skills previously taught. 28.0 43.0 17.2 6.5 5.4 0

Table 5. Percentage of Staff Responses - Inclusive Teaching Modes (N=93)


used collaborative discussions and activities (pair-work, group-work) as well as whole-group and individual activities in my classes.

53.8 20.4 15.1 6.5 1.1 3.2

used a combination of direct teaching (which includes guided practice) and less-guided instruction (e.g. problem-based learning).

48.4 31.2 15.1 2.2 0 3.2

used a multi-modal approach to teaching, which provided students with a mix of inputs and outputs that included visual (e.g. graphs, diagrams), auditory (e.g. group discussions, oral presentations), kinaesthetic (e.g. model building, hands-on practice) and written e.g. (reading, written assignments) modalities.

40.9 37.6 9.7 6.5 4.3 1.1

taught a meta-cognitive approach to learning strategies, i.e. that different strategies suit different learning tasks.

22.6 30.1 24.7 7.5 10.8 4.3

provided students with the opportunity to participate in a peer-mentoring program. 16.1 12.9 22.6 10.8 29.0 8.6

Table 6. Percentage of Staff Responses - Inclusive Teaching Feedback and Assessment (N=93)



took every opportunity to provide students with feedback. 50.5 39.8 9.7 0.0 0.0 0.0

asked students lots of questions, to activate their prior knowledge and to assess their level of understanding. 46.2 44.1 9.7 0.0 0.0 0.0

taught students the skills particular to each kind of assessment. 51.6 34.4 9.7 1.1 0.0 3.2

used formative assessment tasks (i.e. not for marks or for few marks) to build students’ understanding. 38.7 35.5 14.0 5.4 5.4 1.1

gave students detailed assessment criteria, spelling out exactly which learning goals (including academic skills) would be assessed and the weight they would be given. 61.3 26.9 8.6 3.2 0.0 0.0

provided students with rubrics or scales to indicate their level of mastery of each learning goal. 31.2 23.7 19.4 10.8 15.1 0.0

used alternative assessment methods (e.g. peer assessment and self-assessment) in conjunction with assessment by teachers, as a means of increasing the amount of formative feedback each student received. 23.7 21.5 14.0 15.1 24.7 1.1

consulted the literature on inclusive teaching or a learning and teaching professional (e.g., advisor from your ADG or SLC) 14.0 8.6 15.1 23.7 36.6 2.2


Table 7. Percentage of Staff Aware of Students from Equity Groups in their Courses (N=93)

In the last five years I have been aware of students in my class who… Yes No Don’t know

spoke English as a second or other language 94.6 5.4 0.0

belonged to different religions 92.5 1.1 6.5

were returning to study after a long break 91.4 2.2 6.5

were from lower socio-economic backgrounds 83.9 3.2 12.9

had a disability 83.9 10.8 5.4

were single parents 68.8 3.2 28.0

were indigenous Australians 52.7 21.5 25.8

were gay or lesbian 51.6 1.1 47.3

left school early 51.6 5.4 43.0

were refugees 48.4 8.6 43.0

were survivors of trauma 38.7 1.1 60.2

Appendix C: Inclusive Teaching Practice Training Outlines

Note-Taking and Note-Making

Session 1: Listening and note-taking in class

Students introduced to the distinction between note-taking and note-making. Listen to excerpts from a lecture in order to practise listening for auditory cues and recording the hierarchical structure of the information. Student introduced to the Cornell note-taking system and practise using it.

Session 2: Note-making

Students introduced to techniques for making notes in order to summarise and rehearse the information recorded in lectures and classes. They practise using diagrammatic note-making to summarise, rehearse and simplify information recorded in lectures and/or from reading.

Session 3: Note-taking from reading

Students introduced to effective reading techniques to help them extract the main ideas from a text quickly. They practise previewing, skimming and scanning an article. Also introduced to traditional and electronic ways of organising and keeping control of copious notes.

Concept Mapping

Session 1: Introduction to mind-mapping and concept-mapping Students are introduced to the diagrammatic representation methods of mind-mapping and concept-mapping. Students are provided with a demonstration of the diagrammatic mapping software 'Inspiration'. Students make a basic mind-map using the software. As a group students discuss techniques that they used. Session 2: Mind mapping and concept-mapping for exam preparation. Students are introduced to diagrammatic techniques that assist in summarising topic areas covered in lectures. Using concept-mapping students begin a summary of a topic area for the purpose of exam preparation. As a group students discuss techniques that they used. Session 3: Mind mapping and concept-mapping for lab report writing Students are introduced to diagrammatic techniques that assist in mapping out structure for the introduction of a psychology lab report. Students use 'Inspiration' to map out the structure of the introduction to a lab report. As a group students discuss techniques that they used.

Argument Mapping


Session 1: Introduction to argument mapping and rationale

Students are introduced to argument mapping, activities around this to get them thinking along the lines of the way the maps are structured, they complete basic tutorials in Rationale.

Session 2: More advanced mapping

Students covered more advanced mapping using the tutorials in rationale and examples from their Intro to OB article. Students mapped argument from the article and started plotting out the argument from their own essay.

Session 3: Mapping their essay

Students continued mapping the argument from their essay and support. Individual feedback and guidance were provided.

Appendix D: Evaluation of Inclusive Teaching Practices Results Note-taking and Note-making

Table 1. Note-taking (taking notes in class or from reading) evaluation (N = 11)

Frequency of responses

Question Strongly agree Agree Disagree Strongly disagree N/A

Allowed me to make a good summary of what was taught in class. 3 6 1 0 1 Is something I usually do in class. 5 3 2 0 1 Helped me remember what was taught in class. 6 2 1 0 1 Helped me identify the most important points taught in class. 5 5 0 0 1 Is a strategy I needed more time to practice. 1 6 4 0 0 Helped me organise the lecture content into main topics and sub-topics. 6 3 1 0 1 Is a strategy I needed more information on. 0 6 3 2 0 Helped me concentrate on what was being said in class. 4 5 1 0 1 Helped me concentrate on what I was reading. 5 4 1 0 1 Helped me remember the relationships between ideas (e.g. main topics and sub-topics). 5 5 1 0 0 Does not suit my style of writing and thinking. 0 3 5 2 0 Helped me break down complex relationships and processes into simpler, more manageable components.

4 7 0 0 0

Was a useful tool for studying this course. 3 8 0 0 0 Was useful, even though we were provided with PowerPoint slides 4 7 0 0 0 Was useful, even though we were provided with printed notes. 3 7 0 0 0 Saved me time when studying this course. 4 5 0 0 1 Helped me summarise what was taught in class. 4 6 1 0 0 Is time consuming. 1 3 4 2 1

Table 2. Note-making (making something about of the notes I took) evaluation (N = 11)



Answer Options

Strongly Agree Agree Disagree Strongly Disagree N/A

Was useful for revising class notes. 3 8 0 0 0 Was more useful than note-taking. 3 5 3 0 0 Was a useful tool for studying this course. 3 8 0 0 0 Helped me remember what was taught in class. 4 6 0 0 1 Helped me clarify the relationships between ideas. 5 6 0 0 0 Helped me remember the relationships between ideas. 5 6 0 0 0 Helped me organise course content into main topics and sub-topics.

4 5 0 0 1

Helped me understand the logical connections between ideas 5 6 0 0 0 Does not suit my style of writing and thinking. 0 3 4 2 2 Helped me break down complex relationships and processes into simpler, more manageable components.

3 7 0 0 1

Is something I needed more time to practice. 0 7 4 0 0 Is time consuming. 3 3 3 1 0

Table 3 General evaluation (N = 11)


Answer Options

Strongly Agree Agree Disagree Strongly Disagree N/A

Making visual notes (e.g. using ‘mapping’) was a useful strategy. 3 7 1 0 0 Self-testing on notes using the Cornell review column was a useful strategy.

2 6 3 0 0

The training in note-taking and note-making was useful. 3 6 2 0 0

Concept mapping


Table 4. Concept mapping evaluation (N = 4)

Answer Options Strongly Agree Agree Disagree Strongly Disagree N/A

Allowed me to develop a visual representation of the relationships I wanted to represent.

1 3 0 0 0

Helped me understand the relationships between ideas. 1 2 1 0 0 Helped me remember the relationships between ideas. 0 2 1 0 0 Does not suit my style of writing and thinking. 1 0 2 1 0 Helped me break down complex relationships and processes into simpler, more manageable components.

1 1 1 0 1

Helped me understand the logical connections between ideas. 1 3 0 0 0 Helped me show logical connections between ideas. 1 3 0 0 0 Is inflexible. 0 0 3 1 0 Was a useful revision tool. 2 1 0 1 0 Saved me time when studying this course. 1 0 1 1 1 Is time consuming. 0 2 2 0 0 Is a strategy I needed more information about. 1 3 0 0 0

Table 5. Inspiration (concept mapping software) evaluation (N = 4)

Answer Options Strongly Agree Agree Disagree Strongly Disagree N/A

I did not have adequate knowledge of Inspiration software. 0 2 2 0 0 Inspiration software was a useful tool for concept mapping. 3 0 0 0 1 I found Inspiration software easy to use. 2 1 0 0 1 I had enough time to practice using Inspiration software. 2 1 0 0 1 Inspiration software was not user-friendly. 0 0 0 3 1 The training in concept mapping and Inspiration software was useful.

1 1 1 0 1

Argument mapping


Table 6. Argument mapping evaluation (N = 9)

Answer Options Strongly Agree Agree Disagree Strongly Disagree

N/A

Helped me understand how to express arguments (both written and spoken).

4 5 0 0 0

Helped me simplify arguments. 3 4 2 0 0 Helped me clarify the relationships between ideas. 3 6 0 0 0 Does not suit my style of writing and thinking. 0 2 5 2 0 Helped me remember the relationships between ideas. 1 7 1 0 0 Helped me break down complex relationships and processes into simpler, more manageable components.

3 3 2 1 0

Allowed me to develop a visual representation of the relationships I wanted to represent.

2 6 1 0 0

Helped me understand the logical connections between ideas. 3 6 0 0 0 Is inflexible. 0 2 5 2 0 Helped me show logical connections between ideas. 1 7 1 0 0 Was a useful tool for studying this course. 3 5 1 0 0 Saved me time when studying this course. 3 4 2 0 0 Is time consuming. 0 3 4 2 0 I needed more time to practise using argument mapping. 4 2 2 0 0 I needed more information on how to apply argument mapping. 2 4 2 0 0

Table 7. Rationale (argument mapping software) evaluation (N = 9)


Answer Options Strongly Agree Agree Disagree Strongly Disagree

N/A

I did not have adequate knowledge of Rationale software. 1 5 2 1 0 Rationale software was a useful tool for argument mapping. 3 6 0 0 0 Rationale software was not user-friendly. 0 4 5 0 0 I found Rationale argument mapping software easy to use. 1 5 2 0 1 I had enough time to practice using Rationale argument mapping software.

0 6 1 1 1

Documents

Inclusive teaching design to better support all students ...mams.rmit.edu.au/mc0b2wexcrylz.pdf · Inclusive teaching design to better support all students: Toward a best practice