Prof Keithia Wilson Program Lead for the FYE, Griffith Health GIHE Senior Fellow for the FYE 2010 ALTC National Fellow for the FYE 2007 Australian University

Prof Keithia WilsonProgram Lead for the FYE, Griffith Health

GIHE Senior Fellow for the FYE2010 ALTC National Fellow for the FYE

2007 Australian University Teacher of the Year

Prof Keithia Wilson ALTC National Fellow


Acknowledgment to CountryIn the Spirit of ReconciliationFollowing on from Sorry DayI would like to acknowledge & honour the

Traditional Custodians of this land that our Logan campus is built on, the Yugambeh People, and pay respect to their Elders past & present

Overview1. Principles & practices for enabling

commencing students success with FY Assessment

Designing assessment Preparing students to undertake assessment Marking Assessment Post-Assessment feedback process to enable &

support learning 2. Strategies for academic recovery with at-

risk commencing students


Overview1. Current Context for the FYE2. Purposes of FY Assessment & Student

Motivations3. The FY Assessment Lifecycle from the

perspectives of – Students and Staff2. Principles & Models for enhancing

effective Practice with FY Assessment


Information Sources

The First Year Experience & Assessment literature generally

Findings from a recent ALTC Grant on First Year Assessment Practice (2009-2010)

Findings from a current ALTC National Fellowship on the FYE working with FY teaching teams (2010-2012)

Focus on the student voice & understanding the student experience of assessment


Step 1 – Context

How can we improve the assessment process and experience for commencing students?

Understanding the current context for the FYE


Federal Gov’t reform agenda in Higher Education

Aims to –

Widen student participation in Higher Education – A FAIR GO!

Increase the access of students from low SES & disadvantaged backgrounds to university (higher numbers - Target increase from 12% to 20% by 2020)

Increase the success of students from low SES & disadvantaged backgrounds at university (higher retention)

Moving from an elite model of HE (0-15% population participation) to a mass model of HE (16-50%) (Trow, 2004)

25% participation in Australia (50% USA)1.3% of Indigenous Australians attend university (30% lower

graduation rate)


Evolution of Approaches to FY Orientation & Engagement & the FYEFirst Generation Strategies = Co-Curricular - A

focus on designing FYO&E supplemental activities & strategies which are outside of the classroom

Second Generation Strategies = Curricular - A focus on enhancing FY curriculum design, pedagogy & assessment practices

Third Generation Strategies = Whole-of-Institution - A focus on an Institution wide approach to 1st & 2nd generation strategies, with practice standardisation & QA mechanisms for continuous improvement

= Whole-of-School/Program - A focus on the strategic combination of 1st & 2nd generation strategies for a particular disciplinary context (School or Program)


Understanding the Current Context for the FYEMeta Model 1 - FY Transition Practice

Student Diversity Student Transition

* Course Design * Course Delivery * Course Assessment


Student diversity

So what is important to understand about student diversity?

Defining student diversityAssumed knowledge


What is Student Diversity?Traditional

Students (TS)medium-high SESsecond generationhigher entry levels full timeon-campus

Elite Model

Non-Traditional Students (NTS)

low SESfirst-in-family lower entry levels full-time & workingnot on-campus much Indigenous NESB International,

refugeesdisabilityhome care responsibilities from rural & remote

settings

Mass Model


What is assumed knowledge & capabilities?

Academic Skills Information Literacy Computer Literacy Reading Skills Written Communication Numeracy Skills Critical thinking & analysis Independent Learning (self-regulation) viz.

time on task, self-study, time management, uni-work-social life balance, successful student behaviour


What is assumed knowledge & capabilities?Academic/Cultural/Social Capital – the

“Hidden Curriculum”1.Role Understanding student role expectations &

appropriate & effective behaviour (rights & obligations)

2.Success Reading the academic context to accurately determine performance requirements re studying & assessment

3.Support Capacity for help-seeking without fear of negative labelling (dumb/stupid)

4.Personal Identity Sense of belonging & personal fit with university (overcome the “outsider within” phenomenon – “A stranger in a foreign land”)


Griffith University context & Griffith University context & student profilestudent profile

• Student enrolment of 40,000Student enrolment of 40,000• 70% of students are first-in-family at uni (NT) - FIF 70% of students are first-in-family at uni (NT) - FIF

correlates with low SES & lower entry scores/Opscorrelates with low SES & lower entry scores/Ops• 66thth largest low SES student intake in Australia (16% largest low SES student intake in Australia (16%

overall, 30% Logan campus)overall, 30% Logan campus)• 33rd rd largest Indigenous student intake in Australia (600)largest Indigenous student intake in Australia (600)• 30% International student enrolment (N’thn Europe, 30% International student enrolment (N’thn Europe,

China, India, Indonesia, Middle East, Canada)China, India, Indonesia, Middle East, Canada)• 10% students studying in distance mode10% students studying in distance mode• More of our students work in paid employment & More of our students work in paid employment &

longer hours in employment than the national average longer hours in employment than the national average (NTS reality)(NTS reality)


Are NTS capable of being successful at university?The national research evidence shows - The success rate (or tendency to pass

their year’s subjects) of low SES students is 97% of the pass rates of their medium & high SES peers & has been stable over the last 5 years (Bradley et al, 2008:30)

However, they require higher levels of support to succeed e.g., financial assistance, academic support, mentoring & counselling services (Commonwealth of Australia, 2009)


Therefore....implications for assessment practiceTaking account of assumed entry level

knowledge in unit content & assessment design & making this explicit to students

Building foundational academic skill development into unit content & assessment design

Developing commencing students skills as independent, self-regulating learners across the first year

Creating a unit & program level culture that respects & values diversity in all of its forms (age, gender, race etc.)


Student TransitionSo what is important to understand about

Student Transition?

An evidence-based model for conceptualising student transition

Senses of success (Lizzio, 2006)Predictors of student success


Meta-model 2: The ‘Five-Senses’ of Student Success

Sense ofStudent Identity

Sense of Connectedness

Sense of Capability

Sense of Purpose

Sense of Resourcefulness


(Lizzio, 2006)

What predicts commencing students’ satisfaction with their degree program?

Sense of Purpose

Sense of Capability

Good Teaching

Sense of Connection

Perceived Effectiveness of Orientation

Strongly Enhances

Enhances

Enhances

Enhances

Enhances

Time on Task Enhances

CommencingStudent

Satisfaction


What predicts commencing students’ academic outcomes?

Semester 1 GPA

Academic CapitalLow SES

First in FamilyESL

Competing DemandsTime in employment

Time as carer

Prior Academic AchievementEntry Scores (OP)

Task Engagement @ UniAttendance at Orientation

Time on taskStrongly Enhances

Enhances

Reduces

Reduces


What predicts commencing students’ retention?

Semester 1 GPA

Academic Capital -

Competing Demands -

Prior Academic Achievement +

Task Engagement @ Uni +

Sense of Purpose + + +

Student Satisfaction +

StudentRetentioninto yr 2


Therefore....Implications for practiceCurriculum strategies for developing time

on task & self-regulation skills assists assessment success

Strategies for enhancing sense of purpose in curriculum & assessment practices

Building academic & social capital through unit & assessment design throughout the FY

Effective Orientation Programs & early student engagement strategies to encourage attendance at Orientation


Step 2 – Assessment Principles & Practices So...what do students say about their

experience of assessment?

A quick overview of some salient research evidence!


Assessment characteristics which positively influence student learning & engagement

1. perceived validity of assessment tasks & approaches

2. perceived extent to which the learning environment is empowering

3. perceived fairness of the learning environment, especially with assessment tasks & practices


Students’ perceptions of the validity of assessment tasks & approaches (Sambell et al., 1997)

3 sets of priorities -1.Educational values – authentic/meaningful

tasks, perceived to have long term benefits, applying knowledge

2.Educational processes – reasonable demands, encourages independence by making expectations clear

3.Educational consequences – rewards effort and breadth & depth in learning


Students’ meta themes in assessment(Savin-Baden , 2004)

2 forms of student disempowerment:1.Unrewarded learning – the relationship

between quantity of work & its weighting2.Disabling assessment mechanisms

including:Processes – lack of information & inadequate

feedbackForms – assessment methods that do not fit

with espoused forms of learning


Students’ perceptions of fairness(Lizzio, Wilson & Hadaway, 2008)

Strongly influenced by –Extent to which they feel personally

respected by academic staff (convenors & sessional staff) in the learning & assessment process – relational culture

Adequacy of the information & support systems provided for them to “do their job” in relation to assessment


The Reality of Assessment for Commencing StudentsStudent feedback from multiple sources indicates:Strategic nature of Assessment for student

engagement, success & retentionFor many students assessment IS the learningAssessment items which are too difficult, not

meaningful, and not clearly explained are a key cause of student drop-out in the first semester

Well designed & implemented assessment is key to student engagement, learning & retention


Assessment Practice in the First YearSo what is important to our understanding?A model of purposes of FY assessment &

student motivation with assessmentA model for management of the FY

Assessment Lifecycle informed by :Student perspectiveStaff perspective Principles of Good Practice


Purposes of FY AssessmentDiagnostic Transition enabling

Motivating

Formative Learning Summative/evaluative Terminal

Meta-reflective

Assessment of student readiness

Assessment to aid transition to & engagement with uni

Assessment as stimulation for learning

Assessment for learningAssessment as learning Assessment of learningAssessment of

capability/mastery

Assessment of learning process


Student view of ‘motivating assessment’ (Wilson & Lizzio, 2011)

Engaging Design Relevance (personal, academic, professional),

intellectual challenge, teacher enthusiasmEnabling Management Task clarity, task scaffolding, formative

conversations with & support from staffTeacher Authority Clear and firm expectations & standards,

consequences explained, responsibility invoked


Meta-model 3: The first-year assessment lifecycle – student experience

6. Academic Outreach and Recovery


The first-year assessment lifecycle – staff experience

6. Academic Outreach and Recovery

DesignPhase

StudentPreparation Phase

AssessmentPhase

Post-AssessmentPhase


Principles of Effective Practice in First Year AssessmentGood Practice Principles rely on:1. A lifecycle approach to assessment (4 Phases)Design phaseStudent Preparation phase (pre-submission)Assessment phase (marking)Post-Assessment phase (feedback & feed-forward)2. A system’s approach to assessment (3 Systems

levels)Individual Unit/Course/Subject levelFirst Year Program level – horizontal & vertical

integrationDegree Program level


1. Design Phase: Individual Unit/SubjectFit for purpose – Optimising student motivation &

engagement by designing assessment tasks which are perceived to be relevant & meaningful to students & their learning (sense of purpose)

Level of difficulty & complexity – Designing tasks with a conscious understanding of the assumed entry level knowledge & appropriate learning level of commencing students

Progressive knowledge & skill development – Designing assessment tasks across a semester in such a way as to ensure effective, cumulative knowledge & skill development

Variety – Ensuring a variety of different types of items both within a unit/subject, and across units /subjects in the same semester (FY program level planning)


1. Design Phase: Individual Unit/SubjectEarly success -Optimising an experience of ‘early

success’ for students to build academic & personal efficacy & confidence

Formative Assessment - Emphasising early formative assessment designed to develop skill & confidence viz.

Early – weeks 4-6 is optimalSmaller pieces for either no marks, or fewer marks

(10-20%) to encourage recovery from possible failure (building hope)

Speedy, quality feedback (peer or staff, individually or collectively) with feedback by weeks 7-8

Self-assessment of all written items by self-evaluating on the identified criteria (builds meta-competence)


1. Design Phase: Individual Unit/ Subject LevelWritten presentation of assessment tasks which

present the task clearly and without ambiguity– Start with the aim of the taskPresent the sequence of steps involved in the

task, from the simple to the complex, thus breaking down large tasks into manageable chunks

Refer students to the marking criteria for the taskEnsure that all available information is provided as

much as possible in the one place


1. Design Phase – Essays – a “special case”There are a range of data sources to indicate that essays

in the first year are a source of great angst for both students and staff, and may be more complex than we (staff) think :

Professional Staff feedbackAcademic Staff feedbackStudent feedbackQuality of FY essays, especially in the first semesterQuality of student essays in the second & third years &

beyond!Wilson & Lizzio – Findings from ALTC Grant 2009-

2010Prof Keithia Wilson ALTC National Fellow

EngagingAssessment

Design

AssessmentManageme

nt & Support

Deep Approach

to Learning Assessment

Outcomes:

StudentGradesAssessment

Task Efficacy


Engaging Assessment

Design

AssessmentManageme

nt & Support

Deep Approach

to LearningAssessme

nt Outcome - StudentGradesAssessment

Task Efficacy


Engaging Assessment

Design

AssessmentManageme

nt & Support

Deep Approach

to LearningAssessme

nt Outcome - StudentGradesAssessment

Task Efficacy


1. Design Phase – Essays – a “special case” Semester 1 – essay tasks that are: Relatively short (e.g., 750-1000 words) Relatively simple (e.g., summary/description & simple

interpretative analysis such as answers to how or why questions)

Include limited research requirements (e.g., simple searches of readily obtainable information)

Semester 2 – essay tasks that are: Longer (1,200-2,500 words) Incorporate more complex analysis (e.g., critical analysis &

evaluation tasks – compare & contrast) Include more complex research requirements


1. Effective FY Assessment Practice Principles – Design Phase: FY Program Level

Workload distribution – Ensuring roughly equal workload between all units in a semester to assist development of time management skills

Scheduling – coordination of submission dates to stagger the student workload

Threshold/Difficult Units - Identify & front-load any threshold units in a semester to enable student success

Variety of assessment types – Ensure variety across units in a semester

Group Work – Coordinate any group-based assessment tasks across the FY Program & preferably reduce to 1 unit only if the assessment requires additional meetings outside of class


Task DesignProvide me with ‘fit for purpose’ tasks and roles

Meta-model 4: Progressive Enabling: How might we effectively manage the assessment process?

Self-ManagementHelp me by

managing yourself

Systems AwarenessHelp me by

coordinating theexperience


Reflective Task: Assessment Design PhaseHow does this information apply to your FY

context?

What are the opportunities for enhancing FY assessment design?

What are the challenges with enhancing FY assessment design?


2. Student Preparation Phase: Individual Unit/Subject Level

Motivating students – 3 Domains of Motivation

Articulating explicitly the academic relevance of each assessment task (the knowledge and skills that will be useful to students later in their course/degree);

Articulating explicitly the personal relevance of each assessment task (the knowledge and skills that will be useful for students in the future);

Articulating explicitly the professional relevance of each assessment task (the knowledge and skills that will be useful to students later in their career);



Motivating students – 2 Types of RelevanceArticulating future positive relevance

(personally & professionally) of the knowledge and skills to be gained from an assessment task to the course/degree/future employment;

Articulating future negative relevance of the knowledge and skills to be gained from an assessment task to the course/degree/future employment for students who may not engage sufficiently;



Motivating students – ChallengeArticulating explicitly the intellectual challenge of each

assessment task (the challenge offered to students to think and learn);

Articulating to students the investment of work required to be successful with the task (encourages development of self-regulatory behaviour);

Staff StanceStaff conveying to students their own personal enthusiasm

for the task (staff engagement increase student engagement);

Being clear and well organised (assists anxiety management & increases performance ability).



Management Strategies -Scaffolding learning -Designing a process for scaffolding

assessment preparation which prepares students for each assessment task:

Providing detailed goals, criteria, standards for each task to clarify what good performance is – viz. clear goals & standards

Providing opportunities for students to actively engage with, & potentially modify those goals, criteria, standards

Providing high & low quality examples of performance for each assessment task or a similar task

Providing targeted resources such as practice items, quizzes, mini-essay writing, step-by-step processes organised sequentially etc.

Providing multiple & regular opportunities for discussion of assessment tasks & requirements

Ensuring consistent information & resources are provided from all teaching staff (convenors & tutors) on assessment tasks


2. Student Preparation Phase: First Year Program LevelClear Goals and Standards – Ensuring all course

convenors have detailed criteria for all assessment tasks in their units/courses

Responsive culture – Ensuring a responsive FY Program culture in all units/courses, including training of sessional staff consistent messages about performance & success

Consistency of referencing style – Providing a single referencing style only for FY students for their FY of study for all units/courses in their Program

Consistency of information storage by unit convenors for web-site information for all units (same folders)

Consistent terminology to describe same types of assessment tasks across a program e.g., critical reflection/critical analysis/essay/critique


Facilitating Motivation

Help me to engage

Building Capacity Help me to be task

capable


Progressive Enabling: How might we effectively manage the assessment process?


managing yourself

Systems AwarenessHelp me by coordinating expectations


Facilitating Motivation

Help me to engage

Relating Functionally

Help me to feel understood

Managing Information

Help me to understandthe task

Facilitating Process

Help me to solve problems

Managing Procedures

Help me to navigate the rules of the

game

Building Capacity Help me to be task

capable


Progressive Enabling: How might we effectively manage the assessment process?


managing yourself

Systems AwarenessHelp me by

coordinating expectations


Reflective Task: Student Preparation PhaseHow does this information apply to your FY

context?

What are the opportunities for enhancing FY student preparation with assessment ?

What are the challenges with enhancing FY student preparation with assessment?


3. Assessment Marking Phase: Individual Unit/Subject LevelTransparency & Consistency – Ensuring

consistency of marking standards between markers by:

Marker preparation - having agreed, transparent , detailed

standards for preparing markers to assessMarking trial – initially marking 5 or so & then meeting to

compare standards of marking


3. Assessment Marking Phase: Individual Unit/Subject LevelMarker Moderation - facilitating high quality moderation of final assessment

marks (procedural justice) by staff teams viz. * staff team meeting face-to-face to discuss education &

capacity building for staff re marking standards * all staff reading all fails & HDs (depending on numbers)

to clarify understanding of high & low end standards * all staff reading & re-assessing all assessment items on

the margins of each grading category (high Ps & low Cs etc)

final re-adjustment of marks * keeping copies of high & low end examples to be used

anonymously with future cohorts (with student permission)


3. Assessment Marking Phase: Individual Unit/Subject Level

FY Assessment Debriefing & Learning for students

providing speedy feedback (2 weeks optimal)Providing quality individual feedback which can

feed-forward into other assessment tasks - * identifying achievements (encouragement) * identifying clearly & explicitly what students need

to do to improve their performance in an encouraging

way (developmental) * sufficient comments to justify the mark (fair & just)summarising cohort strengths & weaknesses


4. Post-Assessment Phase: Individual Unit/Subject LevelTimely feedback – Ensuring students

receive feedback within a short time frame (1-2 weeks is optimal) during semester

Empowerment - Ensuring opportunities for individual and group discussion of marks and grades

Academic recovery – identifying and intervening with at-risk students who have failed their first assessment item.


4. Post-Assessment Phase: First Year Program LevelEmpowerment - Creating a first year culture of

ensuring opportunities for individual and group discussion of marks and grades

Academic recovery – Program level strategy by identifying & front-loading the threshold (most difficult) unit/s with the aim of ensuring intervening with at-risk students who have failed their first assessment item to assist in their academic recovery

.


Some Strategies for Academic RecoveryPreventative Strategies –Draft submissions for first assessment tasks with

feedback providedRe-submission of a failed assessment task for a

possible passing grade only (1 only in each semester, or semester 1 only, or the first year)


Potential hierarchy of academic recovery interventions1. Students responsible for initiating help-seeking

without prompting;2. Email communication to all students inviting

contact to discuss first assessment and providing information regarding support services;

3. In-class activity discussing first assessment performance and feedback;

4. Targeted written communication to students who failed or nearly failed first assessment inviting contact to discuss first assessment and providing information regarding support services;


Potential hierarchy of academic recovery interventions 5. Self-reflective workbook distributed to students

for independent completion and invitation for follow-up contact;

6. Phone call to students who failed or nearly failed first assessment inviting contact to discuss first assessment and providing information regarding support services;

7. Targeted invitation to students who failed or nearly failed first assessment inviting participation in structured face-to-face consultation and planning session with tutor.


First-Assessment First-Feedback Academic Recovery InterventionKey Idea: Efficacy building for students who ‘fail’ or ‘marginal

pass’ first assessment in a core/threshold courseKey Aspects: Students complete a self-directed workbookIndividual structured session with tutor leading to

an action planFollow-up phone or email contact40% uptakeParticipation results in a 10% increase in

submission rates & 20% increase in pass rates for 2nd assessment item, & 40% increase in passing the course overall


First-Assessment First-Feedback Academic Recovery InterventionAt-risk students’ self-reported evaluations of the

process were uniformly positive –Academic related learning (5.7/7)Personal development (5.02/7)Insight into reasons for under-performance (5.6/7)Increased efficacy & optimism (5.6/7)Process rated as non-aversive (5.3/7)Tutors reported stronger relationships with

students, higher attendance at tutes by those students, & greater student engagement


Reflective Task: Post-Assessment PhaseHow does this information apply to your FY

context?

What are the opportunities for enhancing FY post-assessment feedback processes?

What are the challenges with enhancing FY post-assessment feedback processes?


Working with Diverse Students - implications for curriculum design & assessment Identifying our assumptions about the entry levels of

Assumed Knowledge of our students (x course + x first semester & then first year overall) & designing course content, structure & assessment tasks to take account of FY students’ entry levels

Scaffolding student understanding of assessment tasks by providing resources to make expectations explicit & assist task understanding

Scaffolding student learning in each unit/course by building required attitudes, knowledge & skills progressively across the first semester & first year

T&L in the FYE needs to be conceptualised as a social experience where students are provided with rich & varied opportunities for interaction & dialogue with peers & academic staff & we are building a relational school/program culture


Documents

Prof Keithia Wilson Program Lead for the FYE, Griffith Health GIHE Senior Fellow for the FYE 2010 ALTC National Fellow for the FYE 2007 Australian University