Carleton.ca/edc EDUCATIONAL DEVELOPMENT CENTRE IDENTIFYING AND SUPPORTING AT RISK STUDENTS Ryan Kuhne October 13 th,2015 carleton.ca/edc/atrisk EDUCATIONAL

carleton.ca/edcEDUCATIONAL DEVELOPMENT CENTRE

EDUCATIONAL DEVELOPMENT CENTRE

IDENTIFYING AND SUPPORTING AT RISK STUDENTS

Ryan KuhneOctober 13th,2015carleton.ca/edc/atrisk



WHY IS THIS IMPORTANTo Course design and assessmento Insight into engagement and online interaction o Improve your teaching scoreso Reduce you DFW rates (retention)o Helps students succeed (<60% grad in 6yrs)


IT’S AS EASY AS 1, 2, 3

1. What does “At Risk” mean for your course?

2. How do we identify these Students?

3. What can you do to help?


High Medium Low

WHAT IS AN “AT-RISK” STUDENTo Engagemento Gradeso DFWs

0-9 10-19 20-29 30-39 40-49 50-59 60-69 70-79 80-89 90-100kle95.deviantart.com


DISCLAIMER

flickr.com/photos/skinnylou

Stress, Finances, Time management, Course work load, Personal struggles, Too much freedom, Poor work habits, Study skills, Writing skills, Work-study balance, etc….


carleton.ca/edcEDUCATIONAL DEVELOPMENT CENTRE 7/13/10

6

mscrys.deviantart.com

carleton.ca/edcEDUCATIONAL DEVELOPMENT CENTRE 7/13/10

7

mscrys.deviantart.com


AVAILABLE DATAStudent Data

Instructional

GradesQuizzes

AssignmentsParticipation/

attendance

LMS Engagement DataTime online

First/last accessed# Online sessions

Discussions read/posted

Interactions with other users

Content viewedInterface usage

(ex view grades)

Other Audience Response Systems

Where do they sit

Institutional

Entry Information

Demographics High schoolePortfolio

Previous Instructional

RecordsPast grades

Year of studyProgram infoTeaching eval.

Support Services Accessed

LibraryHelp desk

Counseling

InteractionsStudent – InstructorStudent – TAStudent – Student


CATEGORIZING "AT RISK" STUDENTS?

High-risk (Red) students

Medium-risk (Yellow) students

Low-risk (Green) students

Predictive modeling and historical data suggests that

students who are more engaged and have higher marks early

on in the semester will continue to be high achieving (Green)

and unengaged students with low or missing grades early in

the semester are much more likely to end up "At Risk" (Red).EDUCATIONAL DEVELOPMENT CENTRE flickr.com/photos/nielssienaert


HIGH-RISK STUDENTS TRAITSo Frequently absent from classo Rare-low cuLearn useo Participation (in-class/online) is non

existent or extremely lowo Consistent low or failing gradeso Work not handed in or of poor quality o At risk of failing course

(cumulative grade of D+ or lower)smartphotostock.com


MEDIUM-RISK STUDENTS TRAITSo Irregular attendance patternso Low-average cuLearn useo Participation (in-class/online)

is low-average o Consistently low or erratic

gradeso Work is below average qualityo Minor risk of failing

(cumulative grade between C+ and C-)

flickr.com/photos/e-coli


LOW-RISK STUDENTS TRAITSo Regularly attends classeso Average-high cuLearn useo Participation (in-class/online) is average-

high o Consistent above average or

high gradeso Work is above average qualityo No visible risk of failure (cumulative grade

of B- or higher)12

flickr.com/photos/wlodi


“AT-RISK” IN YOUR COURSEo What’s a failing grade?

Presently or close to, Institutional/Programo Opportunity to improve? o What support are you willing to provide?



IDENTIFYLEARNING ANALYTICS

GRADES&

ENGAGEMENTDATA

EDUCATIONAL DEVELOPMENT CENTRE primeview.fr/wp-content/uploads/2011/06/photodune-2464846-crowd-of-people-l.jpg


o Assess early and ofteno Create checkpointso Grade to passo Sort the gradebooko Export the gradebook

Sort/Conditional Formatting

o Contribution to course totalo Rank & Average

flickr.com/photos/billypalooza/3184530981

dreamstime.com


ENGAGEMENT DATA Rio Salado Collage – Have devised a predictive modeling system that they believe to be 70% accurate after the first week of class.The New Diagnostics - Oct. 30, 2009 Inside Higher Ed

The order in which students registered their devices was an accurate predictor of final student grades.Early identification of at-risk students using a personal response system (Griff & Matter 2008)

General LMS activity can be used to identify at risk students, as early as two weeks from the beginning of the course. "Hits" (not "Discussion Posts") predict student success in online course (Ramos & Yudko 2008)

The number of forum postings, mail sent, and assessments completed function as significantly predictive variables of student final grades. Mining LMS data to develop an “early warning system” for educators (Macfadyen & Dawson 2009)

http://www.insidehighered.com/news/2009/10/30/predict


http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6VCJ-4MV19RG-2&_user=4799849&_coverDate=05/31/2008&_rdoc=1&_fmt=high&_orig=search&_sort=d&_docanchor=&view=c&_acct=C000051236&_version=1&_urlVersion=0&_userid=4799849&md5=016d706a45e329148aec2a1990601c5a




http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6VCJ-4XBF8RB-4&_user=4799849&_coverDate=02/28/2010&_rdoc=1&_fmt=high&_orig=search&_sort=d&_docanchor=&view=c&_searchStrId=1393774588&_rerunOrigin=scholar.google&_acct=C000051236&_version=1&_urlVersion=0&_userid=4799849&md5=981f26c59866f9117429c44493ad43d7

http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6VCJ-4XBF8RB-4&_user=4799849&_coverDate=02/28/2010&_rdoc=1&_fmt=high&_orig=search&_sort=d&_docanchor=&view=c&_searchStrId=1393774588&_rerunOrigin=scholar.google&_acct=C000051236&_version=1&_urlVersion=0&_userid=4799849&md5=981f26c59866f9117429c44493ad43d7


WHAT KIND OF DATA IS AVAILABLEo Content viewedo Discussions (read/posted)o Assessments (viewed/completed/submitted)o First/last accessedo # online sessionso Interface usage (ex view grades) o Social (Interactions with other users)o Hours of time watching videos*


Example course link

https://culearn.carleton.ca/moodle/course/view.php?id=57236


REPORTS > LOGS AND LIVE LOGS

o Filter by course actions / participant• Last time student accessed course• How active a particular student has been

logger.believermag.com


REPORTS > ACTIVITY REPORTo All activity in the course on a single page, sorted

by topic / week numbero Quantity of clicks each item in the course has

recorded


REPORTS > COURSE PARTICIPATION

o Monitor general participation in an activityo Send a message to non-participants

newlinetrophy.com


REPORTS > STATISTICS

Can you spot the first test and the days of there are classes?


ENGAGEMENT ANALYTICSo Visual indicator of risk levelo Indicators identified to

have an impact on student success

o No automaticintervention

o Customizable



ALGORITHMo Integrating an at-risk student model

into learning management systems (Silva, Hobbs & Graf, TISE 2014)


o Have to access through us

https://analytics.google.com/analytics/web/%23report/content-drilldown/a2885440w57982636p59171588/?explorer-table.plotKeys=%5B%5D&_r.drilldown=analytics.pagePathLevel1:/Fall+2015+(September-December)//


SNAPPSOCIAL NETWORKS ADAPTING PEDAGOGICAL PRACTICE

o Aim: to develop a student network visualization tool from the extraction of discussion forum activity.

o Postso Posts/usero Thread depth*o Social network centrality*o University of Wollongongsnappvis.org

http://research.uow.edu.au/learningnetworks/seeing/snapp/index.html


ARE WE ASKING THE RIGHT QUESTIONS?o Assumptions: garbage in – garbage outo Measurement (numbers) vs

Evaluation (the interpretation of the numbers)o Ability to predict risk associated with a student is

powerful, knowing what to do to maximize each student's probability for success is even more powerful



HELP

COMMUNICATE, PROVIDE

FEEDBACK, ADVICE & SUPPORT

EDUCATIONAL DEVELOPMENT CENTRE law.georgetown.edu/continuing-legal-education/cle-cpe-credit/cpe/


HOW CAN YOU HELP?o Clear Learning Outcomeso Over communicateo Feedback (personalized or general)o Examples (personalized or general)o Peer comparison and supporto Emotional supporto Learning support


COURSE SETUPo Share grading rubrics and marking guides.o Design your course to have assessment spread out

through the term and early on in the term so it is easier for students to track their progress and have a chance to correct.

o Build in course check points at several points throughout the term and make students aware of them and discuss the in class.

7/13/10


OVER COMMUNICATEo Use class time, emails, news forum, a blog or twitter feed as

a way to interact with students outside of the classroom. o Dates, deadlines and resources.

• Your office hours (TAs too) and availability• Assessment deadline reminders• Assessment averages and general feedback• Discipline specific resources• Where to get help on campus• Peer study sessions and tutoring• Record tutorials or office hours


FEEDBACK INo Grades (inline and paragraph)o Assignments (comments, file, rubric, marking guide)o Quizzes (question, quiz, grade range, repetition)o Choice (Polls) o Forums (text, rating)o Lesson (self guided feedback)o Workshops (grade for submission & feedback, publish)o Selective Release (Based on grades)


EXAMPLES AND RE-WRITESo Invite students to submit topics/questions to help you create

a review day that focuses on their needs.o Offer optional homework or practice quizzes or exam

questions that will help students get an understanding of what to expect on exams.

o Consider allowing students a re-write option for one assessment in an effort to improve a grade, using your feedback as a guideline.


PEER COMPARISON AND SUPPORTo Share exemplary worko Course or assessment specific FAQ forum o Have students assess each others worko Student Academic Success Centre (SASC)

• Peer Assisted Subject Coaching (PASC) • Peer Assisted Study Sessions (PASS)

o Student Experience Office (SEO)• The Mentorship Network

http://carleton.ca/sasc/

http://carleton.ca/seo/

http://carleton.ca/seo/


INTERVENTION STRATEGYo Email the students to let them know how they are doing in the

course and offer support. (See sample emails) • Establish yourself as someone who not only cares about their success, but is

available to help them along the way.• Try to identify the source of the problem. Is the student struggling to

comprehend material, or is it a problem outside of the classroom?• Have the student work out a plan of action. Help by providing perspective

on assignments left, their weight, expectations, etc. • Be informative. Provide them with some referrals to other areas around

campus that may be able to help. Counseling? Writing services? Mentoring? • Remind them that you're here. Encourage them to come back or contact

you whenever they need help with course material.

Encourage, re-engage and direct to support.


ON CAMPUS RESOURCES*o Awards and Financial Aid

613-520-3600 carleton.ca/awardsResponsible for providing money, encouragement, and financial advice to students.

o Equity Services613-520-5622 carleton.ca/equitySupports Carleton’s commitment to diversity as a source of human excellence, cultural enrichment and social strength.

o Student Experience Office613-520-7595 carleton.ca/seoHelps students make a successful transition to university life and continues to support them through university.

o Health and Counselling Services613-520-6674 carleton.ca/healthOffers medical and counselling services and a health promotion program.

o International Student Services Office

613-520-6600 carleton.ca/issoCentralizes all services for registered international students, like immigration and visa support, and administers exchange programs.

o Paul Menton Centre for Students with Disabilities613-520-6608 carleton.ca/pmcCoordinates academic accommodations and support services for Carleton students with disabilities.

o Student Academic Success Centre(including Learning Support Services)613-520-7850 carleton.ca/sascOffers a range of services to meet your individual learning needs including academic advising, Learning Support Services, Writing Tutorial Services and the Peer Assisted Study Sessions.

o Student Affairs613-520-2573 carleton.ca/studentaffairsHelps to create a living and learning environment that promotes student success in both academic and extracurricular activities.

o Office of the Associate Vice-President

(Students and Enrolment)613-520-2874 carleton.ca/studentsupportProvides comprehensive support for all students, from your first point of contact with the university through to graduation.

o Ombuds Services613-520-6617 carleton.ca/ombudsAn independent office that helps studentsfind solutions to on- and off-campus problems.

o University Safety613-520-2600 ext. 3612 carleton.ca/safetyCommitted to providing a safe campus environment for all member of the Carleton community.

*good2talk.ca is a free, confidential and anonymous helpline providing professional counselling and information and referrals for mental health, addictions and well-being to post-secondary students in Ontario, 24/7/365.

Care Report

http://www.good2talk.ca/

http://carleton.ca/studentaffairs/care-report/

http://carleton.ca/studentaffairs/care-report/



EDUCATIONAL DEVELOPMENT CENTRE metabolon.com/media/33534/shutterstock_163862846_case-study_web.jpg


EX 1: A 2ND YEAR PSYCHOLOGY COURSE1. What does At Risk mean for your course?

Below 60% / D+ or lower

2. Identify the At Risk studentsMake a Check Point calculated column in the Gradebook

40 % - 12 Weekly Tests (best 10/12)20 % - 4 Assignments 40 % - Final Exam

5 Quizzes (take best 4 of 5) and 1 Assignment are completedSeparate students into groups

Nothing handed in, Currently failing (below 50%), D students (between 50%-60%)

3. Contact the students offering helpSee sample emails

7/13/10


EX 1: A 2ND YEAR PSYCHOLOGY COURSE

o The “At Risk Checkpoint” column computes the grade a student would get if the course ended today.

Best 4 of 5 quizzes 4% each > total of 16% Assignment 1 5%

o Nothing handed inLost 21% of course marks, but can still get a up to a B+

o Currently failing (below 50%)Missing 2-3 assessments, Lost 11-20% of course mark, but can still get a up to a B+/A

o D students (between 50%-60%)Lost 11-20% of course mark, but can still get a up to an A+

21%


Nothing Handed in

Dear Student,

I am very concerned about your lack of participation in Course Name. To date, you have not handed in any of the course assignments or taken any of the quizzes. As a result you are currently failing the course and have lost ??% of the term marks available to you. It's important that you take an active role in your own success and if you need help, please do not be afraid to ask.

The University and I care about your success. I encourage you take advantage of the Student Academic Success Centre (SASC). You can receive help with academic improvement planning, study skills workshops, Peer Assisted Study Sessions(PASS), and writing tutorials.Also this course has a PASS facilitator assigned to it. The PASS program has proven to help students learn and meets Date time and location of PASS Session.

You can of course visit me during my office hours, (Date and Time of Office Hours or make an appointment if you have a conflict) and we can discuss how to improve your performance.

If you attempted to drop this course earlier in the term, it's important that you follow-up on Carleton Central to ensure this course no longer appears in your schedule because you are still registered in this course.

Please consider taking an active role in improving your academic performance. Let me know if there is anything I can do to help you.

Best,

http://www.carleton.ca/sasc

http://www2.carleton.ca/sasc/peer-assisted-study-sessions/


Have below an F or below 50%

Dear Student,

I am concerned that you may be experiencing difficulty in Course Name, as you are currently failing the course with an average of less than 50% on your course assessments. It's important that you take an active role in your own success and if you need extra help, do not be afraid to ask. There is still time for you to improve your grades, and by doing so, I know you will succeed.

The University and I care about your success. I encourage you take advantage of the Student Academic Success Centre (SASC). You can receive help with academic improvement planning, study skills workshops, Peer Assisted Study Sessions (PASS), and writing tutorials.Also this course has a PASS facilitator assigned to it. The PASS program has proven to help students learn and meets Date time and location of PASS Session.



Best,




Have D- to D+ or 50%-60%

Dear Student,

I am concerned that you may be experiencing difficulty in Course Name, as you are currently in the D grade range with an average between 50% and 60% on your course assessments. It's important that you take an active role in your own success and if you need extra help, do not be afraid to ask. There is still time for you to improve your grades, and by doing so, I know you will succeed.

The University and I care about your success. I encourage you take advantage of the Student Academic Success Centre (SASC). You can receive help with academic improvement planning, study skills workshops, Peer Assisted Study Sessions (PASS), and writing tutorials.Also this course has a PASS facilitator assigned to it. The PASS program has proven to help students learn and meets Date time and location of PASS Session.



Best,




SAMPLE EMAILDear Student,I are concerned that you may be experiencing difficulty in this course. I want you to succeed and there is still lots of time in the semester to improve your grade by doing the following things:o Attend all tutorials. Attempt the problems yourself. o Read the relevant parts of the text as they are assigned. o Do all the problems as they are assigned. o When you realize that there is something you do not understand, get help ASAP – from a friend, a TA, or from me.o Attend PASS sessions if you can schedule them. These are known to improve students’ grades dramatically. o Come and see me during Office hours ######

I know that university can be very challenging and in my experience, students seeking support at this early point in the term have a much better chance of successfully completing their courses. There is help available and the following resources can get you back on track;

• At the Student Academic Success Centre you can receive help in the following areas: a) Academic Improvement Planning, a step-by-step guide to put you on the pathway of improving your academic performance.

b) Learning Support Services, which include workshops on study skills, writing tutorials, group study space, tutor referrals and more.c) Peer Assisted Study Sessions, a study group program that's focused on this course. Students meet together to go over concepts, course material, etc.d) Writing Tutorial Services provides assistance in the area of academic writing.e) Advising Centre is available for you to go over the requirements for their major, make changes and get support.

• The Ecumenical Chaplaincy is here to provide religious support, counselling, one-on-one support, study groups, meditation and more.• The Health and Counselling Service is a multidisciplinary service providing medical services, counselling services, and health promotion.

Do not give up! You can go on to do well in Course Name if you put in the effort. Best,


http://www2.carleton.ca/sasc/advisingcentre/academic-improvement-planning/

http://www2.carleton.ca/sasc/learning-support-services/


http://www2.carleton.ca/sasc/writing-tutorial-service/

http://www2.carleton.ca/sasc/advisingcentre/

http://www.carleton.ca/ecumenicalchaplaincy

http://www.carleton.ca/health


http://www.itap.purdue.edu/studio/passnote/


WE STILL HAVE A LONG WAY TO GO



TAKE AWAYS…

“at-risk” students generally took that information as either a motivational kick in the rear or were prompted to quickly drop the class -- and were grateful in any case.”

“A double-blind study conducted during the first two years of the Purdue's program, called Signals, revealed that 67 percent of students who learned they were in the middle- or high-risk categories were able to improve their grades." EDUCATIONAL DEVELOPMENT CENTRE The New Diagnostics - Inside Higher Ed






TAKE AWAYS

ASSESS OFTEN,COMMUNICATE MORE, OFFER

SUPPORT


studentsagainstdepression.orgcarleton.ca/edc/atrisk


carleton.ca/edc/events/

Documents

Carleton.ca/edc EDUCATIONAL DEVELOPMENT CENTRE IDENTIFYING AND SUPPORTING AT RISK STUDENTS Ryan Kuhne October 13 th,2015 carleton.ca/edc/atrisk EDUCATIONAL