Identifying At-risk Students

Identifying At-risk StudentsPresented by Ryan Kuhne

Educational Technology ConsultantCarleton University in Ottawa, Canada

Ryan Kuhne from Carleton University Identifying At-risk Students carleton.ca/edc/atrisk

Agenda

What is an “At Risk” student?

1.What does “At Risk” mean for your course?

2.How do we identify these Students?– Grade related data– Engagement related data

3.What can we can do to help?– Now– For Future Courses

carleton.ca/edc/atrisk7/13/102


Why do this?

• It will help students in need!

• It will help keep students engaged!

• It will help you think about your course design and assessments!

• It will give you insight into how your students are interacting with the online portion of your course!

• It will potentially improve your teaching review scores!

• It will potentially reduce you DFW rates! Retention

• It's really easy and does not take much time!

7/13/103


"At Risk" simply refers to a student in danger of under-performing academically.

What is an "At Risk" student?

7/13/104

At Risk

Ryan Kuhne from Carleton University Identifying At-risk Students carleton.ca/edc/atrisk 7/13/105



Academic Analytics

7/13/107

InteractionsStudent – ContentStudent – InstructorStudent – StudentStudent – Interface


Chart from Using Analytics to Intervene with Underperforming College StudentsEDUCAUSE Learning Initiative, Annual ConferenceKunnen, Eric J. and John Fritz 2010

7/13/108

CMS and other technologies

(real time)

SIS data(historic)

Other data

Specific and customizable interventions

Prediction

http://www.educause.edu/Resources/UsingAnalyticstoIntervenewithU/196159


Retention – Institutional

• Signals by Perdue and Sungardsungardhe.com/signalson NBC Nightly News story

• Starfish Retention Solutionsstarfishsolutions.comAlabama State, Seton Hall, U of Chicago, Cincinnati, Pittsburgh…Bb World 08 - http://www.viddler.com/explore/starfish/videos/1/

• Less than sixty percent of undergraduates now graduate within six years.

• The result is lost revenue, lost potential, and lost dreams.

• Increasingly, campuses are being held accountable for their graduation rates…and their drop-out rates. What Faculty Members Need to Know About Retention: A Magna White Paper

7/13/109

http://www.sungardhe.com/signals

http://www.msnbc.msn.com/id/3032619/vp/32634348%2332634348

http://www.starfishsolutions.com/

http://www.viddler.com/explore/starfish/videos/1/

http://www.viddler.com/explore/starfish/videos/1/

https://cumail.carleton.ca/owa/redir.aspx?C=f2cd101df4df43aa954728c6a4b3b473&URL=http://www.magnapubsmail.com/ls.cfm?r=171650464&sid=9951412&m=1046997&u=MAGNAPUB&s=http://www.magnapubs.com/catalog/publications/603530-1.html?s=cj&p=email


Starfish

7/13/1010

Chart from Starfish Corporate OvervieW.pdf

• Identify, Track, Retain - Engage, Motivate, Graduate

• Connects students, instructors, counselors and data

http://www.starfishsolutions.com/pdf/StarfishOverview.pdf


Categorizing "At Risk" students?

High-risk (Red) studentsMedium-risk (Yellow) students Low-risk (Green) students

Predictive modeling and historical data suggests that students who are more engaged and have higher marks early on in the semester will continue to be high achieving (Green) and unengaged students with low or missing grades early in the semester are much more likely to end up "At Risk" (Red).

7/13/1011


High-risk (Red) Students Traits

• Frequently absent from class

• Rare-low WebCT use

• Participation (in-class/online) is non existent or extremely low

• Consistent low or failing grades

• Work not handed in or of poor quality

• At risk of failing course (cumulative grade of D+ or lower)

7/13/1012

Photo by otubo

http://www.flickr.com/photos/otubo/


Medium-risk (Yellow) Students Traits

• Irregular attendance patterns

• Low-average WebCT use

• Participation (in-class/online) is low-average

• Consistently low or erratic grades

• Work is below average quality

• Minor risk of failing (cumulative grade between C+ and C-)

7/13/1013

Photo by Latente

http://www.flickr.com/photos/e-coli/


Low-risk (Green) Students Traits

• Regularly attends classes

• Average-high WebCT use

• Participation (in-class/online) is average-high

• Consistent above average or high grades

• Work is above average quality

• No visible risk of failure (cumulative grade of B- or higher)

7/13/1014

Photo by włodi

http://www.flickr.com/photos/wlodi/


What is an "At Risk" student?

7/13/1015


So what can instructors do?

It’s as easy as 1, 2, 3

1.What does “At Risk” mean for your course?

2.How do we identify these Students?

3.What can you do to help?

7/13/1016


1. What does "At Risk” mean for your course?

1. What do you consider a failing grade for this course? Institutional/Program

2. For the course, does at risk mean that the student is presently failing, or close to failing.

3. Will the student have enough opportunity to improve their grade after our initial contact?

4. What support are you willing to provide?

7/13/1017


2. How do we identify these Students?

• Grade Related Data

• Engagement Data

7/13/1018

Engagement DataTime onlineFirst Access

# Online SessionsDiscussions read/posted

MailFiles Viewed

MyGrades views

GradesAssessmentsAssignmentsParticipation


2. How do we identify these Students?Grade Related Data• What assessments are done to date?• Are there any lab or tutorial grades?• Do you have any participation or attendance grades?

Sort the gradebook columns to see who is doing poorly.

Set up a couple course check points (calculated column).• Figure out the formula for final course grade if the class

were to end today.

We will look at an example in a few slides.

7/13/1019


2. How do we identify these Students?Engagement Data


Engagement DataTracking / Metrics / Analytics

7/13/1021

You are able to export the report data and import it back to the gradebook to use in identifying At Risk students.



Engagement Data • The New Diagnostics - Oct. 30, 2009 Inside Higher Ed

Rio Salado Collage – Have devised a predictive modeling system that they believe to be 70% accurate afterh the first week of class

• Early identification of at-risk students using a personal response systemThe order in which students registered their devices was an accurate predictor of final student grades. (Griff & Matter 2008)

• "Hits" (not "Discussion Posts") predict student success in online coursesGeneral LMS activity can be used to identify at risk students, as early as two weeks from the beginning of the course. Research with a broader range of course subjects and a larger number of courses is needed to substantiate a general claim. (Ramos & Yudko 2008)

• Mining LMS data to develop an “early warning system” for educatorsThe number of forum postings, mail sent, and assessments completed function as significantly predictive variables of student final grades. Findings suggest that for the purposes of monitoring student activity and achievement, predictive models must be developed at the course level. (Macfadyen & Dawson 2009)

7/13/1023

http://www.insidehighered.com/news/2009/10/30/predict

http://www.google.ca/url?sa=t&source=web&cd=6&ved=0CC4QFjAF&url=http://webcentral.uc.edu/eProf/media/attachment/eprofmediafile_299.pdf&ei=laQ0TLO3DYKBnwe8s7jcAw&usg=AFQjCNFAcXBxfY_J9zaeTz4d402WjBokmw&sig2=JF5unJJ72dTfOHbWyWOJYQ

http://www.google.ca/url?sa=t&source=web&cd=6&ved=0CC4QFjAF&url=http://webcentral.uc.edu/eProf/media/attachment/eprofmediafile_299.pdf&ei=laQ0TLO3DYKBnwe8s7jcAw&usg=AFQjCNFAcXBxfY_J9zaeTz4d402WjBokmw&sig2=JF5unJJ72dTfOHbWyWOJYQ

http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6VCJ-4MV19RG-2&_user=4799849&_coverDate=05/31/2008&_rdoc=1&_fmt=high&_orig=search&_sort=d&_docanchor=&view=c&_acct=C000051236&_version=1&_urlVersion=0&_userid=4799849&md5=016d706a45e329148aec2a1990601c5a

http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6VCJ-4MV19RG-2&_user=4799849&_coverDate=05/31/2008&_rdoc=1&_fmt=high&_orig=search&_sort=d&_docanchor=&view=c&_acct=C000051236&_version=1&_urlVersion=0&_userid=4799849&md5=016d706a45e329148aec2a1990601c5a

http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6VCJ-4XBF8RB-4&_user=4799849&_coverDate=02/28/2010&_rdoc=1&_fmt=high&_orig=search&_sort=d&_docanchor=&view=c&_searchStrId=1393774588&_rerunOrigin=scholar.google&_acct=C000051236&_version=1&_urlVersion=0&_userid=4799849&md5=981f26c59866f9117429c44493ad43d7


SNAPPSocial Networks Adapting Pedagogical Practice

• research.uow.edu.au/learningnetworks/seeing/snapp

7/13/1024

http://research.uow.edu.au/learningnetworks/seeing/snapp/index.html

http://research.uow.edu.au/learningnetworks/seeing/snapp/index.html


3. What can we do to help?

7/13/1025

Olive branch


3. Helping Your Current Courses

Hands Off• Use selective release based on grades to show at risk

students extra resources in the course.• Over communicate dates, deadlines and resources.

5 tips to over communicating without seeming obvious in targeting at-risk students

• Encourage students to check the course more frequently.• Put links to tutoring and other support services in the course.

Hands On• Email the students to let them know how they are doing in

the course compared to their classmates (offer help). • Make yourself available for questions and extra help.• Meet with the students to talk about why they are not engaged

with the course.5 goals you should strive for in your meeting.

7/13/1026

http://www2.carleton.ca/edc/atrisk/













3. Helping Your Future Courses

• Design your course to have assessment spread out through the term and early on in the term so it is easier for students to track their progress and have a chance to correct.

• Build in course check points at several points throughout the term and make students aware of them.

• Over communicate – Place info/links in syllabus and online– Your office hours– Your availability to meet– Discipline specific resources– Where to get help on campus– Setup peer study sessions

7/13/1028


EX 1 A 2nd Year Psychology course

1. What does At Risk mean for your course? Below 60% / D+ or lower

2. Identify the At Risk studentsMake a Check Point calculated column in the Gradebook The final grade is calculated as such: 40 % - 12 Weekly Tests (best 10 out of 12) 20 % - 4 Assignments 40 % - Final Exam 5 Quizzes (take best 4 of 5) and 1 Assignment are completed

Separate students into groupsNothing handed in, Currently failing (below 50%), D students (between 50%-60%)

See sample image on next slide3. Contact the students offering help

See sample emails

7/13/1029



7/13/1030



7/13/1031


Nothing Handed in Dear Student,

I am very concerned about your lack of participation in Course Name. To date, you have not handed in any of the course assignments or taken any of the quizzes. As a result you are currently failing the course and have lost ??% of the term marks available to you. It's important that you take an active role in your own success and if you need help, please do not be afraid to ask.

The University and I care about your success. I encourage you take advantage of the Student Academic Success Centre (SASC). You can receive help with academic improvement planning, study skills workshops, Peer Assisted Study Sessions (PASS), and writing tutorials.Also this course has a PASS facilitator assigned to it. The PASS program has proven to help students learn and meets Date time and location of PASS Session.

You can of course visit me during my office hours, (Date and Time of Office Hours or make an appointment if you have a conflict) and we can discuss how to improve your performance.

If you attempted to drop this course earlier in the term, it's important that you follow-up on Carleton Central to ensure this course no longer appears in your schedule because you are still registered in this course.

Please consider taking an active role in improving your academic performance. Let me know if there is anything I can do to help you.

Best, 7/13/1032

http://www.carleton.ca/sasc

http://www2.carleton.ca/sasc/peer-assisted-study-sessions/


Have below an F or below 50% Dear Student,

I am concerned that you may be experiencing difficulty in Course Name, as you are currently failing the course with an average of less than 50% on your course assessments. It's important that you take an active role in your own success and if you need extra help, do not be afraid to ask. There is still time for you to improve your grades, and by doing so, I know you will succeed.




Best, 7/13/1033




Have D- to D+ or 50%-60% Dear Student,

I am concerned that you may be experiencing difficulty in Course Name, as you are currently in the D grade range with an average between 50% and 60% on your course assessments. It's important that you take an active role in your own success and if you need extra help, do not be afraid to ask. There is still time for you to improve your grades, and by doing so, I know you will succeed.




Best, 7/13/1034




Sample EmailDear Student,I are concerned that you may be experiencing difficulty in this course. I want you to succeed and there is still lots of time in the semester to improve your grade by doing the following things:•Attend or watch all tutorials. Attempt the problems yourself. •Read the relevant parts of the text as they are assigned. •Do all the problems as they are assigned. •When you realize that there is something you do not understand, get help ASAP – from a friend, a TA, or from me.•Attend PASS sessions if you can schedule them. These are known to improve students’ grades dramatically. •Come and see me during Office hours ######

I know that university can be very challenging and in my experience, students seeking support at this early point in the term have a much better chance of successfully completing their courses. There is help available and the following resources can get you back on track;

• At the Student Academic Success Centre you can receive help in the following areas: a) Academic Improvement Planning, a step-by-step guide to put you on the pathway of improving your academic

performance.b) Learning Support Services, which include workshops on study skills, writing tutorials, group study space, tutor referrals and more.c) Peer Assisted Study Sessions, a study group program that's focused on this course. Students meet together to go over concepts, course material, etc.d) Writing Tutorial Services provides assistance in the area of academic writing.e) Advising Centre is available for you to go over the requirements for their major, make changes and get support.

• The Ecumenical Chaplaincy is here to provide religious support, counselling, one-on-one support, study groups, meditation and more.

• The Health and Counselling Service is a multidisciplinary service providing medical services, counselling services, and health promotion.

Do not give up! You can go on to do well in Course Name if you put in the effort. Best,

7/13/1035


http://www2.carleton.ca/sasc/advisingcentre/academic-improvement-planning/

http://www2.carleton.ca/sasc/learning-support-services/


http://www2.carleton.ca/sasc/writing-tutorial-service/

http://www2.carleton.ca/sasc/advisingcentre/

http://www.carleton.ca/ecumenicalchaplaincy

http://www.carleton.ca/health


EX 2 A 2nd Year Environmental Science course

1. What does At Risk mean for your course? Below 60% / D+ or lowerThe final grade is calculated as such: 25% - Midterm 50% - Final Exam 15% - Assignment 10% - Online Quizzes (Best 4 out of 5)

2. Identify the At Risk students Make a Check Point calculated column in the GradebookAfter first 3 weeks 2 Quizzes have been completed Use tracking data to see who has not logged in to the course site yet

Separate students into groupsNothing handed in, Mix of Poor Grades and Missing Assessments (between 20%-60%)

See sample image on next slide3. Contact the students offering help

See sample emails

7/13/1036



Take aways….

You have the power to change the life of a student

The New Diagnostics - Oct. 30, 2009 Inside Higher Ed

• “at-risk” students generally took that information as either a motivational kick in the rear or were prompted to quickly drop the class -- and were grateful in any case.”

• “A double-blind study conducted during the first two years of the Purdue's program, called Signals, revealed that 67 percent of students who learned they were in the middle- or high-risk categories were able to improve their grades."

7/13/1038




Please provide feedback for this session by [email protected].

The title of this session is:Identifying At-risk Students

carleton.ca/edc/atrisk

Presented by Ryan Kuhne

[email protected] Technology Consultant

Carleton University in Ottawa, Canada

7/13/1039

mailto:[email protected]



5 tips to over communicate, without seeming obvious in targeting at-risk students

1. Re-establish your/TA office hours. Send out an email to all students as a reminder.

2. In preparation for midterm or final exams, invite the students to submit topics for you to emphasize. Let them help you create the review day that focuses on their needs.

3. Consider incorporating technology into your teaching style. Start a blog, or a wiki as a way to interact with students outside of the classroom in a medium that they know best. Contact the Educational Development Centre for tips and one-on-one training on technologies.

4. Offer optional homework or practice exam questions that will help students get an understanding of what to expect.

5. Consider allowing students a re-write option for one assignment in an effort to improve a grade, using your feedback as a guideline. Most students, regardless of standing, would appreciate a second crack at it.

7/13/1040


5 goals you should strive for in this meeting1. Establish yourself as someone who not only cares about their success, but is

available to help them along the way.

2. Try to identify the source of the problem. Is the student struggling to comprehend material, or is it a problem outside of the classroom?

3. Have the student work out a plan of action. Don't force something down their throat, let them create a plan that is feasible and realistic to them. Help by providing perspective on what assignments are left, their weight, expectations, etc.This is a great tool to help students set some goals based on their grades. Grading Tool.

4. Be informative. Provide them with some referrals to other areas around campus that may be able to help. Counselling? Writing services? Mentoring? Find out how Carleton can help them.

5. Remind them that you're here. Encourage them to come back or contact you whenever they need help with course material. Show them a hand that they can reach out for if they need (metaphorically speaking...)

7/13/1041

Documents

Identifying At-risk Students