Each unit provides a report (either word processor or ... · Web viewAcademic Consulting participants in this class succeeded at a higher rate in CH 101 than their classmates with

Assessment Report, 2014-2015Unit: University Tutorial Center

Submitted by Barbie Windom, Megan Ryals, Brandy Grabow, and Nate Cline

SECTION I:

Unit/Department Outcome:

Students who consistently use SI will have increased success in their tutored course in comparison with non-attendees

Definition:What does X (outcome) look like? “Criteria” for learning the content?

Student success in a course is defined as follows: regular class attendance, being able to select important information, preparing for tests, learning to use resources appropriately, and ultimately performing well on class assignments.

Action Item/Course/significant activity if appropriate:

Fall and Spring SI sessions: sessions last 50 minutes and 3 sessions are offered weekly for select CH and PY courses.Beginning-of-semester training, weekly (for new) or bi-weekly (for returning) supervision of SI leaders and bi-weekly staffing meetings for all.

Method for Assessment Such as survey, focus group, interview, document analysis, etc

Multiple Linear Regression: Course Grade versus SI attendancesSATM and either HS GPA (fall) or Previous GPA (spring) used as predictor variables.

Population (Number): the large group that you want to know about such as “all freshman”

All students enrolled in SI supported sections of Chemistry and Physics courses.

Sample (Number and method): the group of freshman you actually asked to participate

All students enrolled in an SI supported section of CH 101, 201, 221, or PY 211. n = 3548

Response (Number): are only those that actually participated. Please provide a raw number and % of the total group asked to participate.

Fall: All students enrolled in an SI supported section of CH 101, 201, 221, or PY 211 who received a letter grade, except for students whose HS GPA or math SAT or ACT score was not available. n = 3011 (~85%)Spring: All students enrolled in an SI supported section of CH 101, 201, 221, or PY 211 who received a lettergrade, except for students whose previous GPA or math SAT or ACT score was not available. n = 1767 (~85%)

Implementation of method: such as web based or paper survey, number of focus groups conducted, time frame, method and number of contacts. etc

UTC data records were joined with information provided by Registration and Records.

A multiple regression analysis was run. The numerical course grade was the dependent variable. Dependent variables were the number of SI attendances, the student’s HS GPA, and SAT-Math score.

Summary of Results: Summary of only the important results- do not include all your data. Please include actual data such as percentages, raw numbers or themes for qualitative data.

FALL 2014Regression Statistics

Multiple R 0.462763R Square 0.21415Adjusted R 0.213366

SqStandard Error 1.042274Observations 3011

Coefficients St Error t Stat P-valueLower 95%

Upper 95%

Lower 95.0%

Upper 95.0%

Intercept -5.8063 0.307636 -18.87393.51E-

75 -6.4095 -5.2031 -6.4095 -5.2031SI Attendance 0.032532 0.005502 5.913185

3.73E-09 0.021745 0.04332 0.021745 0.04332

HS GPA 1.354643 0.074007 18.304274.81E-

71 1.209534 1.499753 1.209534 1.499753

SAT Math 0.005357 0.000294 18.240211.38E-

70 0.004782 0.005933 0.004782 0.005933

SPRING 2015

Regression StatisticsMultiple R 0.6335

R Square0.40132

2Adjusted R Square

0.400303

Standard Error

0.920835

Observations 1767

Coeffien

tsStandard

Error t Stat P valueLower 95%

Upper 95%

Intercept-

3.35473 0.216167

-15.519

25.19E-

51 -3.7787

-2.9307

6

Attendances0.03287

6 0.008553.8450

570.0001

250.0161

070.0496

46

Prev GPA1.04585

2 0.03598329.065

534.4E-

1520.9752

791.1164

25SATM 0.00378 0.000329 11.519 1.18E- 0.0031 0.0044

5 56 29 41 29

Interpretation: Your interpretation of results in general or of specific pieces of significant data (interpretations may be based on your professional experience, literature in your field, etc)

Using Fall data, the multiple regression model is Course Grade = 0.0325*(SI Attendances) + 1.355*(HS GPA) + .005(SAT Math) – 5.806This suggests that a student could expect an increase of .0325 in his grade for each SI attendance, or approximatelyhalf a letter grade for weekly attendance (15 per semester).Results for spring are similar.


In comparison with other students with similar first test grades, students who use Academic Consulting will succeed at a higher rate than non-participants (fall 2014)


Success is defined as an A, B, C, or S. Students who drop the course prior to the drop deadline are not factored into assessment.


Academic Consulting sessions consist of 3 meetings within 3-4 weeks. The first meeting is held after the first test. The final meeting is held after Test 2.


The percentage of “successes” will be compared for participants versus non-participants who also fail the first test.


Students enrolled in CH 101


All students who participated in Academic Consulting in Fall 2014 (n = 24). Participants came from two CH 101 classes.


15 students in class where Test 1 grades were available.

Summary of Results: Summary of only the important results- do not include all your data. Please

For the class with Test 1 grades available, of the 15 Academic Consulting participants, 3 dropped before the drop deadline. One student made a D and one student made an F. The rest (10/15 =

include actual data such as percentages, raw numbers or themes for qualitative data.

67%) made a C or better.Of the others in the class with first test grades below 70, 3 dropped before the drop deadline, 50 made a D, F, U, or W. 49 made a C or better (48%).One interesting note is that only one of the 15 participants received an F, while in the comparison group, 19/100 students received an F.Academic Consulting participants in this class succeeded at a higher rate in CH 101 than their classmates with similar first test grades. Failure rates were much lower for participants.



Sign-up (SU) tutors will develop their use of appropriate tutoring strategies.


Success with a particular strategy for the program means the majority of SU tutors are using this strategy appropriately and consistently and/or have made measurable improvement during the year.


USC 210, continued SU tutor training and supervision.


Observed and collected data during fall 2014 and spring 2015 semesters.Conducted portfolio reviews in May 2015.Portfolios were examined for themes common among multiple SU tutors. These themes included strengths/recent improvements as well as potential areas for growth. This assessment is primarily a document analysis but the documents were obtained either through observation or self-reflection.


34 SU tutors who worked either fall 2014 or spring 2015, or both


15 Spring 2015 SU tutors’ portfolios were reviewed. These 15 tutors worked in both fall 2014 and spring 2015. This provided information about the SU tutors’ growth across semesters.


See above.

Implementation of method: such as web based or paper survey, number of focus groups

Portfolios included:Rubric (developed in house) completed at end of observation and added to SU tutor portfolioSU tutors’ self-reflections at mid-semester and end of semesterSU tutors’ goal setting reports

conducted, time frame, method and number of contacts. etc

Student feedbackCoordinator feedback from mid-semester observation and debriefing meeting and informal observations throughout semester

Summary of Results: Summary of only the important results- do not include all your data. Please include actual data such as percentages, raw numbers or themes for qualitative data.

Strengths/Common areas of growth across semesters:- Patient, positive atmosphere that is comfortable and productive- Getting the student to write down important information and summaries- Self-confidence- Using positive reinforcement- Incorporating student’s resources- Talk ratio- Avoiding using rhetorical questions (i.e. We know F=ma, right?)

Areas of improvement: - Assertiveness to hold students accountable- Closure (associated with time management)- Consistently meeting goals (some tutors revert to old habits)- Test prediction and review

We have added a lesson in USC 210 about test prediction and review in the last year so that new SU tutors are not hearing about this process for the first time in SU training. Test-taking skills (prediction and review) are becoming part of the “culture” of helping students become independent learners.

SU tutors may still feel uncomfortable holding students accountable for fear that it can be negatively perceived by the student. They also are challenged to use the last 5 minutes of the session for closure because they feel pressured that the student wants to ask “one last quick question”.

Some returning tutors lose focus on goals for improvement after previously demonstrating them during an observation. They revert to former tutoring habits.


SECTION II: Other Data Collected for Decisions and Reporting

Overview of UTC Usage

Summer I 2014

Summer II 2014*

Fall 2014

Spring 2015

Total**

Total number of unique student visitors

184 221 1,929 828 2,786

Total contact hours served

955.9 967.9 11,939.5 8,275.1 22,138.4

* Includes mandatory intensive tutoring for 27 ASPSA students for HI/SOC and COM/SOC in summer START. Contact hours for ASPSA START students was mandatory tutoring 12.5 hours/week during 2nd summer session.

**Total does not equal the sum of the semesters since some students used the UTC in more than one semester

Students served Fall 2014 Spring 2015

Number

Percentage of students

Percentage of contacts Number

Percentage of students

Percentage of contacts

Total Number of enrolled undergrads in CH, MA, PY courses we serve 8,873

100.00% 5,284

100.00%

Total Number of unique undergrads who submitted tutoring by appointment application 1,379 15.54% 737 13.95%Total Number of undergrads requesting SU 1,053 11.87% 76.36% 520 9.84% 70.56%Total Number of undergrads requesting (ind. + group) WA 585 6.59% 42.42% 386 7.31% 52.37%

Orientations for Tutoring by Appointment

Fall 2014 Spring 2015

Number of orientation offerings 51 total41 large + 10

small

49 total34 large + 15

smallNumber of students receiving

orientations1,113 total

1,099 large + 14 small

357 total331 large + 26

smallAverage number of students/orientation

26.8 – large1.4 - small

9.7 – large1.7 - small

Breakdown of Total (ind. + group) WA Usage by SubjectFall 2014 Spring 2015

Requests Received

Requests Assigned

Requests Unassigned

% of Students’ (ind. + group) WA Requests Served*Fall 2014 Spring 2015

ChemistryWA tutor assigned

WA tutor not assigned, used

SU insteadNot served in

either program

MathWA tutor assigned



either program

PhysicsWA tutor assigned



either program

*The total requests received does not equal the total students requesting (ind. + group) WA since some students submitted more than one request.

Tutoring by Appointment

Fall 2014 Spring 2015

Total # of requests(individual + group)WA

+ SU

2,036 1,055

# of (ind. + group) WA requests

614 408

% WA requests/total requests

30.2% 38.7%

# of individual WA tutors 77 86# of group WA tutors 15 14

# of individual WA assignments

206 221

# of group WA assignments

138 78

Total # of assignments 344 299% of (ind.+group)WA

requests filled56.0% 73.3%

% of students whose (ind. + group) WA

request was not filled but used SU instead

24.2% 23.9%

# of Group contact hours 2377.75 1230.25Avg # of students per

group4.37 3.79

# of ind. WA hours tutored

3543.5 3,631.3

Avg. # of ind. WA hours / student

17.6 16.9

# of SU tutors 26 24# of available SU hours 1,311.5 1,013.25

# of unique student visitors

394 213

# of scheduled SU hours 1144.5 (907 hr, 475 ½

hr)

691 (606 hr, 170 ½ hr)

% of scheduled SU appointments (hrs

scheduled/hrs available)

87.3% 68.2%

# of attended (used) SU hours

1087 636

% of used SU appointments that were

scheduled (hrs attended/hrs scheduled)

94.9% 92.0%

% of used SU appointments that were

available (hrs attended/hrs available)

82.8% 62.8%

# of SU missed appointments

43 hr, 34 ½ hr

44 hr, 10 ½ hr

Turn-away tally in sign-up:Week of # of

cancelled appointments

# of filled cancellations

# of students turned away (no appts. current week)

# of students turned away (no appts. following week)

9/8/14 - 9/12/14 14 8 9 09/15/14 – 9/19/14 4 0 6 09/22/14 – 9/26/14 11 2 2 19/29/14 – 10/3/14 (week before fall break) 11 5 26 010/6/14 – 10/10/14 (fall break) 2 1 0 010/13/14 – 10/17/14 13 6 0 010/20/14 – 10/24/14 19 9 7 010/27/14 – 10/31/14 16 10 2 011/3/14 – 11/7/14 8 1 0 0Appointments were available for all requests during 11/10/14 – 12/3/14

Observation/Conclusion:

The number of unique student visitors served by the UTC dropped by ~100 students this year, while the number of contact hours served rose by ~4,000 hours. This means that students who visited the UTC used more hours of support/per person then in 2013-2014.

The number of students seeking tutoring by appointment support continues to rise significantly every fall semester. There were ~150 more requests in fall 2014 than 2015.

We received a total of 614 applications for tutoring by appointment in fall 2014 compared to 562 applications in fall 2013 and 513 applications in fall 2012. This is an increase in 100 tutoring by appointment applications received in just a two year period with no increase in tutoring by appointment resources. There are a lot of student requests that go unmet with our current resources, particularly in fall semesters.

In both fall 2014 and spring 2015 semesters, students requesting tutoring by appointment for physics courses made up the largest group of students we were unable to serve. Action/Decision:

To accommodate for the increase in tutoring by appointment applications, four sections of USC 210, our tutoring training program, will be offered in fall 2015 compared to three sections historically offered. This additional class will allow us to hire and train more tutors to meet the increased demand for tutoring by appointment requests. Additionally, each tutor in their second semester and beyond will be encouraged to take on at least 3 students compared to 2 students that has been suggested in the past.

Using grade information obtained from Registration & Records, we will recruit students who have excelled in physics courses we tutor so that we can provide more assistance to students requesting tutoring by appointment in physics courses.

Continue to increase number of orientations offered per day during peak times of the semester to alleviate room capacity issues.

SI Attendance for Spring 2015 by Course Enrollm

ent#

Attending

Total Attendan

ces

Sessions

Offered

% Attendi

ng

Avg Att Per

User

Avg Session Size

CH 101 957 232 1011 133 24.2 4.36 7.60CH 201 464 67 198 66 14.4 2.96 3.00

SI Attendance for Summer I 2014 by Course Enrollm

ent#

Attending

Total Attendan

ces

Sessions

Offered

% Attendi

ng

Avg Att Per

User

Avg Session Size

CH 101 79 38 218 17 48.1 5.73 12.82CH 201 100 26 62 18 26.0 2.38 3.44CH 221 140 54 249 15 38.6 4.61 16.60CH 223 102 38 218 17 37.3 5.74 12.82TOTAL 421 156 747 67 37.1 4.79 11.15SI Attendance for Summer II 2014 by Course Enrollm

ent#

Attending

Total Attendan

ces

Sessions

Offered

% Attendi

ng

Avg Att Per

User

Avg Session Size

CH 111 40 22 111 19 55.0 5.05 5.84CH 101 69 29 134 19 42.0 4.62 7.05CH 201 63 15 57 17 23.8 3.80 3.35CH 221 82 16 96 13 19.5 6.0 7.38CH 223 115 34 81 11 29.6 2.38 7.36TOTAL 369 116 479 79 31.4 4.13 6.06

SI Attendance for Fall 2014 by Course Enrollm

ent#

Attending

Total Attendan

ces

Sessions

Offered

% Attendi

ng

Avg Att Per

User

Avg Session Size

CH 101 2283 617 2361 261 27.0 3.83 9.05CH 201 219 45 107 36 20.5 2.38 2.97CH 221 709 268 1492 110 37.8 5.57 13.56PY 211 482 99 237 33 20.5 2.39 7.18TOTAL 3693 1029 4197 440 27.9 4.08 9.54

CH 221 720 218 764 99 30.3 3.50 7.72PY 211 314 71 183 34 20.8 2.58 5.38PY 212 331 26 65 31 7.9 2.50 2.10TOTAL 2786 614 2221 363 22.0 3.62 6.12

ACADEMIC CONSULTINGAcademic Consulting was offered to two CH 101 sections in the fall (Petrovich and Maggard) and spring (Petrovich and Bowden) semesters.

Total number of students served: Fall: 24 Spring: 22Total number of appointments: Fall: 64, Spring: 48Number of consultants: Fall: 12, Spring: 10

WSTS: Total Visits in Summer Session II, 2014Appointments (Park

Shops)Drop-In (FYC Commons)

# of START users

9 8

# of total users 24 11# of START

visits11 11

# of total visits 38 16Avg. visits/user 1.58 1.45

# of hours available

105 104.5

Efficiency (# hours used/#

hours available)

19.19% 10.29%

# of consultants on staff

2 2

WSTS: Total Visits to all Locations/FormatsFall 2014 Spring 2015

# of users 375 275

# of visits 726 485Avg. visits/user 1.93 1.76

# of hours available

826 544.5


hours available)

46.3% 50.2%

# of consultants on staff

12 9

WSTS: Park Shops AppointmentFall 2014 Spring 2015

# of users 315 204# of visits 578 376

Avg. visits/user 1.83 1.84# of hours available

556 409.5


hours available)

54.5% 52.4%

WSTS: Park Shops Drop-inFall 2014 Spring 2015



100 94


hours available)

32.3% 41.8%

WSTS: Park Shops Online Writing CenterFall 2014 Spring 2015



49 41


hours available)

31.3% 47.5%

WSTS: First Year College Commons (drop-in sponsored by Housing)

Fall 2014 Spring 2015# of users 24 N/A# of visits 33

Avg. visits/user 1.37# of hours available

72


hours available)

20.6%

WSTS: Lee Residence Hall (drop-in sponsored by Housing)Fall 2014 Spring 2015

# of users 14 N/A# of visits 17

Avg. visits/user 1.21# of hours available

46


hours available)

23.2%

Graduate Writing Center: Total – (appointments)Fall 2014 Spring 2015



339.5 357.5


hours available)

73.7% 49.8%

# of consultants 6 7

Observation/Conclusions: The Avent Ferry satellite center was closed in fall 2015 due to a change in the accessibility to the space for the campus community. The consultant working there added her hours to the Park Shops appointment schedule. The result was that WSTS was open from 10am – 7pm on Monday and Wednesday.

The Graduate Writing Center nearly doubled the number of visits and unique users from the 2014-2015 academic year!

The first Long Night Against Procrastination (LNAP) event with the library was held 4/9/15. The partnership with the DH Hill librarians was used in the following ways - 16 students sought out help with finding resources from research librarians. 8 students sought out help from WSTS consultants (7 of those students were using WSTS services for the first time).

Due to conflicts with consultant schedules, FYC and Lee Hall satellite drop-in sites were not staffed during spring 2015. Despite offering less satellite support in 2014-2015, overall WSTS usage was higher than the previous year.

Piloted YouCanBook.me scheduling software for WSTS after spring break to address the scheduling challenges with the Google appointment system. The system worked very well for the program’s needs.

Action/Decision: Due to poor attendance and inaccessibility for the wider campus community the Avent Ferry satellite location will remain closed and evening appointments hours will be continued in Park Shops.

The UTC developed a partnership with the English Department to offer a graduate assistant position for 2015-2017. The GA will consultant in the Graduate Writing Center and develop administrative skills by supporting the writing programs.

We will remain in contact with librarians at D.H. Hill to continue to host the LNAP. We will consider returning to staff FYC and Lee Hall WSTS evening drop-in services after further examining staffing availability and needs in Park Shops where efficiency rates are significantly higher.

Tutor Hiring

Tutor Training Semester

Number of students enrolled

in USC 210

Returned in a

subsequent

semester

Not rehired(either grade

in course, poor

performance, or GPA < 3.0)

Spring 2014 48 27 7

(3 sections + WSTS) (56.3%)Summer 2014 11

(1 section)9 (81.2%) 0

Fall 2014 50(3 sections + WSTS)

39 (78%) 1

Tutor CRLA Certification Level: 2014-2015 academic

year Level I (10 training hours, 25 tutoring hours)

51

Level II (20 training hours, 50 tutoring hours)

82

Level III (30 training hours, 75 tutoring hours)

60

Total Certifications: 193Additional Hiring Information:Summer START 2014 (8 tutors for ASPSA students, 2 CH, 2 peer supervisors, 4 WSTS)

16

Tutor Development Comments from graduating tutors:

“Tutoring has been a phenomenal experience that I was fortunate to learn about early on in my undergraduate studies. I’ve become much more comfortable talking with unfamiliar faces and overall feel more confident. It has been a rewarding experience that has kept me on top of the fundamentals. The students and the staff at the UTC are all so friendly and easy to talk to, which makes the experience that much more incredible.” – Hayden Brochu

“I have become more assertive when expressing my ideas, and I think much of that is owed to my UTC experience. I have also picked up a lot of useful devices for teaching math and physics concepts. These skills will be tremendously useful as I go on to graduate school in physics, seeking a Ph.D. and hoping to land a professorship one day. I feel strongly that my work at the UTC is an important first step in realizing my dream of teaching physics at the university level.” – Jacob Robbins

“The UTC has been an excellent way for me to hone my communication skills because each unique student processes information in a different way. You have to find a way to connect with each student’s learning style and it’s so rewarding to watch them succeed. Also keeping math and chemistry fresh helped me do very well on my PCAT, so I will be attending UNC’s Pharmacy School this fall. Sitting with students one-on-one also helped my confidence in my interviews, and they noted my ability to juggle school work with a job. I would highly recommend tutoring to anyone with a passion for helping people find their own success.” – Sarah Shockley

“As an education major, I feel that working at the UTC has helped me tremendously when it comes to teaching students how to write, specifically those from an ESL background. I was able to develop more patience with people and myself. It was incredibly challenging at first to sit in silence while a student thinks about the revisions they needed to make, but it was so rewarding. Having patience is such a valuable skill to have, not only in the field of education, but in any career. Working at the UTC also served as a really great way to learn how to communicate with people. It taught you how to use a professional voice with others, which has definitely helped me when having discussions with my own students. Overall, working at the UTC was a great experience and I wish I could have worked there longer!” –- Brooklyn Haas

"My time as a consultant in the Writing Center allowed me to expand my comfort zone and significantly enhance my ability to actively listen and communicate with those that I helped. Spending thirty minutes to an hour with an individual and learning as much as I could about that individual's topic and paper and then shifting gears completely for the very next student was certainly challenging, but it was also very beneficial in that it developed my ability to listen carefully and respond thoughtfully." – Frank Jefferson

“I worked for the UTC for 6 semesters of my undergraduate career. Throughout my experience, I greatly improved in my ability to communicate with people from a variety of backgrounds and needs. Learning the importance of wait time was also invaluable- once I stepped back and allowed the student to talk, I realized the students got so much more out of the tutoring relationship. I feel that I can apply these skills to other areas of my life, both in and outside of academia- and in fact I already have. I will be attending pharmacy school at Virginia Commonwealth University starting in Fall 2015. As a pharmacist, I will be required to talk to people from a variety of backgrounds, and tutoring has directly enhanced these skills.” – Clare Crosland

Tutor Training Products:

Tutor Training Product # of items sold to Tutoring Centers at other institutions(between 5/1/14 - 4/30/15)

A Look at Productive Group Learning Sessions (DVD)

9

A Look at Productive Tutoring Techniques (DVD)

19

Users’ Guide (4th edition) 17Put the Pencil Down (student book) 466Teacher’s Edition, Put the Pencil Down 52# of institutions 88

Observations: The 4th edition of the Users’ Guide was developed and marketed in fall 2014. Current UTC staffing demands provide little time for new product development. With Marcia’s departure, the UTC staff did not have opportunity to travel to any conferences to promote the materials in 2014-2015, yet there was not much significant drop in product sales compared to 2013-2014.

Action/Decision: Continue to market current products. Explore closed captioning the DVD products due to increase interest in this feature.

Documents

Each unit provides a report (either word processor or ... · Web viewAcademic Consulting participants in this class succeeded at a higher rate in CH 101 than their classmates with