1

An Approach to Maintaining Academic Integrity In A Distance Learning Environment

Dr. Robert T. Kitahara Dr. Frederick WestfallAssistant Professor, Business Regional Chair, Business ProgramsTroy University – Florida Region Troy University – Florida Region

Dr. Sallie J. JohnsonProfessor of Distance LearningAir Command & Staff College, Maxwell AFB

2

“Never underestimate the joy people derive from hearing something they already know.”

Enrico Fermi (1901-1954)

3

AGENDA

. University Standards of Conduct

. Case study of recent experiences - quantitative courses delivered in a DL format

. Literature survey – Extent of the Problem

. One Approach – Troy University

. Conclusions/Observations

. Plan for future research

4

……“A student is subject to disciplinary action if:

……In connection with the taking of, or in contemplation of the taking of any examination by any person:

a) A student knowingly discovers or attempts to discover the contents of an examination before the contents are revealed by the instructor;

b) A student obtains, uses, attempts to obtain or use, or supplies or attempts to supply to any person, any unauthorized material or device;

c) A student uses, attempts to use, or supplies or attempts to supply to any person any material or device dishonestly.

Excerpts from Troy UniversityUndergraduate Catalog – Standards of Conduct

5

…...Penalties for Misconduct:

…...Any student who has committed an act of misconduct

……may be subject to one or more of the following penalties:

a) A student’s grade in the course or on the examination affected by the misconduct may be reduced to any extent, including a reduction to failure.

b) A student may be suspended from the University for a specific or an indefinite period, the suspension to begin at any time.”

6

…. “(b) Violations of the Academic Honor Code. …. During examinations, violations of the Academic Honor Code

shall include referring to information not specifically condoned by the instructor. It shall further include receiving information from a fellow student or another unauthorized source.

…. Violations of the Academic Honor Code shall include obtaining, distributing, or referring to a copy of an examination which the instructor/department has not authorized to be made available for such purpose.

…. Violations of the Academic Honor Code shall include assisting, attempting to assist, or conspiring to assist another student in committing the offenses as outlined above.

…. Violations of the Academic Honor Code shall include attempting to commit any offense as outlined above.”

Excerpts from Florida State UniversityStudent Handbook – Codes and Policies

7

“…. Listed below are the disciplinary offenses actionable by the University.

(1) Scholastic Dishonesty. Scholastic dishonesty means plagiarizing; cheating on assignments or examinations; engaging in unauthorized collaboration on academic work; taking, acquiring, or using test materials without faculty permission; submitting false or incomplete records of academic achievement; acting alone or in cooperation with another to falsify records or to obtain dishonestly grades, honors, awards, or professional endorsement; or altering, forging, or misusing a University academic record; or fabricating or falsifying of data, research procedures, or data analysis.”

Excerpts from University of MinnesotaBoard of Regents Policy – Student Conduct Code

8

“…. The following actions constitute misconduct for which students may be subject to administrative action or disciplinary penalties.

…. Dishonesty in connection with any University activity. Cheating, plagiarism, or knowingly furnishing false information to the University are examples of dishonesty. The commitment of the acts of cheating, lying, stealing, and deceit in any of their diverse forms (such as the use of ghost-written papers, the use of substitutes for taking examinations, the use of illegal cribs, plagiarism, and copying during examinations) is dishonest and must not be tolerated. Moreover, knowingly to aid and abet, directly or indirectly, other parties in committing dishonest acts is in itself dishonest.”

Excerpts from Purdue University Student Conduct Code

9

“….THE FOLLOWING ACTS SUBJECT STUDENTS TO DISCIPLINARY ACTION:

…. Engaging in any form of academic dishonesty, such as plagiarism (representing the work or ideas of others as one's own without giving proper acknowledgment), cheating (e.g., copying the work of another person, falsifying laboratory data, sabotaging the work of others), and other acts generally understood to be dishonest by faculty or students in an academic context.”

Excerpts from Santa Clara UniversityStudent Conduct Code

10

QM3341 – Business Statistics IIMGT3373 – Operations Management

Course Requirements:

. 7 online quizzes – 1 hour time limit (Typically students take 30-40 minutes)

. Online Final Examination – 2 hour time limit (Typically students take 60-90 minutes)

. Proctored Examination – 3 hour time limit - Administrative controls - Distance Learning Office must pre-approve proctors - Exam - open-book, open-notes

A Case Study

11

Stating the “Obvious”

Examination copies, instructor materials and author test banks are not intended for general consumption particularly by students

Examinations are intended to assess the individual’s command of the subject matter, not the extent or accuracy of his/her data files or work of others

12

Troy University, its instructors, delivery system designers (Blackboard) and textbook publishers cooperatively implement several strategies:

. Blackboard controls designed to prevent students from printing copies of exams

. Blackboard controls force completion of exams once entered

. Instructions with each exam: - Students must take assessments separately - Students not allowed to make copies of exams - Proctors must return all test question & exam sheets

. Publishers screen applicants for instructor materials

. Publishers screen applicants for author test banks

13

Provide, prior to Proctored Examination, questions representative of type that will appear on exam

For Students’ Benefit

For Instructor BenefitInserted “code” each test bank question to facilitatecorrelation to student inquiries

14

Table 1a Summary of Quiz Timing Irregularities (QM3341 Business Statistics II – Term 2/05)

Timings (Minutes:Seconds) and Scores

QM3341 Quiz 1 Quiz 2 Quiz 3 Quiz 4 Quiz 5 Quiz 6 PE Quiz 8 Final

Student 1 2:19100

2:03100

3:53100

33:2495

36:1175

8:4695

56/0* 17:07100

35:3696

Student 2 1:57100

3:3090

3:02100

26:1495

24:4995

12:5990

56/0* 21:1095

37:3792

Student 3 2:57100

2:2595

1:47100

25:5020

22:5565

23:2585

56/0* 11:1595

30:3994

Student 4 2:01100

3:25100

5:37100

Lock 39:2390

42:5785

32:0290

56/0* 38:4840

46:4894

Student 5 1:47100

2:05100

1:53100

53:5390

56:0585

35:5690

41/0* 9:0595

73:3860

Student 6 1:38100

1:51100

1:47100

48:0415

71:1195

31:3290

41/0* 3:56100

46:3994

* All 6 students from same site (*Administratively changed to 0)

15

. Sent e-mail inquiries few responded.

. One student claimed timings were “reasonable”

. No student admitted to possessing unapproved sources of information

. For quiz 4 - shuffled “codes” timings for all 6 immediately jumped to historical levels

. For remaining quizzes, removed codes timings remaining generally at historical averages

Response to Timing Irregularities

16

Table 1b Summary of Quiz Timing Irregularities (QM3341 Business Statistics II – Term 2/05)

Timings (Minutes:Seconds) and Scores

QM3341 Quiz 1 Quiz 2 Quiz 3 Quiz 4 Quiz 5 Quiz 6 PE Quiz 8 Final

Student 1 2:19100

2:03100

3:53100

33:2495

36:1175

8:4695

56/0* 17:07100

35:3696

Student 2 1:57100

3:3090

3:02100

26:1495

24:4995

12:5990

56/0* 21:1095

37:3792

Student 3 2:57100

2:2595

1:47100

25:5020

22:5565

23:2585

56/0* 11:1595

30:3994

Student 4 2:01100

3:25100

5:37100

Lock 39:2390

42:5785

32:0290

56/0* 38:4840

46:4894

Student 5 1:47100

2:05100

1:53100

53:5390

56:0585

35:5690

41/0* 9:0595

73:3860

Student 6 1:38100

1:51100

1:47100

48:0415

71:1195

31:3290

41/0* 3:56100

46:3994

(*Administratively changed to 0)

17

. 2 Parts: - Part 1: 25 multiple choice questions - Part 2: 5 quantitative problems

. Open-book, open-notes

. All questions for both parts different from previous terms

. Part 2 – show all work

The Proctored Exam

18

. Arranged to take their PE’s from business professor

. Asked to take the examination together - “common notes” . Denied request

. Offered to make copy of notes for each to have

. Students never returned to take exam

. Went to another university official to serve as proctor (5 of the 6 students used the same proctor)

Students 1 and 2 (QM3341)

19

Table 2 Proctored Exam Irregularities (QM 3341 Business Statistics II – Term 2/05)

QM3341AnswerSheets

Group A(Students 1,2,3,4)

Group B(Students 5,6)

Rest of Class Historically “similar test” -same format, same authors’ test bank

Part 1 25 Multiple Choice

- Same 2 Errors- Same 2 Erroneous Choices

- Same 2 Errors(Same as Group A)

- No student missed both questions as A,B- No other pair had identical MC sheets

No pair had identical sheets

Part 2

5 Quantitative Problems

Virtually identical:Same:- Answers- Layout- Detail- Errors- Omissions- Inclusions- Decimal place rounding**- Wording (90%)

Virtually identical:Same:- Answers- Layout- Detail- Errors- Omissions- Inclusions- Decimal place rounding- Wording (90%)

- None matched A,B- No other pair had matching answer sheets

No pair had matching answer sheets

** Decimal accuracy to 6 decimal places

20

Table 1c Summary of Quiz Timing Irregularities (QM3341 Business Statistics II – Term 2/05)

Timings (Minutes:Seconds) and Scores

QM3341 Quiz 1 Quiz 2 Quiz 3 Quiz 4 Quiz 5 Quiz 6 PE Quiz 8 Final

Student 1 2:19100

2:03100

3:53100

33:2495

36:1175

8:4695

56/0* 17:07100

35:3696

Student 2 1:57100

3:3090

3:02100

26:1495

24:4995

12:5990

56/0* 21:1095

37:3792

Student 3 2:57100

2:2595

1:47100

25:5020

22:5565

23:2585

56/0* 11:1595

30:3994

Student 4 2:01100

3:25100

5:37100

L 39:2390

42:5785

32:0290

56/0* 38:4840

46:4894

Student 5 1:47100

2:05100

1:53100

53:5390

56:0585

35:5690

41/0* 9:0595

73:3860

Student 6 1:38100

1:51100

1:47100

48:0415

71:1195

31:3290

41/0* 3:56100

46:3994

(*Administratively changed to 0)

21

Table 1c Summary of Quiz Timing Irregularities (QM3341 Business Statistics II – Term 2/05)

Timings (Minutes:Seconds) and Scores

QM3341 Quiz 1 Quiz 2 Quiz 3 Quiz 4 Quiz 5 Quiz 6 PE Quiz 8 Final

Student 1 2:19100

2:03100

3:53100

33:2495

36:1175

8:4695

56/0* 17:07100

35:3696

Student 2 1:57100

3:3090

3:02100

26:1495

24:4995

12:5990

56/0* 21:1095

37:3792

Student 3 2:57100

2:2595

1:47100

25:5020

22:5565

23:2585

56/0* 11:1595

30:3994

Student 4 2:01100

3:25100

5:37100

L 39:2390

42:5785

32:0290

56/0* 38:4840

46:4894

Student 5 1:47100

2:05100

1:53100

53:5390

56:0585

35:5690

41/0* 9:0595

73:3860

Student 6 1:38100

1:51100

1:47100

48:0415

71:1195

31:3290

41/0* 3:56100

46:3994

(*Administratively changed to 0)

22

. Only 3 of the students responded

. Vehemently denied “illegal” or wrongful activity

. 2 immediately threatened lawsuits

Response to Proctored Exam Irregularities

. Behavior in that course similar

. Identical answer sheets for Part 1

. Virtually-identical answer sheets Part 2 . Same denials

. Threats of lawsuits

Students 1 & 2 also taking MGT3373 Operations Management

23

Table 3 Proctored Exam Irregularities (MGT3373 Operations Management - Term T2/05)

MGT3373AnswerSheets

2 Students (1,2) Rest of Class Historically “similar test” -same format, same authors’ test bank

Part 1 25 Multiple Choice

- Same 2 Errors- Same 2 Erroneous Choices* Same 2 students showed similar behavior in QM3341

- No student sheet matched 1,2- No other pair had identical MC sheets

No pair had identical sheets

Part 2

5 Short Essay Questions(4/5 Open Ended)

Virtually identicalSame:- Answers- Detail- Omissions- Inclusions- Examples- Wording – 90%* Same 2 students showed similar behavior in QM3341

- None matched 1,2- No other pair had matching answer sheets

No pair had matching answer sheets

24

. Students 1 & 2 showed same behavior QM2241 Business Statistics I (Term 1/05)

. 1-hour quizzes in 2-3 minutes

. 2-hour final exam in 5 minutes

. Identical answer sheets for Part 1 of PE

. Identical answer sheets for Part 2 of PE

Relevant Aside

25

. Initial focus = unreasonable assessment timings

. Concern over “open-notes” policy

. Students offered grades based on “items submitted”

. Upon further review = degree of similarity, particularly on quantitative & essay portions, not be explained by “common notes”

. Final grades reassigned as failing

. Apprised of their rights to appeal

Administrative Actions

26

Table 1d Summary of Quiz Timing Irregularities (QM3341 Business Statistics II – Term 2/05)

Timings (Minutes:Seconds) and Scores

QM3341 Quiz 1 Quiz 2 Quiz 3 Quiz 4 Quiz 5 Quiz 6 PE Quiz 8 Final

Student 1 2:19100

2:03100

3:53100

33:2495

36:1175

8:4695

56/0* 17:07100

35:3696

Student 2 1:57100

3:3090

3:02100

26:1495

24:4995

12:5990

56/0* 21:1095

37:3792

Student 3 2:57100

2:2595

1:47100

25:5020

22:5565

23:2585

56/0* 11:1595

30:3994

Student 4 2:01100

3:25100

5:37100

Lock 39:2390

42:5785

32:0290

56/0* 38:4840

46:4894

Student 5 1:47100

2:05100

1:53100

53:5390

56:0585

35:5690

41/0* 9:0595

73:3860

Student 6 1:38100

1:51100

1:47100

48:0415

71:1195

31:3290

41/0* 3:56100

46:3994

Students 1,2 also received 0 on PE for MGT3373 (*Administratively changed to 0)

27

. 2 students admitted to possessing unapproved materials

. One claimed they had copies of all past exams

. Another admitted they had textbook test bank - found on an open website

. Included these in their “notes” “did nothing wrong”

. None of 6 students responded to issue of high degree of similarity of answer sheets

Early Stages of Appeals

28

Final Appeals Resolution

. 5 Students dropped their appeals given the evidence - Accepted their failing grades

. 1 Student persisted. This student took his appeal to: - Several campus deans - Dean of the School of Business - Senior Vice Chancellor for Student Affairs - University Chancellor

. Could not explain the answer sheet “similarities”

. Decision to assign a failing grade upheld

. 8 failing grades held for all 8 cases

29

What It Cost !!. 5+ Months of Aggravation. Personally = 200 Man Hours. Regional Administration & Support Staff = 50 Man Hours. Main Campus Administration & Support = 30 Man Hours. Main Campus Deans & Leadership = 20 Man Hours

What It Entailed. Constant threats of lawsuits. Threats of personal lawsuits. Persistent badgering and lies by students. Faced several “flip-flops” in administrative decisions. Pressure to “let it go”

Is it any wonder that many/most students go unchallenged?

30

Post Script. In Term 4/06 several of the offending students repeated the courses with me again

. In QM3341 two of those students (Students 5 & 6) repeated the same acts!

. Failing grades were assigned to both students, once again for cheating on the Proctored Examination

. Student 5 admitted to the acts and cooperated in the investigation of the case.

. Student 5 verified ALL suspicions in Terms 2/05 and 4/06

. Student 5 was allowed to receive a grade of D (reduced from a C) based upon submitted work

31

Cheating is on the Rise

. At least 50% of students cheat from middle school through college

. 75% of high school students admit to cheating on exams and papers

. 50% of undergraduates admitted to cheating more than once

. On most campuses 75% of students admit to some form of cheating

. 41% of students believe plagiarism occurs “often”

. Surveys show incidence rates on the rise

. 40% incidence rate for graduate students

The anecdotal and statistical evidence that dishonesty in thebusiness world is also growing.

Literature – Extent of the Problem

32

Traditional methods:. Copying from other students. Crib notes. Writing on palms of hands. Whispering answers. Tapping Morse Code. Stealing exam copies

Newer methods:. Cell phones. PDA devices. Programmable calculators. Internet connected laptops. Mobile internet devices.. Hand-held Blackberry units. Tape recorders. Small video cameras. Digital media (CD, DVD, flash drives, mini hard drives,iPods)

Evolution of Methods of Cheating

Instructors and administrators must keep pace.

33

Disturbing Trends in Cheating

. Nationwide most forms at record levels

. Underclassmen cheat more than upperclassmen

. More prominent among fraternity, sorority members & athletes . Those who perceive others aren’t caught likely to cheat

. Younger students cheat more often

. Some surveys showed: 80% of faculty observed some form of serious cheating More than 30% did nothing about it

. Less cheating occurs at universities with strong Honor Codes

34

A Culture of Cheating?. . Students perceive the practice is (more) acceptable

. “Everybody is doing it”

. Hard-core cheaters not concerned with long-term consequences

. Diminishing social stigma

. Cheaters shun accountability (blame teachers, parents)

. Society becoming increasingly tolerant

35

-   Affirm the importance of academic integrity. -   Foster love of learning. -   Treat students as ends in themselves. -   Promote an environment of trust in the classroom. -   Encourage student responsibility for academic integrity. -   Clarify expectations for students. -   Develop fair and relevant forms of assessment. -   Reduce opportunities to engage in academic dishonesty. -   Challenge academic dishonesty when it occurs.- Help define and support campus-wide academic integrity standards

Establishing a Climate of Honesty & Integrity

Faculty and administrators must be concerned and involved.

36

How do you do that in a Distance Learning Environment?

37

Securexam Remote ProctorTM (Prototype)

Will take the place of a human proctor.

One Approach – Troy University

38

39

“Indeed your class attendance gives new meaningto the term ‘Distance Learning’”

40

. Fingerprint scanner and student verification system

. 360 degree field-of-view camera

. Omni directional microphone

. Remote recording of real-time audio and video

. Integrated motion detection software to detect and flag suspicious activity. Specialized exam software to disable unwanted, student computer functions. Connection via USB port. Onsite hardware does not contain student information. Permits use by different users. Target cost $100

Securexam Remote ProctorTM Features

41

What do some people think? Comments found on the web…And now electronic proctors? As Bob Dylan said, "the times they are a‘ changin."

I think this is a fantastic development. I have been concerned about security in distance learning environments for a while… Any kind of testing process is useless unless security and integrity can be attained. I certainly hope this effort is successful. Peter, Southwestern Community College, Sylva, NCYes, I can see it now. Just like the Italian Job, or Oceans 11, or whatever. The student prerecords him or herself dutifully typing, then broadcasts it directly into the device, complete with fake time stamp. I would hate to think our professors would easily adopt the tactics of the CIA or NSA. Ken

I am fascinated by this. …setting aside the (lack of) effectiveness of the Web cam as a proctor, I cannot imagine students would want to have their fingerprints on file and used as an exam verification tool. Also, isn't it a *whole lot cheaper* to hire one or even more real proctors that to set up $200 devices at each desk?To me, this seems like a solution in search of a problem. Touro University

Just sounds like a lot of headache.  I think the honor code should work just fine. Samford University

42

Table 4a Troy University Practices to Maintain Academic Integrity  Prior to 2007 Post 2007

Academic Code    

- Definition of violations Well published Well published

- Penalties/consequences Well published Well published

- Procedures Well published Well published

Student Honor Code Well published Well published

Proctored Exams Human Proctor Remote Proctor

Addressing Common Problems    

- Student verification Picture ID Fingerprint scan

- Copying others work Scrutiny by proctor Remote Proctor

- Receiving assistance from others Scrutiny by proctor Remote Proctor

- Using unapproved materials Scrutiny by proctor Remote Proctor

- Using unapproved crib notes, electronic devices, storage media

Scrutiny by proctor Remote Proctor

- Helping others commit illicit acts Scrutiny by proctor Remote Proctor

- Detecting suspicious behavior Scrutiny by proctor Remote Proctor

- Collusion Post exam analysis Post exam analysis

43

Table 4b Troy University Practices to Maintain Academic Integrity  Prior to 2007 Post 2007

Course/Instruction Design Instructor discretion Committees of experts

Redesign committees Course templates

Controlling the Exam Environment Proctored Exams (Currently optional)

Remote Proctor

Selectable Blackboard Options    

- Large test banks Instructor discretion Instructor discretion

- Randomized tests Instructor discretion Instructor discretion

- Force Exam completion Instructor discretion Instructor discretion

- Presentation of questions Instructor discretion Instructor discretion

- Multiple or single attempts Instructor discretion Instructor discretion

Detection Tools/Statistics    

- Record of student accesses Provided by Blackboard Provided by Blackboard

- Record of time spent Provided by Blackboard Provided by Blackboard

- Record of student postings to various components

Provided by Blackboard Provided by Blackboard

- Record of communications via Digital Drop Box

Provided by Blackboard Provided by Blackboard

44

- New tools & resources to obtain unethical edge

- Job of University, instructors, course delivery system designers & publishers much more difficult

- More apt to threaten lawyers and lawsuits

- Believe university will ultimately back down

- Requires more thought, time & energy to maintain academic integrity

- Multifaceted approaches being pursued

- Courses with large quantitative content impose additional challenges

Conclusions/Observations

45

. If a student has resources which give him/her an unfair advantage over other students, does this constitute unethical behavior, violations of the University’s Standards of Conduct, or cheating?

. If a student obtains instructor materials, such as the authors’ test banks for the course textbook, which give him/her prior knowledge of examination questions and therefore an unfair advantage over other students, does this constitute unethical behavior or cheating?

. Is it possible to write a statement of the Standards of Conduct which is comprehensive and which will withstand the scrutiny of attorneys in a court of law?

Pressing Questions

46

. How must these standards be communicated in a course syllabus? . Will largely technology-based solutions be sufficient?

. Have student attitudes toward academic integrity changed? . What are the special implications of these issues on courses taught in the DL format?

47

- Review of Troy University’s history & comparison with other universities – ID trends in academia’s views of ethical behavior, if any

- Comparison of alternate solutions

- Set of surveys - differences in views of academic integrity between students & instructors - ID acts which both populations consider unethical, violations of Academic Code, or cheating

- Expand literature review & surveys - propensity of students to threaten lawsuits and reaction of universities

Plan for Future Research

48

- Statistical analyses - ID correlations & trends in student & instructor views sorted by appropriate demographic factors

- Survey selected local businesses for current employee attitudes - comparison with academia

- Statistical analyses - ID changes & trends in attitudes (transition from pupil to university student and ultimately to working professional)

- Develop proposals - revisions of university policies toward Standards of Conduct & recommend corresponding adjustments to course syllabi, particularly for DL courses

49

What Message To Take Away?• Academic integrity issues on the rise.• Students are finding new, creative ways to

gain advantages (cheat)• Professors need to be vigilant in pursuit of

irregularities and enforcement of academic integrity (regardless of personal cost).

• Administrators need to ensure Academic Integrity policies will deal with technical alternatives available in the 21st century.

50