Problem-Based Learning at Husson College: Student Outcomes and PerceptionsFebruary 11, 2008Sondra G. Siegel, Ph.D., P.T.Department of Physical Therapy

PBL is an approach to learning that uses a problem to drive the learning rather than a lecture with subject matter, which is taught.Azer, 2001What is Problem-based Learning?

Elements of PBLInstructor creates a scenario that will engage the students in learning

Students work in small groups with a tutor

Tutor is not meant to be a source of information

Steps of PBL (Schmidt, 1983)

Clarify termsDefine and analyze the problemDraw a systematic inventory of the explanations inferred from step 3Formulate learning objectivesCollect additional information (between sessions, individual homework)Synthesize, discuss and test the newly acquired info (next session)

Benefits of PBL Curricula, as compared to conventional teaching methodsMore systematic reasoning (Patel, 1991)Greater development of hypotheses, but increased generation of errors (Patel, 1991)Enhanced clinical reasoning (Thomas, 1997)Enhanced self learning skills (Thomas, 1997) 5.Better self-directed learners, lifelong learning ( Shin, 1993)

Can We Employ PBL at Husson in a PT Program?Barriers:Additional personnel/timeOn-site library resources are limitedResistance to trying something newMaturity level of studentsComplete curriculum vs. single courses

General Research Question:

Can a single course in PBL (rather than an entire PBL curriculum), promote the proposed benefits of PBL?

Specific Research Questions:Does one course in PBL change the students:Expectations regarding future commitment to life-long learning?Independence in learning/motivationAttitudes toward group problem solvingPerceptions of ability to diagnose/perform as a primary health care practitioner

Specific Research Questions, contd:Do students spend more time seeking out information independently, as compared to time they spend at this in a traditional course?

Does one PBL course improve the students ability to critically analyze a case and identify the learning issues?

Course mechanics26 StudentsFour groups of studentsOne primary instructor/coordinatorFour faculty members8 week course, 2 hrs/wkFaculty meetings, 1 hr/wk

Research ParticipantsStudents in the final year of MSPT program

Enrolled in PT620, Primary Health Care

Total of 26 students in course 24 students consented to participateComplete data for 23 students

Approximately 1/3 of the students were non-traditional

Preferred Approach to Learning New Material

Lecture11Small groups, seminar, interactive8

Independent, Self-directed6Hands-on5Visual, demonstration4Reading3

Specific Research Question:Does one course in PBL change the students:Independence in learning/motivationExpectations regarding future commitment to life-long learning?Attitudes toward group problem solvingPerceptions of ability to diagnose/perform as a primary health care practitioner

Survey5 point Likert scale4 categories of randomized itemsIndependent learning/motivationLifelong learningGroup processDiagnostic abilitiesSome items had reversed scoring

Sample survey items:Motivation level/attitude toward independent learning

I learn best when I am provided the important information in a lecture format

Learning is most effective for me when I have to seek out the information on my own

I feel I have the skills necessary to continue as a self-directed learner after I graduate from this program

Sample survey items:

Attitudes toward life-long learning In the future, I intend to rely heavily on journal sources to stay current in my field.

I expect that keeping up in my field of practice will involve a considerable effort and expenditure of time

After completing my physical therapy degree, I will have sufficient knowledge to treat patients without doing additional research

Sample survey items:Group process/group dynamics

I am confident that I can make important contributions to small group learning situations

Learning in small groups stimulates me to engage in the content.

Working in small groups is usually frustrating for me.

Sample survey items:Ability to Diagnose and Perform as a Primary Health Care practitioner

I am confident in my ability to identify signs and symptoms that are indicative of non-musculoskeletal system pathology.

I feel confident that I will know when I need to refer a patient to a physician for a condition that may require medical treatment.

I am well-prepared for the clinical responsibilities of a primary care practitioner..

Specific Research Question:Do students spend more time seeking out information independently, as compared to time they spend at this in a traditional course?

Time spent on PBLPre-course estimate of number of hours spent in independent research for a typical 3-credit course

Post-course estimate of number of hours spent in independent research for this course

P

Specific Research Question:

3.Does one PBL course improve the students ability to critically analyze a case and identify the learning issues?

Case AnalysisProblem solving case (pre- and post-course)Define the learning issues in this following case. In other words, if you were the therapist treating this patient, what questions come to mind that require further research on your part? What would you want to research in order to best understand and treat the patient?

For this exercise, assume you are working as a primary care provider, in a place where it is common for patient visits to be initiated without a referral from an MD.

Your patient is a 49 year old woman whose primary complaint is urinary incontinence.

On interview she tells you that she has had this problem for about 3 yrs, but has been embarrassed to see someone about it.

She tells you that she typically wears an absorptive disposable undergarment at work, which needs to be changed several times each day.

Recently she has been bringing another pair of pants to work since she sometimes soaks through the pad.

She works for MBNA at a local call center. Her job is not strenuous, but her supervisor has reprimanded her for taking frequent breaks to use the restroom.

Over the last month, she has seen a dramatic worsening of the incontinence, and she feels she has almost no control of her bladder.

She has a cast on her left arm. When asked, she says that she has gotten clumsier as she has gotten older, and she tripped and fell on her porch 3 weeks ago and broke her arm.

She has not seen a doctor for the incontinence.

She is a single mother of 3, with kids ranging in age from 9 to 15.

Examples of Identified Learning IssuesProgression of incontinence (prognosis)Secondary problems arise with incontinence?Innervation of pelvic floor muscles?Connection between loss of bladder control and poor balance?Medications for incontinence?State laws regarding workplace accommodations?Lab tests associated with incontinence?

P

Improved attitudes toward lifelong learningImproved attitudes toward independent learningImproved ability to identify case-related issuesA short 8-week problem-based learning course can lead to: Summary

ConclusionsPBL can be an effective teaching method at HussonPBL does not necessarily need to be carried out throughout the whole curriculum to achieve some of its benefitsPBL may need to be combined with some more traditional methods of teaching in order to best reach our students

Student Perceptions of the CourseLess retention of the materialLess knowledge gainedNot sure if we learned the right informationLoved the independent aspect of itShould have more PBL courses in the programShould be combined with a lecture component

Course ChangesHybrid courseOne PBL session per weekOne traditional session per week

More specific objectives given after each case

Change in the assessment proceduresExamCase analysis

Not traditional lecture, or even traditional discussion. Allows the students to determine, in a group, what the important issues are, and what they need to know to fully understand and then solve the problem.*Tutor is consistent from week to week

The tutor is not necessarily a content expert, though students seem to prefer it if the person is a content expert. Theoretically, the tutor doesnt really need to know much about the subject, but is supposed to help the discussion to flow, and to challenge assumptions if the other student do not do so, etc. Many school use people who are outside the field that is being taught as tutors, which is how they deal with the large faculty requirements for PBL classes.*Students read the problem, and make sure that they understand all of the terms. If they dont then the terms will become part of the problem. For example, a problem that I might give could contain the result of a screening test that the students are unfamiliar with, and then theyll need to familiarize themselves with it. On the other hand, there might be one student who knows the term, and can help the other students out.Once they have gone through the first step of clarifying the terms, then they begin to brainstorm about the problem. This is the really fun part of the session. They look for aspects of the case that stimulate further discussion, or that they feel they do not fully understand. They discuss the questions that are raised in their minds based on the case scenario. And they brainstorm potential answers to the questions. They try to find links between their ideas, finding commonalities, finding differences. For example, in a clinical case, they might be given a case with several signs or symptoms. They are looking to find out what the diagnosis is, so they discuss any number of possibilities, and then figure out what else they need to know in order to best answer the question.Steps 2 and 3 dont really occur in order, in my experience. The students are analyzing and coming up with explanations somewhat randomly, and we usually allow the random discussion.Then they try to put some order into what theyve discussed, and define what they need to know before the next session. These are the learning objectives for the next week. Between the end of this session, and the beginning of the next, they then work on their own to find the information, to complete the learning objectives. Because of this, it is helpful if they are very explicit in defining their learning objectives otherwise they do not have enough structure to follow for the next session.They come back the next time, and discuss what theyve found. Its sometimes hard to get a real discussion, as opposed to people just reporting on what theyve done, but we can talk about some of this later.*These results primarily come from medical or dental school, who have investigated their PBL curricula.Most are from programs where the curriculum is entirely or nearly entirely PBL.Enhanced clinical reasoning better performance on written clinical exams, slightly worse performance on basic science exams.Enhanced self-learning skills more use of reference material books, journals, computer searches, especially reported from their clinical sitesBetter self-directed learner, increased motivation also based on clinical sites*There are always obstacles to initiating something new, whether it is for the college, or within a particular program. I am suggesting this is new for Husson just because I, personally, am unaware of any other PBL courses here, but that does not mean that they dont exist.

So, I want to acknowledge that realistically we have some barriers here.

Like I said, PBL does take additional time, but it because the tutors dont have to be content experts, there is more flexible. There are medical schools that utilize upper level students as tutors. For example, Emory employs MD/PhD students, who have completed the first two years of medical school (the didactic portion of the training) but who have not done their clinical rotations, as tutors for PBL. I learned about PBL from a PT who acts as a tutor for courses in a medical school, who is not an expert in the particular area.

We are gradually getting more and more electronic journal access, and for our purposes, our library has increased its holdings over the years. Students use a lot of web sites that unfortunately they are not too critical of, which can be a problem. This leads to the discussion in the tutorials, of how good is the site if a web site was used.

Resistance this is something that we all are challenged by, and I think we need to take a look at it, see if it might work for our courses, which is what we did, and then evaluate. Its always hardest the first time, and it doesnt always work as well the first time, but unless you all are lots better at teaching than I am, new experiments are a challenge. They take evaluation and reworking in order to achieve the best outcome, but often we decide that its worth it.

Maturity level of students. I know of some high schools that teach some courses by PBL, so I think we shouldnt underestimate our students.

Complete curriculum vs. single courses this is actually what my study was about, because as I said, I dont think its feasible nor even desirable for us to go 100% PBL.

*Gained approval from the esteemed Husson College IRB (I was not present for the discussion).*The first two were my generic goals for PBL. What I wanted the students to realize is that this is real life for them. They will be presented with situations, and no one is going to ask them specific questions, as in an assignment. They are going to have to look at the situation and figure out what they dont know, figure out what they need to know, and go from there. I was hoping that this course would convince them of the value of lifelong learning.

So, I wanted to know whether this exercise helped them to realize this. Did it promote the notion that life-long learning was important, and that they would have to keep it up after graduating?

Did this motivate them to pursue more independent learning. Students are typically motivated by exams, and again, in real life we dont have written exams. We have some sort of situation to deal with.

The question about group problem solving was included because our students do a lot of group work, but it is not this type of group work. It was my impression that at this point they are very tired of group work, so I just was interested in whether this type of group work was more enjoyable that putting together a group project, and whether it would encourage them to pursue group problem solving in the future, which is different than a group project.

Finally, the last goal was particular to this class, and so if of less interest to the rest of you. Essentially I wanted to know whether this class achieved the goal of teaching them some of the skills the would need to be a competent primary health care practitioner in a setting in which they could practice without referral from an M.D. It would have been very distressing if they had not improved in this area, but it was a possibility, if the course was ineffective.*The second question had do with how much time they spent doing independent work, utilizing their resources in this course as compared to a traditional course. Again, it would have been very distressing to find that there was not a difference, but you dont know until you ask the question.

Finally, does this 8-week course improve their ability to analyze a case and identify the issues that need to be investigated. So, here Im asking not only whether their attitudes toward learning improved, and not whether they perceive themselves as better able to analyze a case, but I gave them a case and actually looked at their ability to identify the learning issues.*I asked this question as part of an initial survey, just as demographic information. It was an open ended question, Describe your preferred approach to learning news material.

In retrospect, I wish I has given them options, and had them choose one, but several students indicated more than one preferred approach, so I included them all. This is why this chart doesnt add up to 23.

Still, I think you can see that most of them chose lecture as their preferred means for learning new material. Not surprising. If we had had a group that all said that they were independent, self-directed students, we might interpret the results differently.*These questions were answered via a survey that I created. The responded to the same survey pre and post course. The*The students responded to a survey that I created that had 46 items on it. Each item fell into one of these 4 categories. I analyzed each category individually, and will present the results for each of these four categories.

The students were presented with statements for which they had to indicate strongly agree, agree, neutral, disagree, and strongly disagree.

They were given the same survey immediately prior to the class, and again after completing the class.

The surveys were analyzed completely anonymously, with each student choosing his or her own code and entering it on the pre and post course survey.

I used paired t-tests to analyze the means of scores for each construct pre and post course. I am going to give you examples of some of the statements included in each category, and will then show you the results.*Y axis are the average likert scale scores, and the scale goes from the maximum to the minimum for the class. This scale will be repeated on each of the survey slides.

Small increase, but its a statistically significant increase. The numbers are hovering between neutral and agree, and progressing more toward agree.

Need to keep in mind that it is an 8-week course, the final semester of the MSPT program, and students are somewhat resistant to admitting they have anything more to learn, or that we have anything more to teach them. They are ready to be out working, and are not thrilled about any coursework at this point! So, a little increase may be important.*Like I said before, this is one of the major goals for the course for me. In addition to having the students gain knowledge to work as primary care providers, so this was really important. I do remember student telling me that after she graduated, she wouldnt need to listen to anyone elses advise, because once she got her license, she could practice however she wanted. She was not the norm, but still that attitude does exist. Also, this is one of the goals of college education, and its unclear how we need to promote it and convince the students of it. These cases may be very real, and are useful in this.*Again, you see a statistically significant increase in this value. It doesnt look huge, but again, you have to remember that it is just a 5 point scale. I was really pleased to see this as a result.**Like I said before, our students have done many group projects, and even though in my eyes, this group work was very different, their attitudes toward group dynamics were relatively unchanged. This was not too surprising, and actually served to convince me that the other differences were real, when in this construct, there was not difference after the course.*Its a good thing this is different, because otherwise I would have had to conclude that their perception was that the course taught them nothing. Still, since it is a class involving cases and not hands on work, their confidence is not that great. It think they gain confidence in their skills after they have used them in the clinic. We have seen this in other outcome tools that we have used they are not confident in patients with certain diagnoses, even after they have the didactic information, until after they have treated those patients in the clinic. And if during the course of their education, they do not see many patients of that type, regardless of their classroom exposure, they are not confident. So anyway, they have improved in their perception of their own diagnostic abilities.*The second question had do with how much time they spent doing independent work, utilizing their resources in this course as compared to a traditional course. Again, it would have been very distressing to find that there was not a difference, but you dont know until you ask the question.

Finally, does this 8-week course improve their ability to analyze a case and identify the issues that need to be investigated. So, here Im asking not only whether their attitudes toward learning improved, and not whether they perceive themselves as better able to analyze a case, but I gave them a case and actually looked at their ability to identify the learning issues.*Initial survey asked them approximately how many hours they spend doing independent research for a typical 3-credit course, per week.

The post course survey asked them to estimate the number of hours per week they spend on this for PBL.*Almost double. I think they overestimated on the pre-test, but I cant make that statement for certain. But I really dont believe that they spend 2 hours per week looking for information on their own. They spend that time reading perhaps, or studying. Some of them said as many as 5 or 6, and some were honest enough to say 0.**On the y axis what you see is the number of issues identified. There was no attempt to rate the issues, because there is no attempt during the brainstorming portion of this to judge the issues. The students are simply trying to identify issues that they feel they need additional information on, or need to further research. These can be different for different people. For example, the list that you just saw included one statement on innervation of the pelvic floor muscles. This would not be an issue for a person who remembers their anatomy.

What you can see is that the number of issues that were identified increased by the end of the course. There was a significant difference between the number of issues identified after the course, which suggests that the students became more proficient in identifying the issues that were relevant to the case. *Not part of the objectives of the course, but sort of interesting.

Some of these were not surprising. Again, these are the perceptions of the students, so the fact that they felt that they had less retention was not unusual. Its common for student for students to feel that they have not learned as much, but the studies do not support this. In fact studies suggest that their long term retention of the material is better with PBL.

Student feedback has informed some of the changes that weve made in the course. **