COUN 547: RESEARCH & EVALUATION IN COUNSELING

Spring 2016

Instructors: James R. Morton, MEd, MA, NCC Julia Champe, PhD, LPC (ID) Class Room: Pulliam 119 Class Hours: Tuesday 2:30 – 4:50p Office and Office Hours: James Morton Wham 223Q E-mail: jrmorton@siu.edu Dr. Champe Wham 223R E-mail: jchampe@siu.edu Main Office Phone: 618 453-2311

Course Description: This is an introductory graduate-level course in research design and methods and

program evaluation. The course prepares MS level advanced counselors-in-training (CITs) to understand research design and evaluation, including basic statistics, so that they can become skilled and informed consumers of published research to inform practice in various settings. The course will also focus on the process of conducting research in the field in accordance with ethical guidelines and standards of practice and institutional policies and procedures.

Course Purpose: The purpose of the course is to guide CITs to a better understanding of research design and methods so that they can become more skilled and discerning consumers of published research in their work settings (CMH, MCF, and SC). CITs will also learn about program evaluation and implement an evaluation of a program as part of the course requirements.

Course Structure and Instructor Expectations: This course focused on two areas. Part One covers research design and methods to include ethical issues in research, literature review, designing and conducting research, and interpreting research findings. Part Two addresses how to conduct a program evaluation.

Class time is largely comprised of lectures, group activities and discussions. Your learning will be enhanced by these class activities and discussions. To maximize your in-class experience, you should come to class having read assigned readings, being prepared to discuss them and becoming actively involved in your own learning by asking questions and engaging in class

SYLLABUS

activities. You will also have a few weeks of independent work in order to complete the projects that are required of you.

Required Texts

American Counseling Association (2014). ACA Code of Ethics. Alexandria, VA: Author.

http://www.counseling.org/docs/ethics/2014-aca-code-of-ethics.pdf?sfvrsn=4

Creswell, J. W. (2015). Educational research: Planning, conducting, and evaluating quantitative (5th ed.). Pearson.

Required Readings

Astramovich, R. L., & Coker, J. K. (2007). Program evaluation: The accountability Bridge Model for counselors. Journal of Counseling and Development, 85, 162-172.

Dimmitt C. (2010). Evaluation in school counseling: Current practices and future Possibilities. Counseling outcome research and evaluation, 1, 44-59. DOI: 10.1177/215013781036130

Everall, R. D., Altrows, K. J., & Paulson, B. L. (2006). Creating a Future- A Study of Resilience in Suicidal Female Adolescents.

Galvan, J. L. (2013). Writing literature reviews (5th Ed.). Glendale, CA: Pyrczak Publishing. (9781936523030)

Priest, S. (2001). A Program evaluation primer. The Journal of Experiential Education, 24, 34-40.

Wang, M., Nyutu, P. N., & Tran, K. K. (2012). Coping, Reasons for Living, and Suicide in Black College Students.

Recommended

American Psychological Association. (2010). Publication manual of the American Psychological

Association (6th ed.). Washington, DC: Authors

Gay, L.R., Mills, G.E. & Airasian, P. (2012). Educational research: competencies for analysis and application (10th ed.). Upper Saddle River, NJ: Pearson.

Schreiber, J. & Asner-Self, K. (2011). Educational research. Hoboken, NJ: Wiley.

Instructor reserves the right to assign additional readings

SYLLABUS

Course Objectives: CACREP Standards, 2009

By the end of the course, it is expected students will be able to:

1. Understand how to critically evaluate research relevant to the practice of clinical mental health/school/marriage, couple, and family counseling.

2. Understand research methods such as qualitative, quantitative, single-case designs, action research, and outcome-based research

3. Understand statistical methods used in conducting research and program evaluation 4. Understand principles, models, and applications of needs assessment, program evaluation,

and the use of findings to effect program modifications 5. Use research to inform evidence-based practice 6. Understand ethical and culturally relevant strategies for interpreting and reporting the results

of research and/or program evaluation studies 7. Know models of program evaluation for clinical mental health/school/marriage, couple, and

family counseling programs. 8. Know basic strategies for evaluating counseling outcomes in clinical mental

health/school/marriage, couple, and family counseling. 9. Know current methods of using data to inform decision making and accountability (e.g.,

school improvement plan, school report card). 10. Apply relevant research findings to inform the practice of clinical mental health counseling. 11. Develop measurable outcomes for clinical mental health counseling programs,

interventions, and treatments. 12. Analyze and use data to increase the effectiveness of clinical mental health/school/marriage,

couple, and family counseling interventions and programs.

Caution

While there are many definitions of what constitutes plagiarism, for the purposes of this course, plagiarism is defined as using another person’s ideas or words and claiming them as your own (e.g., without providing an appropriate reference). The University's policies and procedures regarding academic dishonesty are available in the Graduate School Catalog.

Grading: Total = 1,000

§ Attendance, Participation = 45 points § Homework =105 points (15 points each) § Project 1: Critique/Analysis = 150 points (15 points each) § Project 2: Research Prospectus = 300 points § Project 3: Program Evaluation = 300 points § Final = 100 points

SYLLABUS

Final grades will be determined as follows:

A=900-1000

B=800-899

C=700-799

D=600-699

F=below 600

Evaluation and required projects

Attendance and Class Participation (45 points) It is expected that you will participate fully in all class discussions. You are also expected to read the assigned readings for each week in order to facilitate quality discussions

Homework (105 points, (7 * 15))

Homework will be assigned; you will email them to the instructor by noon of the due date AND bring in a hard copy to class (except for Homework 4 and 6).

Homework 1: Select 3-5 research related academic articles of your interest. You are encouraged to tie this with your future homework efforts. Read, analyze, and synthesize the articles in alignment with the article critique handout. Please write in the academic narrative form with respect to the APA style (2010). Be sure to include title page, content, and reference page. Length: no more than 8 content pages. Papers requiring additional work to meet the basic standard will be returned for revisions before receiving grades. Please email the paper to James by noon of the due date. For example, for the first homework, you will email your work by noon of Tuesday, January 26, 2015.

Homework 2: Please note that this can take you about three hours! Go to the SIU OSPA website and follow the directions to the CITI training under compliance. Complete the Human Subjects Training. Print your certificate and bring a copy to class next week.

https://www.citiprogram.org/Default.asp?

Homework 3: Share your initial thoughts on the research problem of interest and perspective research questions that include justifications from the literature (no more than one page – reference list (minimum of 3 articles) can be on a second page). Bring hard copies for every member of the class (CIT and instructor) and be prepared to discuss.

SYLLABUS

You are advised to focus this exercise toward your major area of emphasis; e.g., if you are in school counseling your topic should relate to this professional area, if you are in clinical mental health counseling it should relate to this professional area, or relate it to marriage, couples, and family if this is your professional area.

Homework 4: CITs revise their research questions considering feedback received in class along with justification from a minimum of 5 articles in the literature. Do not bring copies to class.

Homework 5: CITs will turn a statement about how they will select a sample of persons (maximum of a page) that relates to their research topic. Bring hard copy for each CIT and instructors.

Homework 6: CITs will review instrumentation and data analysis based on a minimum of 5 articles. Students may opt for a quantitative or qualitative approach. The construction of this homework ought to be in the questions/answer format. That is, give the question and then provide the answer. The academic narrative format will be done in your Project 1 construction. Quantitative 1. What were the hypotheses of the study? 2. What instruments were used? Are psychometric properties discussed (validity, reliability,

norm groups)? 3. Do instruments seem appropriate and sufficient to define and describe the variables

investigated in the study (consider the hypotheses)? 4. Where, when, and how was data collected? Do the data collection procedures appear

appropriate? 5. How were the data analyzed?

Qualitative 1. What were the research question(s) of the study? 2. What were the forms of data collected and how did the researcher identify and adjust for

subjectivity, positionality, and reflexivity during the data collection and analysis?(trustworthiness, transparency, and rigor)?

3. Do the components of the data set convincingly describe the phenomenon, case study, or grounded theory that was studied? Considering the research question, was it answered with the data and analysis conducted?

4. Where, when, with whom, and how was data collected? 5. How were the data analyzed? How did the author(s) manage and report on the subjectivity in

analyzing the data?

SYLLABUS

Homework 7: CITs present briefly (10 minutes only!) on their research topic. Consider your research question, statement of the problem, social significance/rationale, hypotheses, possible instruments, etc. Have a two-page handout (third page for references) for each CIT and instructors. You may use powerpoint.

As always, remember that effective class participation includes: respectful and non-dominating involvement in class discussions, and showing initiative and investment in classroom activities. You are expected to be a team player in your group projects. Work with your colleagues, taking on a fair share of the work. Team members will submit an evaluation of their contribution to the group project as well as that of their colleagues.

Project 1: Critique/Analysis of Articles (150 points)

(10 articles, for 10 points on each)

Requirements: Select 10 research articles from peer-refereed journals (electronic or print) related to a your topic that is of interest to you and write a critical analysis of each article.

For this exercise, we are not looking for a summary of what the researchers found, but rather, for your own interpretation of what was done, how it was done, was it appropriate or not, and why. You are advised to look at articles from counseling journals and may only go to other disciplines (with instructor approval) if there is a dearth of published research in your topic of interest. This paper must follow APA format. A rubric is also provided. Due dates are indicated on the course schedule.

A topic sentence outline is due on 23 February. Include references. The purpose in having a topic sentence outline is to have you fabricate the structure of your literature review with an opportunity to receive feedback from the instructor. Graded as Pass/Fail. Needs to be a Pass before writing paper. Please plan accordingly as the Project 1 due date will not change.

Recommended Journals Journal of Counseling & Development (JCD); Journal of Mental Health Counseling; Educational and Psychological Measurement; Journal of Multicultural Counseling and Development (JMCD); Measurement and Evaluation in Counseling and Development (MECD); Professional School Counseling. The Family Journal. A longer list is available on D2L.

NOTE: At least two articles should be qualitative research articles and two articles should be quantitative research articles. You will run an originality check on Turnitin and submit a copy of the originality summary report along with an emailed copy of your submission.

SYLLABUS

Project 2: Research prospectus (300 points)

As counseling professionals, our clinical practice is informed by empirical evidence. We believe that counselors must be not just passive consumers of research, but rather, active ‘doers’ and participants in research. It is therefore important that counselors be well versed in the intricacies of conducting research, as well as be equipped with skills that allow them to be discerning consumers of research. In this assignment, you will pick a topic of interest to you (if you were to complete a thesis, what might you focus on?) and complete an initial draft of chapters 1-3 of a thesis. You will be provided with an outline for this assignment. While this is for a hypothetical study where you will not be required to actually conduct the research, you should include a Human Subjects Application in your appendices. The form can be found at http://ospa.siu.edu/compliance/index.html. CITs are encouraged to submit proposals for presentation at state, regional and national conferences. We are more than happy to work with you upon further developing your idea for actual research.

NOTE: Chapter 2 must contain at least 15 references

You will run an originality check on Turnitin and submit a copy of the originality summary report along with your submission.

Project 3: Program Evaluation (300 points: 150 from group and 150 individual)

Professional counselors are ethically required to engage in evaluation of the programs that are offered to clients. Sometimes this occurs at the agency/school level and involves program review. Other times, these efforts are focused on determining the effectiveness of a local level program. This assignment is designed to broaden your understanding of your role in the program evaluation aspect of professional counseling in the venue of your specialization. The project will use a team of classmates working collaboratively to conduct a program evaluation. It will result in a written and oral group report. This report will be submitted for instructor evaluation and be presented to your classmates for peer review. The grading is split between the outcome of the group and the individual contribution.

Steps:

1. Team members will form a tentative plan for conducting program evaluation at a site that they have access to, agree upon and at which they can obtain authorization to complete an evaluation. In Team members will review mission statement and program goals. Each team will design a plan for program evaluation and work collaboratively to complete the evaluation as well as compile a final report. Note that some teams may be assigned a program to evaluate. Students construct individual contributions to project for which they will be graded on their individual contribution. (This allows for individuals to gain credit despite the outcome of the project. It holds individuals accountable to what each person will contribute.)

SYLLABUS

2. Writing a working draft: Your team will submit a rough draft of the overall purpose of the evaluation, goals for the study, target population and method of collecting data. Teams will select or design the instrument to be used for data collection. Each team will operationalize their data collection procedures as outlined in their draft.

3. Data analysis and Final report: Team will focus on the final write-up and presentations. Each team may use whatever display options they choose. Because Wham 228 is a smart classroom, there are many options. Each team will need to prepare a handout of their targeted program and purpose of the evaluation. Your final evaluation report addressing the areas identified above and any additional areas as may be relevant for your program evaluation.

Requirements for Your Team Paper Your Final report should include the following components:

• The program goals or objectives you wanted to evaluate. • The questions you set out to answer • The information that was required to answer the question (or determine whether the

objectives had been attained). • Design used to collect the information. • Sources of that information. • Method(s) for collecting the information including

o Sampling procedures o Instrument used o How data were collected

• How data were analyzed. • How data were interpreted (e.g., degree to which data supported the goal of the

evaluation). • Limits of the report and what might be done next. • Recommendations for how results might be implemented to make changes to the program for improvement or maintenance of current progress. • Budget (No evaluation is without cost even if done "in house." There may be duplicating costs, the cost of taking time to develop questionnaires, write reports, gather data, etc. Discuss with your team members how these costs will be equitably distributed.)

Final Exam (100 points)

The final exam will be posted on D2L. You will receive more information in the course of the semester.

PLEASE NOTE: Any assignment turned in late will be penalized 10% per 24 hours starting the minute after it is due.

SYLLABUS

COUN 547

Spring 2016

Course Outline

Date Class Topics To Do

January 19

Session 1

Introduction and course overview

The Role and Importance of Research

DUE TODAY:

Creswell (2015) Chapter 1

Read the sample of quantitative research article at the end of the chapter.

Bring in the 2014 ACA Code of Ethics

For next week:

Creswell (2015) Chapter 1 and 2

Dimmitt C. (2010). Evaluation in school counseling: Current practices and future Possibilities. Counseling outcome research and evaluation, 1, 44-59. DOI: 10.1177/215013781036130

January 26

Session 2

Introduction to Research

- Types of research

Ethical Considerations in research

Research Questions

Homework 1 DUE

For next week:

Creswell (2015) Chapter 4

Priest, S. (2001). A Program evaluation primer. The Jouranl of Experiential Education, 24, 34-40.

Astramovich, R. L., & Coker, J. K. (2007). Program evaluation: The accountability Bridge Model for counselors. Journal of Counseling and Development, 85, 162-172.

SYLLABUS

February 2

Session 3

Determining the research question and justifying need. Types of variables.

Program Evaluation

-What is program evaluation?

CITs assignments to Program Evaluation projects

Homework 2 Due:

Please note that this will take you at least three hours!! Go to the SIU OSPA website and follow the directions to the CITI training under compliance. Complete the Human Subjects Training. Print your certificate and bring a copy to class next week.

https://www.citiprogram.org/Default.asp?

Homework 3: DUE: CITs present initial thoughts on their research questions with justification from the literature (see assignment in syllabus above).

For next week:

Galvan (2013), Chp 8;

Creswell (2015) Chapter 3

February 9

Session 4

Reviewing the Literature

What is known? What do we need to know? Why?

Homework 4: DUE:

CITs revise their research questions with justification from a minimum of 5 articles in the literature

For next week:

Creswell (2015) Chapter 5 and 6.

Other readings as assigned

February 16

Session 5

Quantitative Sampling & Instrument

Who/what answers the question and how?

Folks pick one research design for presenting on April 5th or 12th

Homework 5: DUE:

CITs will turn a statement about how they will select a sample of persons (maximum of a page) that relates to their research topic. Bring hard copy for each CIT and instructor.

For next week:

Creswell (2015) Chapters 7 and 8. Maxwell

SYLLABUS

Experimental, Correlational, Grounded theory, Phenonmenology.

(2005): pp 82 - 91.

February 23

Session 6

Qualitative Sampling and Instrument

Program Evaluation models

Homework 6: DUE: CITs will review instrumentation and data analysis in a minimum of 5 articles. (see assignment in syllabus)

Due: Topic sentence outline of Project 1.

For next week:

Review Creswell (2015) Chapter 3

March 1

Session 7

Teams develop program evaluation plan

Design instrument to collect program evaluation data and set up on survey platform of choice. Your plan should be ready to implement over spring break.

It is expected that you will be in Wham 306 or in the computer lab working with your group at this time. Instructor will be on hand for consultation on group projects

For next week:

Review Creswell (2015) Chapter 6 and 8.

March 15

Spring Break

No Class

SYLLABUS

March 22

Session 8

Quantitative and Qualitative Data analysis

- Numeric data - Non-Numeric data

Discussion of results/Implications for research and practice

Due: Project 1-Critical analysis articles

Due: your research question, statement of the problem, social significance/rationale, hypotheses, possible instruments, etc. Due the Friday after class!

March 29

Session 9

Research Project

- Brief presentations and discussion

Homework 7: Due: CITs present briefly on their research topic. Consider your research question, statement of the problem, social significance/rationale, hypotheses, possible instruments, etc. Again, due the Friday before class!

For next week: Creswell (2015) Chapters 10, 11, and 12

April 5

Session 10

Quantitative Research

Experimental and non-experimental research designs

Students present

For next week: Creswell (2015) Chapters 13, 14, and 15.

April 12

Session 11

Qualitative Research

Students present

The Big Five

For next week:

Creswell (2015) Chapters 16 & 17

April 19

Session 12

Mixed methods

Action Research

Project 2 Research Prospectus (chapters 1-3) due 4/22 at noon.

April 26

Session 13

Team Presentation: The team presents results of the program evaluation project.

SYLLABUS

May 3

Session 14

Students complete written report for the program evaluation project.

Project 3 Written Reports due

May 10

Final Exam On D2L

SYLLABUS

Appendix

The SIU Syllabus Attachment that is now required has little information on Graduate Policy.

Please note the following in addition to the SIU Syllabus Attachment.

Academic Integrity

From the SIUC Student Conduct Code: “In keeping with the principles and values of the University, students shall not:

1. Plagiarize or represent the work of another as one’s own work; 2. Prepare work for another that is to be used as that person’s own work; 3. Cheat by any method or means; 4. Knowingly or willfully falsify or manufacture scientific or educational data and

represent the same to be the result of scientific or scholarly experiment or

research;

5. Knowingly furnish false information to a university official relative to academic matters;

6. Solicit, aid, abet, conceal, or attempt acts of academic dishonesty.”

The complete student conduct code, including procedures in the case of a suspected violation, may be found at http://policies.siuc.edu/policies/conduct.html. Counselors-in-training (CITs) in this course are expected to understand and follow the complete SIUC Student Conduct Code.

Plagiarism is taken very seriously in this course. Note that representing someone else’s writing or work as your own reflections or work IS PLAGIARISM regardless of intent to cheat. Drafts of written work will be submitted to an online engine for originality analysis. Counselors-in-training (CITs) are expected to follow APA style in citing the work of others.

Incomplete Coursework

“An INC is assigned when, for reasons beyond their control, students engaged in passing work are unable to complete all class assignments. An INC must be changed to a completed grade

SYLLABUS

within one semester following the term in which the course was taken, or graduation, whichever occurs first. Should the student fail to complete the course within the time period designated, that is, by the end of the semester following the term in which the course was taken or graduation, whichever occurs first, the incomplete will be converted to a grade of F and the grade will be computed in the student's grade point average."

Students should not reregister for courses in which an INC has been assigned with the intent of changing the INC grade. Re-registration will not prevent the INC from being changed to an F.” (Graduate Catalog, http://registrar.siu.edu/records/incgrade.htm) If you are carrying an overall percent of C or better in the class at that time and meet the criteria in the university policy, you may request an INC by providing the instructor with an email AND a hard copy. Please note that a C is not acceptable coursework in Counselor Education and has serious consequences. Refer to the Counselor Education Master’s Handbook regarding policy.

Course Drop Policy

Students are permitted “a maximum of 6 credit hours or 50% of total semester enrollment of dropped courses during any given semester with a maximum of 12 credit hours of dropped courses over 60 hours of enrollment. Exceptions to this policy must be approved in writing by the Office of the Provost and Vice Chancellor for Academic Affairs. For the purposes of this policy, a dropped course is defined as any course dropped after the official date for receiving a full refund.”

Accommodating Religious Observances

“Southern Illinois University at Carbondale will make reasonable accommodation for individual student religious observances. … Students absent from classes because of observance of major religious holidays will be excused. Students must notify the instructor at least three regular class periods in advance of an absence from class for a religious holiday and must take the responsibility for making up work missed. … If an examination must be scheduled on a date that conflicts with a student’s required observance of a religious holiday, the student shall be given reasonable opportunity to make up the examination. Students must notify the instructor of the class if they plan to miss an examination. This should be done at least three regular class meeting periods in advance of the absence or at the time the announcement of the examination is made, whichever is later.”

For more information, please see: http://registrar.siu.edu/records/pdfrec/fl02geninfosched.pdf

SYLLABUS

Academic Accommodations

If you need academic accommodation by virtue of a documented disability, please contact the course instructor as soon as possible to discuss your needs. Students with documented needs for an accommodation must meet the same achievement standards required of all other students, although the exact way in which achievement is demonstrated may be altered. All requests for academic accommodations should be made as early as possible in the semester and arranged through Disability Support Services (DSS). Contact information for DSS: (618) 453-5738, DSSsiu@siu.edu, Woody Hall B-150, http://www.siu.edu/~dss/. Students with undocumented learning differences or who suspect that they may struggle in this course for any reason are welcome to schedule an appointment with the instructor within the first week of the semester to set up an individualized study plan to facilitate mastery of the material in this course. This should be done as soon as possible to allow the student to gain practice with the study plan before the course material becomes progressively more complex. Don’t wait until you are “in over your head!”

Emergency Procedure

Southern Illinois University Carbondale is committed to providing a safe and healthy environment for study and work. Because some health and safety circumstances are beyond our control, we ask that you become familiar with the SIUC Emergency Response Plan and Building Emergency Response Team (BERT) program. Emergency response information is available on posters in buildings on campus, available on BERT’s website at www.bert.siu.edu, Department of Safety’s website www.dps.siu.edu (disaster drop down) and in Emergency Response Guideline pamphlet. Know how to respond to each type of emergency.

Instructors will provide guidance and direction to students in the classroom in the event of an emergency affecting your location. It is important that you follow these instructions and stay with your instructor during an evacuation or sheltering emergency. The Building emergency Response Team will provide assistance to your instructor in evacuating the building or sheltering within the facility.