COH 609 dec 2011

National UniversitySchool of Health and Human ServicesDepartment of Community HealthMasters of Public Health

COURSE OUTLINECOH 609 - Public Health Program Evaluation

DECEMBER, 2011

Class Meetings: November 21, 23, 28, 30, December 5, 7, 12, 14, 17, +online exam

Professor: Suzanne Evans, MS, EdD, NBCT [email protected]

Office: National University Technology and Health Sciences Center, Room 1143678 Aero Court, San Diego, CA 92123-1788858-309-3484; cell: 847-431-0343

Office Hours: M & W 3:00-5:00 p.m. or by appointment

TEXTBOOKS:Required Textbook:

Harris, M. (2010). Evaluating public and community health programs. San Francisco, CA: Jossey Bass. ISBN: 978-0-470-40087-6

Internet Access and e-mail Access E-College Ecompanion

Required Evaluation Guides - available on ecollege MMR. (1999). Framework for Program Evaluation in Public Health

http://www.cdc.gov/mmwr/preview/mmwrhtml/rr4811a1.htm CDC. (2005). Introduction to Program Evaluation for Public Health Programs

http://www.cdc.gov/eval/evalguide.pdf Kellogg Foundation. (2004). Evaluation Handbook

http://www.ojp.usdoj.gov/BJA/evaluation/links/WK-Kellogg-Foundation.pdf Kellogg Foundation. (2004). Logic Model Development Guide

http://www.uwsa.edu/edi/grants/Kellogg_Logic_Model.pdf OTHERS AS ASSIGNED

Recommended Textbooks American Psychological Association. (2010). Publication manual of the American Psychological Association. Washington, DC: APA.

COH 609 Outline December 2011- page 1

COURSE DESCRIPTION:Fundamentals of evaluation methods applied to public health interventions. Effective use of measurement tools to evaluate achievement of program goals and objectives. Includes analysis of validity and reliability of measurement instruments. Emphasis on reach, effectiveness, acceptance, implementation, and maintenance of community programs. Includes fundamentals of proposal development.

COURSE LEARNING OUTCOMES:Upon successful completion of this course, students will be able to:

Define program evaluation components including pretests, posttests, comparative designs, control groups, reliability, and validity.

Evaluate the internal and external validity of a measurement instrument. Calculate the reliability of a measurement instrument. Describe the components of the RE-AIM framework for public health program

evaluation. Design evaluation measurement instruments for the components of RE-AIM. Develop a proposal for a public health program in response to a request for

proposals or applications.

PROGRAM MISSION: MASTER OF PUBLIC HEALTHThe National University MPH program prepares educated, ethical and high-functioning public health professionals that serve the global community by advancing health and social justice. The NU MPH program employs collaborative administration where faculty, students, and public health professionals collaborate to disseminate public health scholarship through teaching, research, and community service.

PROGRAM LEARNING OUTCOMES:

PLO

COH 609 1. Assess individual and community needs for health education. Mastery 2. Plan health education strategies, interventions, and programs. Mastery3. Implement health education strategies, interventions, and programs. Developed4. Conduct evaluation related to health education. Mastery5. Administer health education strategies and interventions. 6. Serve as a health education resource person. Developed7. Communicate and advocate for health and health education. Developed8. Apply appropriate research principles and techniques in health education. Developed

PROGRAM LEAD FACULTY & DEPARTMENT CHAIR:GinaMarie Piane, [email protected](858) 309-3474


mailto:[email protected]

TENTATIVE SCHEDULE:Instructor reserves the right to change the timetable in order to better address the learning needs of the students in the class.

DATE TOPIC READING ASSIGNMENT Due next class session

WRITTEN ASSIGNMENT Due next class session

ACTIVITIES/ ASSIGNMENTS Due next class session

Week 1

MondayNov 21

INTRODUCTIONS CHAPTERS 1, 2 Overview Framework For

Program Evaluation STEP 1-

STAKEHOLDERS CASE STUDY ACTIVITIES SYLLABUS AND

EXPECTATIONS

READING: Harris - Chapters 1,2,3,5 MMR Framework pp 1-34 CDC Intro.to Program Eval Intro & Steps 1 CDC Intro.to Program Eval Step 2 CDC Evaluation Guide – Smart Goals pp. 1-7

WRITING ASSIGNMENT: READING Response/Activity Step 1- Activity 1A & BDraft Section V- Stakeholders

Wednesday Nov 23

PROCESS Review STEP 1

Stakeholders STEP 2 -DESCRIBE

PLANNINGOBJECTIVESSMART GOALS

CASE STUDY ACTIVITIES

READING:Harris - Chapter 5 CDC Intro.to Program Eval Step 2 Kellogg Logic Model Development Guide- Innonet Logic Guide CDC Logic Model Development

WRITING ASSIGNMENT: READING Response/Activity Step 2- Activity 2A & BDraft Section VI- Description LOGIC MODEL

Week 2

Monday November

28

PROCESS Review STEP 2

OBJECTIVESSMART GOALS

CASE STUDY ACTIVITIES LOGIC MODELS

Logic

READING: Harris - Chapter 6 CDC Intro.to Program Eval Step 3

WRITING ASSIGNMENT: READING Response/Activity Logic ModelsDraft Section VI- Description - LOGIC MODEL

WednesdayNov 30

CHOOSING DESIGNS

STEP 3 - FOCUS THE DESIGN The Question


READING: Harris - Chapters 7 & 9 Harris – Chapter 8 & 10 CDC Intro.to Program Eval - Step 4 & 5

WRITING ASSIGNMENT: READING Responses Step 3- Activity 3A & B Draft Section VI- Design

Week 3


MondayDec 5

QUANTITATIVE DATA

QUALITATIVE DATA STEP 4- GATHER

EVIDENCE STEP 5- JUSTIFY

CONCLUSIONS CASE STUDY ACTIVITIES

READING: Harris - Chapters 7 & 9 Harris – Chapter 8 & 10 CDC Intro.to Program Eval - Step 4 & 5

WRITING ASSIGNMENT: READING Responses Step 4 - Activity 4 A& B Step 5 - Activity 5 Draft Section VII- Gather evidence Draft Section VIII- Justify

Wednesday Dec 7

Working Session –Complete Steps 1-5 finalComplete surveyPrepare presentations

READING: Harris - Chapters 11 CDC Intro.to Program Eval Step 6

Step 4 - Activity 4 A& B Step 5 - Activity 5

Evaluation Plan Section I-VIII CompleteSurvey

Presentations

Week 4Monday Dec 12

PRESENTATIONS1-3 Share responses

STEP 6 SHARE & USE

CASE STUDY ACTIVITIES STANDARDS OF

GOOD EVALUATIONS

READING: Harris - Chapters 4 Kellogg Evaluation Handbook- REAIM

WRITING ASSIGNMENT: READING Responses Step 6 - Activity 6A & BDraft Section IX Ensure Use

Presentations

Wednesday Dec 14

PRESENTATIONS4-6 Share responses PEER REVIEWS

REAIM MANAGING

EVALUATIONS FinancialEthical considerations


READING: Cultural Articles as assigned

WRITING ASSIGNMENT: EVAL PLAN

PresentationsPreparation for final

SaturdayDecember

17

PRESENTATIONS7-8 Share responses PEER REVIEWS

CULTURAL COMPETENCY

PARTICIPATORY EVALUATIONS

Review Final ExaminationOnline by 12/19

READING: Review for final

WRITING ASSIGNMENT: Submit Evaluation Plan online in dropbox

Preparation for final Final Exam

COURSE REQUIREMENTS:The readings, discussions, and assignments will focus on the exciting field of public health evaluation. Students are expected to complete all reading assignments prior to attending class. Participation in discussions and class activities is essential. Written


assignments must be submitted on the due date unless special circumstances are discussed with the professor in advance.

COH 609 Specific Course Requirements: Professionalism: Students are expected to:

demonstrate dispositions expected of students pursuing professional degrees demonstrate professionalism by participating in all class sessions/discussions

and submitting work on time be honest in all coursework.

Attendance: It is expected that students will: attend all class sessions -an absence is assessed each time a student is not in

attendance during a regularly schedule class period, whether or not it is an excused absence.

acknowledge that in accordance with National University policy, more than three absences from class or discussions, excused or unexcused, will result in a less than satisfactory grade.

Written Assignment Expectations: All assignments completed out of class are to be: typed/word-processed, font size 12, double spaced written in standard English, error free in sentence construction, grammar,

punctuation, and spelling. APA format

Late Work: Guidelines for late work are as follows: all assignments are due at date indicated. unless prior permission has been given to the candidate, late work will not be

accepted and no credit will be given for that assignment. with prior permission late work is accepted and will receive a deduction of 20% in

the grade

Assignments:This course enables the students to participate in the following required learning experiences and assignments:

1. Classroom Attendance, Dialogue, Discussion and Discourse: You will complete assigned readings in the text and evaluation guides, view lectures and participate in discussions. Credit is given for attendance and it is expected that everyone attend class on time and actively participate in class activities and discussions. It is expected that the assigned readings will be completed prior to the scheduled class so you can engage in class discussions. Class will begin promptly at the designated start time. Habitually arriving late or leaving early will result in an absence. Students who miss class are responsible for all material presented in class and are advised to consult with their fellow classmates to find out what was covered during their absence. Up to 5 points, each class session may be earned for attendance and participation (40 points).

2. Planning Exercises: You will be completing written exercises to apply understanding of evaluation concepts. The assigned readings are outlined in the schedule and the activity sheets provided. Exercises will be submitted each class session. (30 points).


3. Logic Model: You will design a logic model and incorporate the model into the Evaluation Plan. Development of the Logic Model (30 points).Signature Assignment

4. Survey Questionnaire Project: You will plan the steps in the creation of a questionnaire. The project will include a short paper describing the process and the creation of a questionnaire and cover letter to be used in your evaluation plan (20) points.

5. Evaluation Plan: You will design a program evaluation on a public health topic or on the grant proposal developed in COH 607. Complete a detailed paper in APA format. Process Outlines will be due during the development process. (60 points). Signature Assignment

6. Evaluation Plan Presentation: PowerPoint Presentation for the class on the Evaluation Proposal will be shared. (30 points). Signature Assignment

Exams:Comprehensive Final Examination: Using knowledge and application complete a multiple choice / essay exam.(50 points). Specific questions used as direct measures.

GRADING FACTORS Grades are based on points accumulated from all assignments, collaborative group activities, discussions and written assignments.

1. Classroom Attendance, Dialogue, Discussion and Discourse 402. Planning Exercises 303. Logic Model 304. Survey 205. Evaluation Plan 606. Evaluation Plan Presentation 307. Comprehensive Final Examination 50

TOTAL 260

Course Grading (expressed as percentages):A 96-100% B+ 87-89% C+ 77-79% D+ 67-69%A- 90-95% B 84-86% C 74-76% D 64-66%

B- 80-83% C- 70-73% D- 60-63%

Definition of Grades: Graduate Courses:A Outstanding AchievementB Commendable AchievementC Marginal Achievement D Unsatisfactory *


F Failing * *Student receiving this grade in a course that is required for his/her degree program must

repeat the course.

Definition of Grades: Incompletes (I) A grade of “I” may only be issued when a student has attended at least

two-thirds of the course sessions and is unable to complete the requirements due to uncontrollable and unforeseen circumstances. Students must communicate these circumstances (preferably in writing) to the instructor prior to the final day of the course. If an instructor decides that an “Incomplete” is warranted, he/she issues a grade of “I” and notates the conditions for removal of the “Incomplete” in the student’s record. Students receive notification via e-mail about the assignment of an “Incomplete,” including the conditions for its removal. The information remains in place until the “Incomplete” is removed or the time limit for removal has passed. An “Incomplete” is not issued when the only way the student could make up the work is to attend a major portion of the class the next time it is offered. Students must resolve “Incomplete” grades no later than six months after the official course end date. Students can be required to remove an “Incomplete” in a shorter period of time at the discretion of the instructor. The Office of the Registrar sends a courtesy e-mail reminder regarding the “Incomplete” on record when four months has elapsed since the course end date, provided the instructor has not specified a shorter end date. An “I” that is not removed within the stipulated time becomes an “F“ or a “U” based on the grading criteria of the course. No grade points are assigned. The “F” is calculated in the grade point average.

U Unsatisfactory A permanent grade indicating that a credit attempt was not acceptable. An “Unsatisfactory” grade merits no grade points and is not computed in the grade point average.

W Withdrawal Signifies that a student has withdrawn from a course after midnight of the ninth day of the class session. A “Withdrawal” is not allowed after the twenty-first (21st) day of the session. This is a permanent mark with no grade points assigned.

S Satisfactory: Credit is granted but no grade points are assigned. IP In Progress: A designation representing a project course that allows up to six months

or more for completion. No grade points are assigned for the “IP” grade.

BIBLIOGRAPHY:Websites & ToolkitsAmerican Evaluation Associationwww.eval.orgOutcome Mapping, 2001 International Development Research Centre www.idrc.caThe Handbook for Evaluating HIV Education. Centers for Disease Control and Prevention www.cdc.gov/nccdphp/dahs/publications/hiv_handbook/index.htmWhat, Me Evaluate?, 1986 National Crime Prevention Council www.ncpc.orgAssociation for the Study and Development of Community (June 2001). Principles for

evaluating comprehensive community initiatives. Washington, D.C.: National Funding Collaborative on Violence Prevention.www.peacebeyondviolence.orgCenters for Disease Control and Prevention (1999). Framework for program evaluation in

public health. www.cdc.gov/eval/framework.htm#cbph.Community Toolbox (online). Evaluating community programs and initiatives.

http://ctb.lsi.ukans.edu/tools/EN/part_1010.htmInnovative Network (online). Helping agencies succeed. www.innonet.org.National Science Foundation (2002). The 2002 User-Friendly Handbook for Project

Evaluation www.nsf.gov/pubs/2002/nsf02057/start.htm.


Office for Substance Abuse Prevention (1991). Prevention Plus III: Assessing alcohol and other drug prevention programs at the school and community level.www.ncadi.samhsa.gov.

Mob erg, DUPL. (1984). Evaluation of prevention programs: A basic guide for practitioners.www.uhs.wisc.edu/who.

Sierra Health Foundation (2000). We did it ourselves: An evaluation guide book.www.sierrahealth.org.

United Way of America (1996). Measuring program outcomes. Arlington, Va.University of Wisconsin-Cooperative Extension (1998). Evaluating collaborative. O.K. Kellogg Foundation (1998). Evaluation handbook. Battle Creek, MI.www.wkkf.org. O.K. Kellogg Foundation Evaluation Toolkit: http://www.wkkf.org/default.aspx?

tabid=75&CID=281&NID=61&LanguageID=0 University of Kansas Community Tool Box: Evaluation Model:

http://ctb.ku.edu/tools/en/sub_section_main_1007.htm Re-AIM http://www.re-aim.org/

Books & Journals Butterfish, F. D. (2006). Process evaluation for community participation. Annual Review of

Public Health, 27(1), 323-340. Cameron, R., Joplin, M. A., Walker, R., McDermott, N., & Gough, M. (2001). Linking science

and practice: Toward a system for enabling communities to adopt best practices for chronic disease prevention. Health Promotion Practice, 2(1), 35-42.

Cargo, M., & Mercar, S. L. (2008). The value and challenges of participatory research: strengthening its practice. Annual Review of Public Health, 29(1), 325-350.

Cargo, M., Deformer, T., Levesque, L., Horn-Miller, K., Comber, A., & Macaulay, A. C. (2008). Can the democratic ideal of participatory research be achieved? An inside look at an academic-indigenous community partnership. Health Educ. Res., 23(5), 904-914.

Clinton-Sherrod, A.M., Morgan-Lopez, I.E., Gibbs., Hawkins, S.R., Hart, L., Ball, B., Irvin, N., & Littler, N. (2009). Factors contributing to the effectiveness of four school-based sexual violence interventions. Health Promotion Practice. 10(1), 19S-28S.

Coffield, A., Maiese, D. R., & Fox, E. (1996). Objectives-Setting for Improved Health The Public Health Service Healthy People Program. [Washington, D.C.]: Dept. of Health and Human Services, Office of Public Health and Science, Office of Disease Prevention and Health Promotion. http://purl.access.gpo.gov/GPO/LPS30295.

Cronbach, L.J. and associates (1980). Toward reform of program evaluation. San Francisco: Jossey-Bass.

Dunton, G.F., Lagloire, R., & Robertson. T. (2009). Using the RE-AIM framework to evaluate the statewide dissemination of a school-based physical activity and nutrition curriculum: "Exercise Your Options&quot. American Journal of Health Promotion: AJHP. 23(4), 229-32.evaluation. San Francisco: Jossey-Bass Publishers, 1994.

Flicker, S. (2008). Who benefits from community-based participatory research? A case study of the positive youth project. Health Educ Behav, 35(1), 70-86.

Flicker, S., Maley, O., Ridgley, A., Biscope, S., Lombardo, C., & Skinner, H. A. (2008). e-PAR: Using technology and participatory action research to engage youth in health promotion. Action Research, 6(3), 285-303.

Flicker, S., Savan, B., Kolenda, B., & Mildenberger, M. (2008). A snapshot of community-based research in Canada: Who? What? Why? How? Health Educ. Res., 23(1), 106-114.

Glasgow, R. E., Klesges, L. M., Dzewaltowski, D. A., Bull, S. S., & Estabrooks, P. (2004). The future of health behavior change research: What is needed to improve translation of research into health promotion practice? Annals of Behavioral Medicine, 27(1), 3-12.


Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the public health impact of health promotion interventions: The RE-AIM Framework. American Journal of Public Health, 89(9), 1322-1327.

Goodstadt, M., Hyndman, B., McQueen, D. V., Potvin, L., Rootman, I., & Springett, J. (2001). Evaluation in health promotion: Synthesis and recommendations. In I. Rootman, M. Goodstadt, B. Hyndman, D. McQueen, L. Potvin, J. Springett & E. Ziglio (Eds.), Evaluation in health promotion: Principles and perspectives. Copenhagen: World Health Organization.

Hallfors, D., & Godette, D. (2002). Will the `Principles of Effectiveness' improve prevention practice? Early findings from a diffusion study. Health Educ. Res., 17(4), 461-470.

Hawe, P. P., & Potvin, L. P. (2009). What is population health intervention research? Canadian Journal of Public Health, 100(1), I8.

Ioannidis, J. P. A. (2005). Contradicted and initially stronger effects in highly cited clinical research. JAMA, 294(2), 218-228.

J.S. Wholey, H.P. Hatry, and K.E. Newcomer, (Eds.), Handbook of practical programKelly JT. (2009). Evaluating employee health risks due to hypertension and obesity: self-testing

workplace health stations. Postgraduate Medicine. 121(1), 152-8.Koelen, M. A., Vaandrager, L., & Colomer, C. (2001). Health promotion research: dilemmas and

challenges. J Epidemiol Community Health, 55(4), 257-262. Kuczmarksi MF, & Cotugna N. (2009). Outcome evaluation of a 3-year senior health and

wellness initiative. Journal of Community Health. 34(1), 33-9.Lavis, J. N., Posada, F. B., Haines, A., & Osei, E. (2004). Use of research to inform public

policymaking. The Lancet, 364(9445), 1615. Lipsey, M. W., & Cordray, D. S. (2000). Evaluation methods for social intervention. Annual

Review of Psychology, 51, 345-375.Lovisi Neto BE, Jennings F, Barros Ohashi C, Silva PG, & Natour J. (2009). Evaluation of the

efficacy of an educational program for rheumatoid arthritis patients. Clinical and Experimental Rheumatology. 27(1), 28-34.

Macaskill, L., Dwyer, J. J. M., Uetrecht, C., Dombrow, C., Crompton, R., Wilck, B., & Stone, J. (2000). An evaluability assessment to develop a restaurant health promotion program in Canada. Health Promotion International, 15(1), 57-69.

McDonald, D. (1999). Developing guidelines to enhance the evaluation of overseas development projects. Evaluation and Program Planning, 22(2), 163-174.

McKenzie, J. F., Neiger, B. L., & Smeltzer, J. L. (2005). Planning, implementing, and evaluating health promotion programs: A primer. San Francisco: Pearson/Benjamin Cummings.

McLaughlin, J. A., & Jordan, G. B. (1999). Logic models: A tool for telling your program's performance story. Evaluation and Program Planning, 22(1), 65-72.

McQueen, D. V., & Jones, C. M. (2007). Global perspectives on health promotion effectiveness. New York, NY: Springer.

McVey G, Gusella J, Tweed S, & Ferrari M. (2009). A controlled evaluation of web-based training for teachers and public health practitioners on the prevention of eating disorders. Eating Disorders. 17(1), 1-26.

Meltzer Graffunder C, & Charles D. (2009). Empowerment evaluation: building prevention science and organizational capacity to prevent sexual violence. Health Promotion Practice. 10(1), 71S-73S.

Nastasi, B., & Hitchcock, J. (2009). Challenges of evaluating multilevel interventions. American Journal of Community Psychology, 43(3-4), 360-376.

Nutbeam, D. (1996). Achieving "best practice" in health promotion: Improving the fit between research and practice. Health Education Research, 11(3), 317-326.

Nutbeam, D. (1998). Evaluating health promotion--progress, problems and solutions. Health Promotion International, 13(1), 27-44.

Nutbeam, D. (1999). The challenge to provide "evidence" in health promotion. Health Promotion International, 14(2), 99-101.


Opatz, J. P. (1987). Health promotion evaluation: Measuring the organizational impact. Stevens Point, Wis: National Wellness Institute, University of Wisconsin-Stevens Point Foundation.

Parkinson, S. (2009). Power and perceptions in participatory monitoring and evaluation. Evaluation and program planning, 32(3), 229-237.

Patton, M.Q. (1997). Utilization-focused evaluation (3rd edition). Newbury Park, CA: Sage Publications.

Pinto, R. M. (2009). Community perspectives on factors that influence collaboration in public health research. Health Educ Behav, 36(5), 930-947.

Rada, J., Ratima, M., & Howden-Chapman, P. (1999). Evidence-based purchasing of health promotion: Methodology for reviewing evidence. Health Promotion International, 14(2), 177-187.

Raphael, D. (2000). The question of evidence in health promotion. Health Promotion International, 15(4), 355-367.

Rossi, P.H., Freeman, H. L., & Lipsey, M.W. (1999). Evaluation: A systematic approach (6th Edition). Thousand Oaks, CA: Sage.

Salihu HM, Mbah AK, Jeffers D, Alio AP, & Berry L. (2009). Healthy start program and feto-infant morbidity outcomes: evaluation of program effectiveness. Maternal and Child Health Journal. 13(1), 56-65.

Saunders, R. P., Evans, M. H., & Joshi, P. (2005). developing a process-evaluation plan for assessing health promotion program implementation: a how-to guide. Health Promot Pract, 6(2), 134-147.

Scarinci, I. C., Johnson, R. E., Hardy, C., Marron, J., & Partridge, E. E. (2009). Planning and implementation of a participatory evaluation strategy: A viable approach in the evaluation of community-based participatory programs addressing cancer disparities. Evaluation and program planning, 32(3), 221-228.

Schensul, J. (2009). Community, culture and sustainability in multilevel dynamic systems intervention science. American Journal of Community Psychology, 43(3-4), 241-256.

Scott SA, & Proescholdbell S. (2009). Informing best practice with community practice: the community change chronicle method for program documentation and evaluation. Health Promotion Practice. 10(1), 102-10.

Scriven, M. (1991) Evaluation thesaurus (4th Edition). Thousand Oaks, CA: Sage.Shadish, W. R., & Cook, T. D. (2009). The renaissance of field experimentation in evaluating

interventions. Annual Review of Psychology, 60(1), 607-629Shadish, W.R., Cook, T.D., and Leviton, L.C. Foundations of program evaluation:Theorists

and their theories. Newbury Park, CA: Sage, 1991.Smith, M. K. (1998). Empowerment evaluation: Theoretical and methodological considerations.

Evaluation and Program Planning, 21(3), 255-261.Speller, V., Learmouth, A., & Harrison, D. (1997). The search for evidence of effective health

promotion. British Medical Journal, 315, 361-363.Steckler, A. B., & Linnan, L. (2002). Process evaluation for public health interventions and

research. San Francisco, Calif: Jossey-Bass.Thayer, C. E., & Fine, A. H. (2001). Evaluation and outcome measurement in the non-profit

sector: stakeholder participation. Evaluation and Program Planning, 24(1), 103-108.Valente, T. W. (2002). Evaluating health promotion programs. New York: Oxford University

Press.van Osch L, Lechner L, Reubsaet A, Steenstra M, Wigger S, & de Vries H. (2009). Optimizing

the efficacy of smoking cessation contests: an exploration of determinants of successful quitting. Health Education Research. 24(1), 54-63.

Vijgen, S. M. C., van Baal, P. H. M., Hoogenveen, R. T., de Wit, G. A., & Feenstra, T. L. (2008). Cost-effectiveness analyses of health promotion programs: a case study of smoking prevention and cessation among Dutch students. Health Educ. Res., 23(2), 310-318.

Wallerstein, N. B., & Duran, B. (2006). Using community-based participatory research to address health disparities. Health Promotion Practice, 7(3), 312-323.


Wallerstein, N., Polascek, M., & Maltrud, K. (2002). Participatory evaluation model for coalitions: The development of systems indicators. Health Promotion Practice, 3(3), 361-373.

Wandersman, A. (2009). Four keys to success (theory, implementation, evaluation, and resource/system support): High hopes and challenges in participation. American Journal of Community Psychology, 43(1-2), 3-21.

Wandersman, A., Imm, P., Chinman, M., & Kaftarian, S. (2000). Getting to outcomes: A results-based approach to accountability. Evaluation and Program Planning, 23(3), 389-395.

Webb, D., & Wright, D. (2000). Postmodernism and health promotion: Implications for the debate on effectiveness. In J. Watson & S. Platt (Eds.), Researching health promotion . London: Routledge.

Weiss, C.H. (1997). Evaluation (2nd edition). New York: Prentice-Hall.World Health Organization European Working Group on Health Promotion Evaluation. (1998).

Health promotion evaluation: Recommendations to policymakers. Copenhagen: World Health Organization Regional Office for Europe.

Worthen, B.R. & Sanders, J.R., & Fitzpatrick, J. L. (1996) Program evaluation:Alternative approaches and practical guidelines (2nd Edition). New York:Addison-Wesley.

Wurzbach, M. E. (2004). Community health education and promotion: A guide to program design and evaluation. Sudbury, Mass: Jones and Bartlett.

Journals:American Journal of Health Behavior American Journal of Health Education (formerly Journal of Health Education) American Journal of Health Promotion American Journal of Health Studies American Journal of Public Health American Journal of Public HealthCalifornia Journal of Health PromotionCanadian Journal of Public HealthEuropean Journal of Public HealthFamily and Community HealthHealth Education and Behavior(formerly Health Education Quarterly); Health Education Research (online version available) Health Promotion PracticeJournal of Epidemiology and CommunityJournal of Health and Social BehaviorJournal of School Health Journal of Public Health and PolicyJournal of Public Health Management and PracticeLeadership in Public HealthMorbidity, Mortality Weekly ReportPublic Health Reports

Organizational Resources:American Association for Health Education AAHE. http://www.aahperd.orgAmerican School Health Association ASHA. http://www.ashaweb.org American College Health Association ACHA- Health Education section: http://www.acha.org Agency for Healthcare Research and Quality. http:www.ahrq.govAmerican Public Health Association. http://www.apha.orgAmerican Cancer Society (http://www.cancer.org/docroot/home/index.asp)American Heart Association (http://www.americanheart.org/)Association of Schools of Public Health. http://www.asph.orgAssociation of Community Health Improvement (http://www.communityhlth.org/)Association of State and Territorial Health Officials. http://www.astho.org


http://www.asph.org/

http://www.americanheart.org/

http://www.cancer.org/docroot/home/index.asp

http://www.apha.org/

http://www.aahperd.org/

CA Department of Health (http://www. dph.ca.gov)CA. Department of Health Care Services (http://www.dhcs.ca.gov)Centers for Disease Control and Prevention. http://ww.cdc.govCenter for Law and the Public’s Health. http://www.publichealthlaw.netCommunity Tool Box (http://ctb.ku.edu/en/)Council on Education for Public Health. http://www.ceph.orgCenter for Minority Health (http://www.cmh.pitt.edu/)Environmental Protection Agency. http://www.epa.govEta Sigma Gamma http://www.cast.ilstu.edu/temple/esg.htm Health Resources and Services Administration. http://www.hrsa.govHealthy People 2010 (http://www.healthypeople.gov/)Healthy People 2010 Tool Kit: A Field Guide to Health Planning (http://www.healthypeople.gov/state/toolkit/default.htm)International Union for Health Promotion and Education. www.iuhpe.orgInternational Association of National Public Health Institutes (http://www.ianphi.org/) Kaiser Family Foundation State Health Facts (http://www.statehealthfacts.org/)Making Health Communication Programs Work (http://www.cancer.gov/pinkbook)National Association of County and City Health Officials. http://www.naccho.orgNational Association of Public Health Policy ( http://www.naphp.org/)National Center for Health Statistics (www.cdc.gov/nchs)National Coalition of Health Education Organizations, NHEO. http://www.hsc.usf.edu/CFH/cnheo/index.htmlNational Environmental Health Association. http://www.neha.orgNational Heart, Lung, and Blood Institute. http://www.nhlbi.govNational Institute of Occupational Safety and Health. http://www.niosh.govNational Institutes of Health. http://www.nih.govNational Library of Medicine. http://www.nlm.govNational Public Health Performance Standards. http://www.cdc.gov/od/ocphp/nphpspOccupational Safety and Health Administration. http://www.osha.govPan American Health Organization. http://ww.paho.orgRobert Wood Johnson Foundation. http://www.rwjf.orgRockefeller Foundation. http://www.rockfound.orgSociety for Public Health Education. http://www.sophe.orgSubstance Abuse and Mental Health Services Administration. http://www.samhsa.govW.K. Kellogg Foundation. http://www.wkk.orgWorld Health Organization. http://www.who.orgU.S. Census Bureau (http://www.census.gov/)U.S. Department of Health & Human Services (www.cdc.gov/nchs/hus.htm)


http://www.cdc.gov/nchs/hus.htm

http://www.census.gov/

http://www.who.org/

http://www.wkk.org/

http://www.samhsa.gov/

http://www.sophe.org/

http://www.rockfound.org/

http://ww.paho.org/

http://www.osha.gov/

http://www.cdc.gov/od/ocphp/nphpsp

http://www.nlm.gov/

http://www.niosh.gov/

http://www.nhlbi.gov/

http://www.neha.org/

http://www.cdc.gov/nchs

http://www.naccho.org/

http://www.cancer.gov/pinkbook

http://www.statehealthfacts.org/

http://www.iuhpe.org/

http://www.healthypeople.gov/state/toolkit/default.htm

http://www.healthypeople.gov/

http://www.hrsa.gov/

http://www.epa.gov/

http://www.ceph.org/

http://ctb.ku.edu/en/

http://www.publichealthlaw.net/

http://ww.cdc.gov/

SCHOOL OF HEALTH AND HUMAN SERVICES REQUIREMENTSGENERAL POLICIES:

Academic Integrity:Students are required to submit their own, original work that is written or developed for this particular class.

Attendance:Students are expected to attend all class sessions. An absence is assessed each time a student is not in attendance during a regularly scheduled class period, whether or not it is an excused absence. An instructor may withdraw a student from class prior to the sixth session in graduate courses if there are more than two unexcused absences. Students who have more than three absences, excused or unexcused, cannot be given a satisfactory grade.

Cell Phones:Cell phones may not be used during class. Students should shut off or switch phones to silent mode.

Laptop Computers:Laptop computers may be used in class for taking notes and accessing materials posted on eCompanion. Instructors may ask students to close computers if use becomes disruptive to learning.

Civility:As a diverse community of learners, students must strive to work together in a setting of civility, tolerance, and respect for each other and for the instructor. Rules of classroom behavior (which apply to online as well as onsite courses) include but are not limited to the following:

Conflicting opinions among members of a class are to be respected and responded to in a professional manner.

Side conversations or other distracting behaviors are not to be engaged in during lectures, class discussions or presentations

There are to be no offensive comments, language, or gestures

Diversity:Learning to work with and value diversity is essential in every degree program. Students are required to act respectfully toward other students and instructors throughout the course. Students are also expected to exhibit an appreciation for multinational and gender diversity in the classroom and develop leadership skills and judgment appropriate to such diversity in the workplace.


Disability:National University complies with the Americans with Disabilities Act of 1990 and Section 504 of the Federal Rehabilitation Act of 1973. If you need accommodations due to a documented disability, please contact the Office of Scholarships and Special Services at (858) 642-8185 or via e-mail at [email protected]. Information received by this office is confidential and is only released on a 'need-to-know' basis or with your prior written consent. Accommodations can only be granted upon approval by the Committee for Students with Disabilities (CSD).

Ethics:Ethical behavior in the classroom is required of every student. Students are also expected to identify ethical policies and practices relevant to course topics.

Plagiarism:Plagiarism is the presentation of someone else's ideas or work as one's own. Students must give credit for any information that is not either the result of original research or common knowledge. If a student borrows ideas or information from another author, he/she must acknowledge the author in the body of the text and on the reference page. Students found plagiarizing are subject to the penalties outlined in the Policies and Procedures section of the University Catalog, which may include a failing grade for the work in question or for the entire course. The following is one of many websites that provide helpful information concerning plagiarism for both students and faculty: http://www.indiana.edu/~wts/pamphlets/plagiarism.shtmlQuoting, Paraphrasing, and Summarizing (http://owl.english.purdue.edu/handouts/research/r_quotprsum.html )

Technology:Students are expected to be competent in using current technology appropriate for this discipline. Such technology may include word processing, spreadsheet, and presentation software. Use of the internet and e-mail may also be required.

The following website provides information on APA writing and citation style required for papers: http://www.nu.edu/LIBRARY/ReferenceTools/citations.html

Writing Standards of SHHS: Students are expected to demonstrate writing skills in describing, analyzing and evaluating ideas and experiences. Written material must follow specific standards regarding citations of authors' work within the text and references at the end of the paper. Students are encouraged to use the services of the University's Writing Center when preparing materials (see National University Services below). Grades will be assigned for written material in accordance with the University catalog and the following general criteria adopted by the School of Health and Human Services.


http://www.nu.edu/LIBRARY/ReferenceTools/citations.html

http://owl.english.purdue.edu/handouts/research/r_quotprsum.html

http://www.indiana.edu/~wts/pamphlets/plagiarism.shtml


NOTE: Letter grades may be assigned for any or all of the following reasons:

"A" Range: Outstanding achievement, significantly exceeds standards Unique topic or unique treatment of topic, takes risks with content; fresh approach. Sophisticated/exceptional use of examples. Original and "fluid" organization; all sentences and paragraphs contribute; sophisticated transitions between

paragraphs. Integration of quotations and citations is sophisticated and highlights the author's argument. Confidence in use of Standard English; language reflects a practiced and/or refined understanding of

syntax and usage. Sentences vary in structure, very few, if any mechanical errors (no serious mechanical errors).

"B" Range: Commendable achievement, meets or exceeds standards for course. Specific, original focus, content well handled. Significance of content is clearly conveyed; good use of examples; sufficient support exists in key areas. Has effective shape (organization), effective pacing between sentences or paragraphs. Quotations and citations are integrated into argument to enhance the flow of ideas. Has competent transitions between all sentences and paragraphs. Conveys a strong understanding of standard English; the writer is clear in his/her attempt to articulate main

points, but may demonstrate moments of "flat" or unrefined language. May have a few minor mechanical errors (misplaced commas, pronoun disagreement, etc.), but no serious

mechanical errors (fragments, run-ons, comma splices, etc.)

"C" Range: Acceptable achievement, meets standards for course Retains overall focus, generally solid command of subject matter. Subject matter well explored but may show signs of underachievement. Significance is understood, competent use of examples. Structure is solid, but an occasional sentence or paragraph may lack focus. Quotations and citations are integrated into argument. Transitions between paragraphs occur but may lack originality. Competent use of language; sentences are solid but may lack development, refinement, style. Occasional minor mechanical errors may occur, but do not impede clear understanding of material. No serious mechanical errors (fragments, run-ons, comma-splices, etc.).

"D" Range: Unsatisfactory achievement; does not meet acceptable standards. Note: The "D" grade is a passing grade for undergraduate work; work that is not of "passing quality" should receive grade "F".

Significance of content is unclear. Ideas lack support, elaboration. Lacks sufficient examples or relevance of examples may be unclear. Support material is not clearly incorporated into argument. Expression is frequently awkward (problematic sentence structure). Mechanical errors may often impede clear understanding of material. May have recurring serious mechanical errors (fragments, run-ons, comma splices, etc.).

"F" Range: Fails to meet minimal standards Ignores assignment. Lacks significance. Lacks coherence. Includes plagiarized material (intentional or unintentional). Lacks focus Difficult to follow due to awkward sentence or paragraph development. Mechanical errors impede understanding. Problems with writing

National University Library:National University Library supports academic rigor and student academic success by


providing access to scholarly books and journals both electronically and in hard copy. Print materials may be accessed at the Library in San Diego or through document delivery for online and regional students. Librarians are available to provide training, reference assistance, and mentoring at the San Diego Library and virtually for online or regional students. Please take advantage of Library resources:

URL: http://www.nu.edu/library. Contact the Library:

o [email protected] o (858) 541-7900 (direct line) o 1-866-NU ACCESS x7900 (toll free)

Use the Library Training Tools (on the Library Homepage) for additional help o Recorded class presentations o Tutorials & Guides (APA/MLA, Peer-Review, and more)



http://www.nu.edu/library

ASSIGNMENT: READING RESPONSES & EXERCISES

You will be responding to various texts and completing written exercises to initiate the process of an evaluation. The assigned readings are outlined in the schedule and the activity sheets provided. Responses will be submitted each class session and do not need to be typed.

EXERCISE 1- 5 POINTSCDC Eval Guide- Step 1- Activity 1A & B

EXERCISE 2- 5 POINTS

CDC Eval Guide- Step 2- Activity 2A & B EXERCISE 3- 5 POINTS

CDC Eval Guide Step 3 - Activity 3 A & B

EXERCISE 4- 5 POINTSCDC Eval Guide Step 4 - Activity 4 A & B QUANTITATIVE CDC Eval Guide Step 5 - Activity 5

EXERCISE 5- 5 POINTSCDC Eval Guide Step 4 - Activity 4 A & B QUALITATIVE CDC Eval Guide Step 5 - Activity 5

EXERCISE 6- 5 POINTSCDC Eval Guide Step 6 - Activity 6A & B


ASSIGNMENT LOGIC MODEL

You will design a logic model as a part of the public health program evaluation document. A logic model presents a picture of how your effort or initiative is supposed to work. It explains why your strategy is a good solution to the problem at hand. In its simplest form, a logic model is a graphic representation that shows the logical relationships between: inputs: resources that go into the program; outputs: activities the program undertakes; outcomes: changes or benefits that result. Effective logic models make an explicit, often visual, statement about what activities will bring about change and what results you expect to see for the community and its people. Having a logic model keeps everyone moving in the same direction, providing a common language and a common point of reference.

The Logic Model should answer these questions:• Does your organization have adequate resources to implement the activities and achieve the

desired outcomes? If you need further resources, is that reflected in your activities? • Have you included all the major activities needed to implement your program and achieve

expected outcomes? Would the activities list enable someone who is unfamiliar with your program to understand its scope?

• Have you expressed your outcomes in terms of change? Have you identified who/what will experience that change, and over what time period?

• Do activities, outputs, and short- and long-term outcomes relate to each other logically (the “if-then” relationship)?

• Does your logic model clearly identify the scope of your program’s influence? • Have you considered a variety of perspectives? It is a good idea to get feedback from colleagues

and stakeholders. (remember, the online logic model builder makes collaboration easy, and gives you a

• Is it visually engaging- simple, yet contains the appropriate degree of detail for the purpose. • Does it provokes thought, triggers questions.• Is it meaningful, does it make sense, is it doable, and can it be verified? Does it provide the

program description that guides the evaluation process; help match evaluation to the program, help know what and when to measure, help focus on key, important information

• Remember, the logic model is a depiction of the program – evaluation needs to match the program. Tailor questions and indicators to particular program and purpose of the evaluation.

• Match evaluation questions to the program – stage of program development and purpose: improvement, assess worth/merit; new knowledge; compliance

• Are there indicators- determine how we will measure it. What would be the indicators for the outcomes and process variables you have selected? How would you know it?

The Logic Model will be a flow chart that includes:

1. Problem statement or situation: The originating problem, or issue, set within a complex of sociopolitical, environmental and economic circumstances. The situation is the beginning point of Logic Model development. Clearly articulates the problem your work is trying to solve. It frames a particular challenge for the population you serve and that your work will try to solve.

2. Goal: Represents a general, big picture statement of desired results. It is the overall purpose of what you are trying to measure (your program, intervention, etc). Answers


what you are trying to accomplish. The answer to this question is the solution to your problem statement, and will serve as your goal. Goals serve as a frame for all elements of the logic model that follow. They reflect organizational priorities and help you steer a clear direction for future action. Goals should include the intended results—in general terms—of the program or initiative and specify the target population you intend to serve.

3. Inputs or Resources: Identify the available resources for your program. This helps you determine the extent to which you will be able to implement the program and achieve your intended goals and outcomes. List the resources that you currently have to support your program. Include such resources as staff, time, money, staff, space, technology, research base, partnership, equipment, and materials. Identify the major resource categories for your program and be specific about these resources.

4. Outputs: What we do and whom we reach: activities, services, events, products and the people reached. Outputs include such elements as workshops, conferences, counseling, products produced and the individuals, clients, groups, families, and organizations targeted to be reached by the activities. Activities are the actions that are needed to implement your program—what you will do with program resources in order to achieve program outcomes and, ultimately, your goal(s). Common activities include: developing products (e.g., promotional materials and educational curricula); providing services (e.g., education and training, counseling or health screening); engaging in policy advocacy (e.g., issuing policy statements, conducting public testimony); or building infrastructure (e.g., strengthening governance and management structures, relationships, and capacity). It is often helpful to group related activities together. The number of activity groups depends on your program’s size and how you administer it. They lead to desired outcomes—benefits for participants, families, communities, or organizations—but are not themselves the changes you expect the program will produce. They do help you assess how well you are implementing the program. Whenever possible, express outputs in terms of the size and/or scope of services and products delivered or produced by the program.

5. Outcomes /Impacts: What results: the value or changes for individuals, families, groups, agencies, businesses, communities, and/or systems. Outcomes express the results that your program intends to achieve if implemented as planned. Outcomes are the changes that occur or the difference that is made for individuals, groups, families, organizations, systems, or communities during or after the program. Outcomes answer the questions: “what difference does the program make? What does success look like?” They reflect the core achievements you hope for your program. Outcomes should: represent the results or impacts that occur because of program activities and services ; be within the scope of the program’s control or sphere of reasonable influence, as well as the timeframe you have chosen for your logic model ; be generally accepted as valid by various stakeholders of the program ; be phrased in terms of change ; be measurable.

6. Chain of Outcomes: Outcomes include short-term, interndiate and long-term benefits. Some outcomes must occur before others become possible. This is referred to as the “chain of outcomes.” Short-term outcomes: what change do you expect to occur either immediately or in the near future? Short-term outcomes are those that are the most direct result of a program’s activities and outputs. They typically focus on changes in awareness, knowledge, skills, attitudes, opinions and intent. Intermediate outcomes: what change do you want to occur after that? Intermediate outcomes are those outcomes that link a program’s short-term outcomes to long-term outcomes. They typically include changes in behaviors, decision-making or policy change, and actions.


Long-term outcome: what change do you hope will occur over time? Long-term outcomes are those that result from the achievement of your short- and intermediate-term outcomes. Often called impact, they include such as changes in social, economic, civic, and environmental conditions. Often long-term outcomes will occur beyond the timeframe you identified for your logic model.

7. Assumptions: The assumptions are the beliefs, principles, ideas we have about the program, the people involved and the way we think the program will operate. Assumptions underlie all that we do. Assumptions that underlie a program’s theory are conditions that are necessary for success, and you believe are true. Assumptions include our ideas about the problem or situation; the way the program will operate; what the program expects to achieve; how the participants learn and behave, their motivations, etc.; the resources and staff; the external environment; the knowledge base; and the internal environment. Your program needs these conditions in order to succeed, but you believe these conditions already exist – they are not something you need to bring about with your program activities. In fact, they are not within your control. These assumptions can refer to facts or special circumstances in your community, region, and/or field. Faulty assumptions are often the reason for poor results.

8. External Factors: Aspects external to the program that influence the way the program operates, and are influenced by the program. Dynamic systems interactions include the cultural milieu, Biophysical environment, economic structure, housing patterns, demographic makeup, family circumstances, values, political environment, background and experiences of participants, media, policies and priorities, etc. These are elements that affect the program over which there is little control.

See Template in e-College


RUBRIC LOGIC MODEL

Marginal 0

Acceptable1-2

Commendable3-4

Outstanding 5

1. Problem statement or situation

X .5

Limited or not complete

Although problem articulated, minimally addressed, generic Not well detailed, -demonstrates thought.

The originating problem articulated. Frames a particular challenge for the population that work will try to solve. Detailed, -demonstrates thought.

The originating problem set within sociopolitical, environmental and economic circumstances. Clearly articulates the problem trying to solve. Frames a particular challenge for the population that work will try to solve. Complete, detailed, -demonstrates thought.

2. Goal X 1

Limited; unclear . Stated yet is imprecise.

Imprecise.-some information expressed. Somewhat connected and logical. Demonstrates understanding.

Mostly specific and connected, clear and logical. Demonstrates understanding.Articulated

Specific and connected, clear and logical. Clearly and precisely stated, thorough, & logical. Demonstrates clear understanding.Well articulated.

3. Input or Resources

X 1

Generic description - all areas not addressed.

Identifies some of the available resources for the program. Includes some resources . Identifies some major resource categories..Explained & represented in flowchart, map, or table. Some understanding is evident. Needs some clarification.

Identifies most of the available resources for the program. Includes most resources. Identifies most major resource categories.. Explained & represented in flowchart, map, or table. Demonstrates understanding.Purpose and description is evident.

Identifies the available resources for the program. Includes multiple resources such as staff, time, money, staff, space, technology, research base, partnership, equipment, and materials. Specifically identifies the major resource categories. Explained in detail and fully represented in flowchart, map, or table. Demonstrates clear understanding. Purpose and description is evident and well articulated.

4. Outputs X 1

Unclear and/or have some ambiguity. Detail is needed to provide exactness in meaning.

Includes some outputs- does not address activities and people. Mostly represented in flowchart, map, or table. Demonstrates minimal understanding.Does not address all areas. All information not directly related to the topic.

Includes outputs including activities and people. Outputs address most desired outcomes. Addressed with clarity and appropriateness. Represented in flowchart, map, or table. Demonstrates understanding. Purpose and description is evident.

Includes variety of outputs including activities and people. Address activities, products services, policy advocacy, & infrastructures. Outputs lead to desired outcomes. Specific, addressed with clarity and appropriateness. Explained in detail and fully represented in flowchart, map, or table. Demonstrates clear understanding. Purpose and description is evident and well articulated.

5. Outcomes /Impacts

6. Chain of Outcomes

X 1.5

Unclear or not stated. Detail is needed to provide exactness in meaning. Does not portray the sequence of outcomes.

Addresses some of the results the program intends to achieve. Outcomes somewhat reflect the core achievements; the impacts that occur- & are sometimes measurable. Includes short-term, intermediate and long-term benefits. Minimal chain of outcomes. Mostly represented in flowchart, map, or table.

Addresses most of the results the program intends to achieve . Outcomes mostly reflect the core achievements; represent the results or impacts that occur; phrased in terms of change; and are mostly measurable. Includes short-term, intermediate and long-term benefits. There is a chain of outcomes represented in flowchart, map, or table.

Clearly expresses the results the program intends to achieve if implemented as planned. Outcomes reflect the core achievements; represent the results or impacts that occur; are within the scope of the program’s control and timeframe; phrased in terms of change; and are measurable. Includes clear short-term, intermediate and long-term benefits. There is a clear chain of outcomes-discussed in detail, and fully represented in flowchart, map, or table. Demonstrates clear understanding. Well articulated.

7.Assumptions X.5

Unclear or not stated. Detail is needed to provide exactness in

Provides generic assumptions that do not fully reflect the beliefs, principles, ideas about the

Addresses assumptions that reflect the beliefs, principles, ideas about the program, the people

Clear assumptions that reflect the beliefs, principles, ideas about the program, the people involved and the way the program will operate.


meaning. All areas not addressed

program, the people involved and the way the program will operate. analysis and synthesis, and interpretations. Does not clearly link to problem or goal. Minimal detail .

involved and the way the program will operate. Addresses some ideas about the problem ; the way the program operates& expects to achieve; how the participants learn and behave; the staff & resources; the external and the internal environment. Lacks consistent detail .

Addresses multiple ideas about the problem or situation; the way the program will operate; what the program expects to achieve; how the participants learn and behave, their motivations, etc.; the resources and staff; the external environment; the knowledge base; and the internal environment. Consistently relevant detail makes the meaning exact

8.External Factors

X.5

Unclear or not stated. Detail is needed to provide exactness in meaning. All areas not addressed.

Minimally identifies aspects external to the program . Few elements that affect the program over which there is little control shared. Generic ideas.

Identifies aspects external to the program . Some elements that affect the program over which there is little control shared. Addresses limitations, scope.

Clearly identifies aspects external to the program . Dynamic elements that affect the program over which there is little control shared. Addresses limitations, scope, in detail.

Total 30 POSSIBLE

Comments


Survey Questionnaire Project

1. Plan these Steps in the Survey Questionnaire Project a. Establish the goals of the questionnaire - What you want to learn b. Determine your audience/ sample - Whom you will interview c. Choose interviewing methodology - How you will interview d. Create a letter to accompany the questionnairee. Create your questionnaire - What you will ask f. Determine how you will collect and analyze the data- What does it meang. Determine how you will use the data- How will you use the data

2. In a short paper, utilizing the following headings, explain your process and plan

GoalsAudience & SampleMethodologyCollection and analysis of DataUse of data

3. Appendix – attached final copies of the following

Letter Questionnaire


RUBRIC – SURVEY QUESTIONNAIRE PROJECT

Criteria & scoring

Marginal 1

Acceptable2

Commendable3

Outstanding 4

Audience X .5

Limited understanding of the audience

Some understanding of the audience

Clear understanding of the audience

Thorough understanding of the audience

Purpose X .5

Limited; lack of understanding of use and dissemination - not clear. Imprecise.

Imprecise.-some information expressed clearly and critiqued. Some supporting evidence is used.

Clearly stated. Analysis is significant. Information expressed clearly and analyzed. Supporting evidence.

Information is clearly and precisely stated. Analysis is thorough, logical, and significant. Information expressed clearly and critiqued with supporting evidence.

Methodology & letter

x.5

No clear letter or plan for administration. Missing clear introduction and/or conclusion

Generic letter or plan for administration Topics and ideas discussed somewhat randomly. Missing clear introduction and/or conclusion.

Clear letter and plan for administration Logical paragraph order, good development of ideas in clear structure. Introduction and/or conclusion.

Clear letter and detailed plan for administration Very well organized, good development of ideas, strong sentences, effective. Clear introduction and/or conclusion.

SurveyDevelopment

Survey questions do not present enough information to assess.

Survey questions are limited and offer some information required to assess.

Survey items sufficiently cover important items for assessment

Survey items are well thought out and include all important items to adequately assess.

Survey Format

Survey questions are not appropriate for requested information. Format is sloppy, margins, layout is inconsistent.

Survey questions are not listed or asked in a logical format. Format has some inconsistencies and missing areas.

Survey questions are asked in logical sequence which respondents can follow. The format is correct, title is appropriate.

Survey questions are asked in a logical and well thought out sequence which respondents can follow. The format is correct, title appropriate, and laid out wisely.

Survey Mechanics

Writing is unstructured, inconsistent. Many spelling and punctuation errors, many sentence fragments, run-ons.

Writing lacks clearly defined organization, ineffective Some careless spelling and punctuation errors- 4 or more misspellings and/or grammatical errors

Logical writing, clear and functional, clear structure. Few spelling and punctuation errors -no more than two misspellings and/or grammatical errors

Very well organized, strong sentences. Flawless spelling, punctuation, and capitalization.

Collection, analysis & usex.5

No clear or limited plan for collection, analysis or use

Generic plan for collection, analysis or use. Topics and ideas discussed somewhat randomly.

Clear plan for collection, analysis and use. Logical order and clear structure.

Clear l and detailed plan for collection, analysis and use . Very well organized, effective.

Total Possible 20

Comments


ASSIGNMENT PROGRAM EVALUATION PLAN

You will design a Public Health Program Evaluation Document. The document will be a continuation of the grant proposal students developed in COH 607 or an alternative topic. Be sure to develop an evaluation plan to ensure your program evaluations are carried out efficiently in the future and to ensure your evaluation plan is documented so you can regularly and efficiently carry out your evaluation activities.

Plans must include the following sections:I. Title Page (name of the organization that is being, or has a

product/service/program that is being, evaluated; date)

II. Table of Contents

III. Executive Summary (one-page, concise program background , type of evaluation conducted, what decisions are being aided by the findings of the evaluation, who is making the decision

IV. Engagement of Stakeholders All groups identified - those involved in program operations; those served or affected

by the program; and primary intended users of the evaluation. Address rights of human subjects, human interactions, conflict of interest Cultural competency addressed

V. Description of the Program Statement of need -describes the problem, goal, or opportunity that the program

addresses; the nature of the problem or goal, who is affected, how big it is, and whether (and how) it is changing; problem/opportunity to which program is responding to, the program’s specific objectives

Expectations - program's intended results-what the program has to accomplish to be considered successful; background about organization and program that is being evaluated; organization description/history.

Activities - everything the program does to bring about changes- describe program components, elements, strategies, and actions; principal content of the program, delivery model

Resources- include the time, talent, equipment, information, money, and other assets available to conduct program activities include program costs and the cost-benefit ratio as part of the evaluation; staffing (description of the number of personnel and roles in the organization that are relevant to developing and delivering the program

Program's stage of development - reflects program maturity; address three phases of development: planning, implementation, and effects or outcomes; program documentation.

Program's context - the environment in which the program operates; the area's history, geography, politics, and social and economic conditions, and what other organizations have done.

Logic model - sequence of events input, output, and long-term goals, short-term; flow-chart, map, or table to portray the sequence of steps leading to program results; clear description of program inputs, activities/processes; clear description of outcomes and impact.


VI. Evaluation Design Purpose - general intent of the evaluation. (gain insight, improve how things get done,

determine what the effects of the program are, affect those who participate) Users - specific individuals who will receive evaluation findings. Uses - what will be done with what is learned from the evaluation. Answer specific questions- clarity and appropriateness of research questions, including

which stakeholders will utilize the answers Methods-(experimental, quasi-experimental, and observational or case study designs) Agreements- summarize the evaluation procedures and clarify everyone's roles and

responsibilities; describe how the evaluation activities will be implemented. Addresses evaluation impact ,practical procedures, political viability, cost effectiveness,

service orientation, complete and fair assessment, fiscal responsibility .

VII. Gathering Evidence Indicators - general concepts about the program and its expected effects into specific,

measurable parts; description of independent and dependent variables and how they will be measured and analyzed.

Sources of evidence -people, documents, or observations, criteria used to select sources clearly stated; who participated.

Quality -the appropriateness and integrity of information gathered Quantity-the amount of evidence gathered Logistics - the methods, timing, and physical infrastructure for gathering and handling

evidence time periods sampled, data collection method and tools, any limitations caused by this method and tool

Address Information scope and selection; information sources, valid and reliable information

VIII. Justification of Conclusions Standards -the values held by stakeholders about the program. Analysis and synthesis -methods to discover and summarize an evaluation's findings;

detect patterns in evidence, by analysis, synthesis, mixed method Interpretation - figure out what the findings mean; interpretations and conclusions Judgments - statements about the merit, worth, or significance of the program, compare

against one or more selected standards. Recommendations-actions to consider as a result of the evaluation; recommendations

regarding the decisions that must be made about the service/program. Addresses identification of values, analysis of quantitative and qualitative information

IX. Use and Dissemination of Lessons Learned Design -how the evaluation's questions, methods, and overall processes are

constructed. Preparation - steps taken to get ready for the future uses of the evaluation findings. Feedback - communication that occurs among everyone involved in the evaluation. Follow-up - the support that users need during the evaluation and after they receive

evaluation findings. Dissemination - the process of communicating the procedures or the lessons learned

from an evaluation to relevant audiences in a timely, unbiased, and consistent fashion.

X. Reflection on Standards for "good" evaluationsUtility Standards


1. Stakeholder Identification: People who are involved in (or will be affected by) the evaluation should be identified, so that their needs can be addressed. 2. Evaluator Credibility: The people conducting the evaluation should be both trustworthy and competent, so that the evaluation will be generally accepted as credible or believable. 3. Information Scope and Selection: Information collected should address pertinent questions about the program, and it should be responsive to the needs and interests of clients and other specified stakeholders. 4. Values Identification: The perspectives, procedures, and rationale used to interpret the findings should be carefully described, so that the bases for judgments about merit and value are clear. 5. Report Clarity: Evaluation reports should clearly describe the program being evaluated, including its context, and the purposes, procedures, and findings of the evaluation. This will help ensure that essential information is provided and easily understood. 6. Report Timeliness and Dissemination: Significant midcourse findings and evaluation reports should be shared with intended users so that they can be used in a timely fashion. 7. Evaluation Impact: Evaluations should be planned, conducted, and reported in ways that encourage follow-through by stakeholders, so that the evaluation will be used.

Feasibility Standards 1. Practical Procedures: The evaluation procedures should be practical; to keep disruption of everyday activities to a minimum while needed information is obtained. 2. Political Viability: The evaluation should be planned and conducted with anticipation of the different positions or interests of various groups. This should help in obtaining their cooperation so that possible attempts by these groups to curtail evaluation operations or to misuse the results can be avoided or counteracted. 3. Cost Effectiveness: The evaluation should be efficient and produce enough valuable information that the resources used can be justified.

Propriety Standards 1. Service Orientation: Evaluations should be designed to help organizations effectively serve the needs of all of the targeted participants. 2. Formal Agreements: The responsibilities in an evaluation (what is to be done, how, by whom, when) should be agreed to in writing, so that those involved are obligated to follow all conditions of the agreement, or to formally renegotiate it. 3. Rights of Human Subjects: Evaluation should be designed and conducted to respect and protect the rights and welfare of human subjects, that is, all participants in the study. 4. Human Interactions: Evaluators should respect basic human dignity and worth when working with other people in an evaluation, so that participants do not feel threatened or harmed. 5. Complete and Fair Assessment: The evaluation should be complete and fair in its examination, recording both strengths and weaknesses of the program being evaluated. This allows strengths to be built upon and problem areas addressed. 6. Disclosure of Findings: The people working on the evaluation should ensure that all of the evaluation findings, along with the limitations of the evaluation, are accessible to everyone affected by the evaluation, and any others with expressed legal rights to receive the results. 7. Conflict of Interest: Conflict of interest should be dealt with openly and honestly, so that it does not compromise the evaluation processes and results. 8. Fiscal Responsibility: The evaluator's use of resources should reflect sound accountability procedures and otherwise be prudent and ethically responsible, so that expenditures are accounted for and appropriate.

Accuracy Standards


1. Program Documentation: The program should be described and documented clearly and accurately, so that what is being evaluated is clearly identified. 2. Context Analysis: The context in which the program exists should be thoroughly examined so that likely influences on the program can be identified. 3. Described Purposes and Procedures: The purposes and procedures of the evaluation should be monitored and described in enough detail that they can be identified and assessed. 4. Defensible Information Sources: The sources of information used in a program evaluation should be described in enough detail that the adequacy of the information could be assessed. 5. Valid Information: The information gathering procedures should be chosen or developed and then implemented in such a way that they will assure that the interpretation arrived at is valid. 6. Reliable Information: The information gathering procedures should be chosen or developed and then implemented so that they will assure that the information obtained is sufficiently reliable. 7. Systematic Information: The information from an evaluation should be systematically reviewed and any errors found should be corrected. 8. Analysis of Quantitative Information: Quantitative information - data from observations or surveys - in an evaluation should be appropriately and systematically analyzed so that evaluation questions are effectively answered. 9. Analysis of Qualitative Information: Qualitative information - descriptive information from interviews and other sources - in an evaluation should be appropriately and systematically analyzed so that evaluation questions are effectively answered. 10. Justified Conclusions: The conclusions reached in an evaluation should be explicitly justified, so that stakeholders can understand their worth. 11. Impartial Reporting: Reporting procedures should guard against the distortion caused by personal feelings and biases of people involved in the evaluation, so that evaluation reports fairly reflect the evaluation findings. 12. Metaevaluation: The evaluation itself should be evaluated against these and other pertinent standards, so that it is appropriately guided and, on completion, stakeholders can closely examine its strengths and weaknesses.


RUBRIC - EVALUATION PLAN

Marginal 1

Acceptable2

Commendable3

Outstanding 4

Title page, Contents & SummaryX1.5

Title Page, Table Of Contents And Executive Summary not complete

Title Page, Table Of Contents And Executive Summary minimally addressed, generic

Title Page, Table Of Contents And Executive Summary completely addressed - all items covered in detail

Title Page, Table Of Contents And Executive Summary complete, detailed, summary demonstrates thought.

StakeholdersX2

Limited; stakeholders appear to be understood but some links are not often evident. Stated yet is imprecise.


Clearly stated. Analysis is significant. Information expressed clearly and analyzed. Most have supporting evidence. Stakeholders appear to be understood but some links are not often evident.

Specific and connected, clear and logical. Information is clearly and precisely stated. Analysis is thorough, logical, and significant. Information on stakeholders are expressed clearly and critiqued with supporting evidence.

DescriptionX2

Generic description - all areas not addressed.

Statement of need, expectations, activities, resources, stage of development and program context addressed. Logic model explained and represented in flowchart, map, or table. Some understanding is evident.Needs some clarification.

Statement of need, expectations, activities, resources, stage of development and program context addressed. Logic model explained and fully represented in flowchart, map, or table. Demonstrates understanding.Purpose and description is evident and articulated.

Statement of need, expectations, activities, resources, stage of development and program context addressed. Logic model explained in detail and fully represented in flowchart, map, or table. Demonstrates clear understanding.Purpose and description is evident and well articulated.

Evaluation DesignX2

Unclear and/or have some ambiguity. Accuracy of statements is not clear. Detail is needed to provide exactness in meaning.

Purpose, users, uses described. Specific questions addressed. Explains some methods and agreements, mentions evaluation activities .Does not addresses all areas: evaluation, impact procedures, effectiveness, and viability All information not directly related to the topic

Purpose, users, uses described. Specific questions addressed with clarity and appropriateness. Mostly explains methods and agreements, describes how the evaluation activities will be implemented. Addresses evaluation, impact procedures, effectiveness, and viability .

Purpose, users, uses described. Specific questions addresses with clarity and appropriateness. Completely explains methods and agreements, describes how the evaluation activities will be implemented. Addresses evaluation, impact procedures, effectiveness, and viability in detail.Clearly addresses .

Gathering EvidenceX2

Unclear or not stated. Supporting evidence is often missing. Detail is needed to provide exactness in meaning.

Indicators & sources of evidence mostly addressed. Quality, quantity, and logistics vaguely discussed in detail. Limitations, scope, and selection mentioned. Supporting evidence is usually present

Indicators & sources of evidence addressed. Quality, quantity, and logistics discussed . Addresses limitations, scope, and selection . Relevant examples explain most ideasSupporting evidence is present

Indicators & sources of evidence clearly addressed. Quality, quantity, and logistics discussed in detail. Addresses limitations, scope, and selection in detail. Supporting evidence is present. Consistently relevant detail makes the meaning exact

Justifications & Conclusions X1

Unclear or not stated. Supporting evidence is often missing. Detail is needed to provide exactness in meaning. All areas not addressed Recommendation weak.

Addresses standards. Provides generic analysis and synthesis, and interpretations. Does not clearly substantiate judgments. Provides basic recommendations not fully linked to evaluation

Addresses standards. Provides analysis and synthesis, some interpretations and does substantiate judgments. Provides recommendations based on evaluation

Addresses standards. Provides clear analysis and synthesis, solid interpretations and substantiate judgments. Provides detailed recommendations based on evaluation

Use & Dissemination x1

Unclear or not stated. Supporting evidence is missing. Detail is needed to provide exactness in meaning. All areas not

Minimally identifies uses in design and preparation. Provides some feedback. Lists basic follow-up ideas. Generic ideas for

Completely identifies uses in design and preparation. Delineates feedback among everyone involved in the evaluation. Lists

Clearly identifies uses in design and preparation. Delineates detailed feedback among everyone involved in the evaluation. Lists strong


addressed. dissemination. substantiated follow-up ideas. Relevant ideas for dissemination.

follow-up ideas. Creative and relevant ideas for dissemination.

Reflection on StandardsX2

Mentions without reflection. No demonstration of understanding and application

Addresses some standards; Utility, Feasibility, Propriety, and Accuracy. Reflects generically with little understanding and application

Addresses all standards; Utility, Feasibility, Propriety, and Accuracy. Reflects on all components addressing strengths and weaknesses, Demonstrates an understanding and application

Fully addressees all standards; Utility, Feasibility, Propriety, and Accuracy. Reflects on each component addressing strengths and weaknesses, Demonstrates clear understanding and application

OrganizationMechanics and Grammar

X .75

Some sources are accurately cited in body and reference section using current APA style. Headings inconsistent and do not follow APA format

Some sources are accurately cited in body and reference section using current APA style. Some Headings follow APA format

Most sources are accurately cited, in the body and reference section using current APA style. Most Headings follow APA format

All sources are accurately cited in the body and reference section using current APA style. Headings follow APA format

APA Format

X .75

Writing is unstructured, inconstant paragraph order, transitions missingMissing clear introduction and/or conclusion. Many spelling and punctuation errors, many sentence fragments, and run-ons.

Topics and ideas discussed somewhat randomly, writing lacks clearly defined organization, ineffective paragraph orders, and few transitions Missing clear introduction and/or conclusion. Some careless spelling and punctuation errors, some errors in sentence structure

Logical paragraph order, clears and functional transitions, good development of ideas in clears structure . Introduction and conclusion. Few spelling and punctuation errors, sentences structure mostly correct

Very well organized, good development of ideas, strong sentences, effective and varied transitions. Clear introduction and conclusion.Flawless spelling, punctuation, and capitalization, varied and interesting sentence structure.

Total 60 possible

Comments


ASSIGNMENTEVALUATION PLAN PRESENTATION

Class teams will present their Program Evaluation Plan in presentations to the entire

class. A PowerPoint Presentation will be used to share information about the

Evaluation Plan. Create a visual PowerPoint presentation that identifies and explains

your Evaluation Plan.

The presentation may be only 15-20 minutes in length.

Presentation must include

Engagement of Stakeholders Description of the Program Evaluation Design Gathering Evidence Justification of Conclusions Use and Dissemination of Lessons Learned Summary

Peer evaluations will be conducted on the presentations.


RUBRIC - EVALUATION PLAN PRESENTATIONCriteria & scoring

Marginal 1

Acceptable2

Commendable3

Outstanding 4

StakeholdersX1

Limited; stakeholders appear to be understood but some links are not often evident. Imprecise.




DescriptionX1

Limited; description limited & imprecise.




Evaluation DesignX1

Limited; design lacking all components Imprecise.




Gathering EvidenceJustifications & Conclusions

X1

Limited; gathering evidence and justifications limited . Imprecise.




Use & Dissemination X1

Limited; lack of understanding of use and dissemination - not clear. Imprecise.




Presentation X2.5

Knowledge and comprehension are evident.Organization is lacking.Details should be added.Greater clarity or accuracy is needed in the explanation within some slides.Background and graphics distract from the presentationDelivery techniques (posture, gesture, eye contact, and vocal expressiveness) detract from the understandability of the presentation, and speaker appears uncomfortable.

Knowledge and comprehension are evident.Organization is evident.Most ideas are clearly identified, interpreted, applied, explained and/or critiqued Background and graphics do not support the presentationDelivery techniques (posture, gesture, eye contact, and vocal expressiveness) make the presentation understandable, and speaker appears tentative.

Some higher order thinking is evidentOrganization is evident. Most ideas are clearly identified, interpreted, applied, explained and/or critiquedAccurate explanation issues and interventions shared.Background and graphics support the presentationDelivery techniques (posture, gesture, eye contact, and vocal expressiveness) make the presentation interesting, and speaker appears comfortable.

Higher order thinking is evident and presented in a clear logical flow.Insightful and thorough interpretation, application, explanation, and/or critique of ideas.Clear, precise, accurate, and detailed explanation of issues and interventionsBackground and graphics enhance the presentationDelivery techniques (posture, gesture, eye contact, and vocal expressiveness) make the presentation compelling, and speaker appears polished and confident.

Total Possible 30

Comments


Documents

COH 609 dec 2011