The National Human Services Training Evaluation Symposium:

- A 10 Year Retrospective -

10 Yrs of Picturesque Settings

The Faculty Club

10 Years of Enlightening Presentations

Looking back and moving forward……..

1998: The First

In the beginning

1999: The 2nd Annual

Keynote: Bernadette McCarthy—Using Training Evaluation to Improve Services

Program Examined: Training outcomes: performance monitoring Delphi Techniques Nominal Group Techniques Knowledge testing Promoting & evaluating skill development Using training to influence policy Levels of competence TOL

2000: The 3rd Annual

Kirkpatrick’s model, but what about Level Four?

Keynote: Icebreaker on the first evening instead Program examined:

Evaluation of Interdisciplinary training: Child Welfare workers and supervisors

Strategic training evaluation Embedded evaluation TOL/measuring behavioral change Skills assessment

2000

2001: The 4th Annual

The year of “Affinity Action Groups”

Keynote: Dave Wegenast on “Return on Investment” of the Evaluation Symposium

Program examined: Assessing underlying competencies Models for evaluating core curriculum Evaluation of statewide training program The logic model Integration of TOL Evaluating group behavior Blackboard—E Learning

2001

2002: The 5th Annual

Outcomes in Child Welfare

Keynote: Mark Courtney–Evaluation in Child Welfare Services: Big Questions

Program examined: Training Evaluator Competency SPrA CPS system Compliance with practices standards Needs assessments

2002

2003: The 6th Annual

Measuring outcomes, the long and winding road…

Keynote: Elena Cohen—Getting the whole picture: The CFSR, Training and Frontline Practice Links

Program Examined: Chain of evidence Panel presentation on each of Kirkpatrick’s levels Project briefings

2003

2004: The 7th Annual

Improving Evaluation Capabilities

Keynote: Wayne Stevenson—Training as an Avenue to Support Practice Improvement

Program Examined: National Frameworks: National Evaluation of Child Welfare

Training & North American Certification Project TOL Staff Retention Organizational Culture

2004

2005: The 8th Annual

Expanding our horizons: Enhancing current methods, creating new methods…

Keynote: Eileen Gambrill Program Examined:

Future trends in child welfare training Addressing fairness & equity in training evaluation Training evaluation in large systems Linking training outcomes to children and families Instruments to measure skill acquisition Practice assessments in a large system

2005

2006 – The International House

2006: The 9th Annual

Evidence-based training: What does it mean when we say training

is effective?

Keynote: Kurt Kraiger – Alternative Frameworks for Training Evaluation

Program Examined: How evaluation brings attention to gaps in training Teaching & training methods that contribute to knowledge

retention Logic models Measuring knowledge/skill Evaluating assumptions, values and beliefs

2006

2007: The Berkeley City Club

2007: 10th Annual

Looking back and moving forward…

Keynote: Peter Watson—Training Evaluation as a tool for organizational development

Program Examines: Findings from the National Evaluation of Child Welfare

grants Measuring TOL Revisiting Level 1 The future of training evaluation Using tests to predict job performance Evaluation of training content

Lessons Learned…(NSDTA Journal Article)

1. You can teach a turkey to climb a tree, but it's easier to hire a squirrel.

2. Effective utilization of training is influenced by individual, organizational, and training design factors.

3. Training evaluation can help administrators differentiate between training and non-training solutions.

Lessons Learned, cont’d

4. We know what we don’t know.

5. Sometimes the training evaluation “tail” wags the training content “dog”.

6. Be careful what you wish for, the focus on outcomes cuts both ways.

THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT

THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT

Increased funding, value, and visibility of training and development

THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT

Increased funding, value, and visibility of training and development

Expanded roles and functions (e.g., NSDTA nine roles and competencies – the advent of the HSTD consultant) – HSTD as a career

THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT

Increased funding, value, and visibility of training and development

Expanded roles and functions (e.g., NSDTA nine roles and competencies – the advent of the HSTD consultant) – HSTD as a career

Increased accountability of child welfare/human services

THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT

Increased funding, value, and visibility of training and development

Expanded roles and functions (e.g., NSDTA nine roles and competencies – the advent of the HSTD consultant) – HSTD as a career

Increased accountability of child welfare/human services

Introduction of litigation into the HSTD arena

THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT

An increased understanding of the importance of transfer of learning without a substantial increase in evaluation efforts beyond the reaction/satisfaction level

THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT

An increased understanding of the importance of transfer of learning without a substantial increase in evaluation efforts beyond the reaction/satisfaction level

Technology – An increase in distance models and alternative delivery systems

THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT

An increased understanding of the importance of transfer of learning without a substantial increase in evaluation efforts beyond the reaction/satisfaction level

Technology – An increase in distance models and alternative delivery systems

Agency-University partnerships

THE CHANGED NATURE OF HUMAN SERVICES TRAINING AND DEVELOPMENT

An increased understanding of the importance of transfer of learning without a substantial increase in evaluation efforts beyond the reaction/satisfaction level

Technology – An increase in distance models and alternative delivery systems

Agency-University Partnerships Development of the field and emerging profession of

HSTD (e.g., Code of Ethics, roles and competencies, journal, landmark events 10 years NHSTES & 20 years NSDTA conference)