EVALUATION OVERVIEW:Key Questions and Roles

Mary L. Fennell, Ph.D. (EOC Chair)Professor and Chair, Brown University June 19, 2007

NCCCP: Overall Mission & Model:

• …a national network of community cancer centers that will expand cancer research and deliver state-of-the-art care to a greater number of Americans in the communities where they live;

• …a three year pilot program that will extend NCI reach into more states, cities, towns, rural areas, inner-city areas, focusing on…• Delivering high-quality care to more people• Increase participation in clinical trials• Reduce cancer healthcare disparities

• …allow for expanded collection, storage, sharing of blood and tissue samples needed for research;

• …explore the utility of a national database of electronic medical records (link to caBIG).

The LAUNCH…

• The NCCCP is now fully planned and made public;

• Sites have been selected; work is beginning;

• An Evaluator is about to be chosen…

• BUT: we’ve already been at work on the Evaluation Plan, and have a structure in place to oversee it….

Evaluation Overview: the Organizations Involved:

• The Evaluation Oversight Committee:– Mary L. Fennell, PhD (chair)– Thomas E. Gribben, MD– Mark C. Hornbrook, PhD– Timothy P. Johnson, PhD– Bryan J. Weiner, PhD– Jane Zapka, ScD

• The EOC exchanges information and communicates with the NPAC; both report to Maureen Johnson, NCCCP Project Officer

• EOC connects with SAIC through Joy Beveridge, and to consultant, Arnold Kaluzny…

Evaluation Structure….continued

• …and the EOC is linked to the NCI RFP Development Subcommittee (Johnson, O’Brien, Chu, Clauser, Kaluzny).

• Separate contract to an independent program evaluator:– SOW specifies both qualitative and quantitative data

collection and analysis…– As well as a set of key questions that must be

addressed.

Evaluation Research in General….

• Uses a variety of sampling and data collection techniques in order to – Document the history of program development;– Assess:

• the process of program building,• the impact/outcomes of the program itself;• program costs, future feasibility, sustainability.

– Provide feedback to NCI, pilot projects, and other stakeholders groups.

How Do We Build an Evaluation of This Important Work?

• The evaluation of this work must itself be:– “State of the art” evaluation methods;– Designed and monitored by independent

experts (separate evaluator and EOC);– Match complexity of pilot activities by broad

mix of evaluation expertise (EOC membership);

– Adopt an interdisciplinary framework as well as a multiple-methods/mixed methods model.

NCCCP: Key Research Questions: We expect the pilot sites will demonstrate improvement in…

1. Physician participation in NCI clinical trials– Especially early phase trials– Especially minority patients

2. Knowledge of infrastructure requirements, needed interfaces, specific components of caBIG for community hospital-based cancer programs;

3. Knowledge of infrastructure/policies/costs needed for biospecimen collection, annotation and storage;

4. Reduction of disparities in cancer care delivery…

Key questions….continued:Improvement should be demonstrated in…

5. Quality of care throughout the cancer care continuum, by use of:

– Multidisciplinary team care; – Improved care coordination, from prevention and

screening through survivorship and end-of-life care;– Measurable quality improvement in a specific cancer

site, throughout the care continuum;– Enhanced patient experience with care.

6. Within each pilot site, there should be increased top management support of cancer care and research efforts.

Additional Pilot Activities to be Evaluated:

• Determine how enhanced use of IT/EMRs can contribute to program aims;

• Improve connection of pilot program to other community-based cancer programs and resources;

• Assess cost, feasibility and sustainability of the NCCCP model.

On to Dr. Steve Clauser….

OVERVIEW OF EVALUATION CONTRACTOR ROLE AND COLLABORATIONS…