July 2017 Revision

14 | P a g e

APPENDIX 2 - ACADEMIC DAY INFORMATION Slides from Academic Day presentation outlining the expectations of the audit:

1

Family Medicine Scholarly Projects

Alexander Singer MB BAO BCh CCFPQI/Informatics Stream LeadAssociate Professor

Part 1Defining the Requirements of the Scholarly Project

R1 Scholarly Project• Written paper and presentation

• Completed as small group (2-3)

• Done during Family Medicine Block Time (FMBT)

• Quality Improvement (QI) using chart audit

• Project must be unique to stream• The same topic cannot be repeated within your stream in the same

year. Get approval from your Education Director/Stream Lead

• Relevant to Family Medicine and your stream

2

R1 Scholarly Project• Identify a potential “care gap”• Explain and justify “benchmarks” for “best care” derived

from:• Literature review & collaboration with your health care team

• Define your population-of-interest• Outline your EMR search/chart audit strategy• Use findings to develop a QI plan

• What SYSTEM changes must occur to move towards benchmark?• Who needs to be involved in the change?• Barriers to implementation?

R1 Scholarly Project• Expectations – Written Paper

• Title page• Introduction• Methods• Results• Discussion• References and Appendix

• Expectations – Oral Presentation• Group presentation at home clinic• 20-30 minutes with 5 minutes for questions

3

R2 Scholarly Project

• Written paper and presentation

• Completed individually

• QI or Research based on chart audit

• Project must be unique to stream

• Relevant to Family Medicine and local site

R2 Scholarly Project - QI• Identify “care gap”• Explain and justify benchmarks

– Literature review, collaboration with health care team• Define population• Outline search strategy• Conduct chart audit• Use findings to develop QI plan

– How do findings compare to benchmarks?– What system changes need to be put in place to

encourage practice improvement

4

R2 Scholarly Project - Research• Identify suspected “care gap”• Conduct chart audit for preliminary data

– Proof for further study• Identify specific research question• Conduct literature review for background• Describe population• Define data sources and research methods• Proposed Statistical Analysis & rationale• Knowledge Translation plan

R2 Scholarly Project• Expectations – Written Paper

– QI• Title page, Introduction, Methods, Results,

Discussion, References and Appendix– Research

• ICMJE format–Abstract, Introduction, Methods, Results,

Discussion, References, Tables and Illustrations

5

R2 Scholarly Project• Expectations – Oral Presentation

– Individual presentation at end of R2 year• FM Scholar Day – May, 2017

– 10-minutes with 5 minutes for questions

• Must present on FM Scholar Day and if not it is YOUR responsibility to make alternative arrangements!

Completion of R1 and R2 projects are a program requirement for graduation!

Facilitate the education of patients, families, trainees, other health professional colleagues, and the public, as appropriate

Contribute to the creation, dissemination, application, and translation of new knowledge and practices

Critically evaluate medical information, its sources, and its relevance to their practice, and apply this information to practice decisions

6

What Do We Mean by Family Medicine Scholarship?

• The Family Medicine Laboratory = The Clinic

• Engagement with “evidence” at various levels

• Implementing Evidence in Patient Centred and

appropriate ways

QualityImprovement

7

What is Quality Improvement?

Plan, Do, Study, Act (PDSA)

8

QI initiatives• Small-scale cycles of

interventions linked to assessment.

• Goal of improving the process, outcome, and efficiency of complex systems of healthcare.

• Chart audits are often used as part of QI efforts.

• Not done to root out bad quality, but rather to measure quality.

Purposes of Quality ImprovementTo measure quality of care in order to improve it.

• Document something• Determine if the outcome is what is wanted• Find a defect in the process• Fix it• Re-measure to determine if the fix worked

Practices and health systems that agree upon initiatives and processes of care can use audits to determine how well they are following them?

9

Agreeing on processes?

Clinicians are idiosyncratic individuals• “How consistently are you caring for your

PRACTICE POPULATION’S DIABETICS?” as opposed to “How are you caring for MRS SMITH?”

• What are the determinants of• First choice of anti-hypertensive?• A labour floor’s cesarean section rate?• A clinic’s rate of checking ACRs in diabetics?

What is Chart Audit?

A chart audit is an examination of medical records (electronic and/or hard copy), to determine what has been done, and see if it can be done better.

10

What should the audit be on?• Can be virtually any aspect of healthcare.

• The data being reviewed should be ACCURATE and must be AVAILABLE in the medical record.

• The data is confidential – issue of ethics approvals.

• Identify a clinically significant care gap• Choose, explain, and justify your external benchmark• Identify, explain, and justify your internal benchmark• Explain and justify the specific measurement outcomes• Describe the population you plan to sample• Collect data• Report findings• Make specific recommendations for system change

Planning a chart audit

11

• Intervention/tests that will significantly benefit patients if implemented.

• Requires “good quality” evidence of benefit.• Statistically significant.• Sufficient magnitude to be clinically significant.• Patient oriented outcome; meaning something patients

will notice.

Clinically Significant Care Gap

12

• Evidence‐informed frequency of provision of some important aspect of care that has already been achieved in a real world setting.  per individual (e.g. occurs at each visit, occurs a minimum 

of twice a year)

across a whole practice population (e.g. at least 95% of all people over the age of 65)

across a specific population (e.g. at least 90% of  patients on opioids for more than three months have a signed 

opioid contract)

External Benchmark

13

• Target level of implementation which you have determined is most appropriate for your local practice setting. 

• Done in consultation with local clinicians • Must have already made some attempt to implement the 

intervention. • May equal the external benchmark

• If unrealistic, too difficult to measure, or otherwise inappropriate, may be different 

• Might be a proxy measure for the external benchmark.

Internal Benchmark

• Actual variable assessed in the chart audit which is compared to the external and internal benchmarks.

• An example of specific outcome measures are 

clinical variables such as blood pressure, lab values 

or characteristics of care provision (i.e. 

hospitalization or mortality.

Specific Outcome Measure

14

Any Questions?

1

Family Medicine Scholarly Projects

Part 2How to choose a question and write your queries

Alexander Singer MB BAO BCh CCFPQI/Informatics Stream LeadAssociate Professor

R1 Scholarly Project• Written paper and presentation

• Completed as small group (2-3)

• Done during Family Medicine Block Time (FMBT)

• Quality Improvement (QI) using chart audit

• Project must be unique to stream• The same topic cannot be repeated within your stream in the same

year. Get approval from your Education Director/Stream Lead

• Relevant to Family Medicine and your stream

2

R1 Scholarly Project• Identify a potential “care gap”• Explain and justify “benchmarks” for “best care” derived

from:• Literature review & collaboration with your health care team

• Define your population-of-interest• Outline your EMR search/chart audit strategy• Use findings to develop a QI plan

• What SYSTEM changes must occur to move towards benchmark?• Who needs to be involved in the change?• Barriers to implementation?

R1 Scholarly Project• Expectations – Written Paper

• Title page• Introduction• Methods• Results• Discussion• References and Appendix

• Expectations – Oral Presentation• Group presentation at home clinic• 20-30 minutes with 5 minutes for questions

3

R2 Scholarly Project

• Written paper and presentation

• Completed individually

• QI or Research based on chart audit

• Project must be unique to stream

• Relevant to Family Medicine and local site

R2 Scholarly Project - Research• Identify suspected “care gap”• Conduct chart audit for preliminary data

– Proof for further study• Identify specific research question• Conduct literature review for background• Describe population• Define data sources and research methods• Proposed Statistical Analysis & rationale• Knowledge Translation plan

4

R2 Scholarly Project• Expectations – Oral Presentation

– Individual presentation at end of R2 year• FM Scholar Day – May, 2017

– 10-minutes with 5 minutes for questions

• Must present on FM Scholar Day and if not it is YOUR responsibility to make alternative arrangements!

Completion of R1 and R2 projects are a program requirement for graduation!

Plan, Do, Study, Act (PDSA)

5

What is Chart Audit?

A chart audit is an examination of medical records (electronic and/or hard copy), to determine what has been done, and see if it can be done better.

• Identify a clinically significant care gap• Choose, explain, and justify your external benchmark• Identify, explain, and justify your internal benchmark• Explain and justify the specific measurement outcomes

• Describe the population you plan to sample• Collect data• Report findings• Make specific recommendations for system change

Planning a chart audit Discussed in previous session

6

Asking the question: The PICO approach

PopulationIntervention or IssueComparatorOutcome

Asking a “Good” Question

• Hot topics?• Clinical practice guidelines• Chronic Disease Management (PCQI)• Data quality• Choosing Wisely

• Data Quality (data completeness and capture) for chosen question needs to be considered

7

Identify patient population

• You need to define the population you wish to assess. Consider: age, gender, disease status, treatment

status, etc.

• In many cases, the topic itself will help to define this. If you want to look at cervical cancer screening

rates, the population has to be limited to women. You may wish to exclude those who have had hysterectomies…

8

Identify Measures (and intervention you are comparing)

• Define EXACTLY what you will measure• What is a YES (criteria met)? / What is a NO

(not met)?

• Do a literature review to help in defining/justify measures – ideally variables that have been used successfully in the past or have clear clinical relevance.

Data Discipline

9

Determine a sample size

• For optimal results calculate necessary power and statistical significance.

• A common rule of thumb is to try for 10-20% of the eligible charts.

• Multiple queries can also lead to interesting breakdowns of the information.

• Expect to assess ~30-50 charts or have 10+ queries to provide data which is granular enough.

Collect Data

• Review each chart to determine if the individual meets the selection criteria (e.g., correct age, gender, etc.)

• Complete one audit tool (paper or line in the spreadsheet) for each individual you include in the sample.

10

How to conduct a chart audit in Accuro?

How to make queries work• Use coded fields as much as possible

• Free text is the enemy• Examples: Labs, Demographics, Problem List (for the

most part)• Ask answerable questions• Build cohorts with manageable numbers• Let the computer do the heavy lifting

11

Overview of the Query Builder

What is the question you want to answer?

12

Storing Query Data in Accordance with Policies and Security & Storage of PHI

Printing Results: Perform only if you will need ongoing access to the output of your query at a given point in time.• Make sure you know which printer you are

connected to. • Retrieve the print job immediately. • Store all written PHI in a secure file and keep files

in a secure place at all times.

Storing Query Data in Accordance with Policies and Security & Storage of PHI

Exporting Results: Perform only if you need to do work on the data.Export‐> Computer ‐> C$(\\Client) (V:) ‐> Documents & Settings ‐> Your username 

(i.e. mkrahn3) ‐> Select a Folder‐> Save. 

– Do not save queries on the desktop.

– De‐identify if possible and delete information not required: Keep only the minimum amount of information you need to know in order to analyze the data. 

– Save the document as an excel file & encrypt the workbook.

– Delete the original export (CSV file).

13

What Kind of Data is Best?

• The accuracy of query results depends on data correctness and completeness in the EMR.

• Many fields in the EMR that are ‘queryable’ are not always up-to-date (e.g. patient problem list).

CAUTION: DATA QUALITY FOR SOME FIELDS IN THE EMR IS POOR

14

• COPD: 48.55%

• CHF: 49.49%

• CAD: 67.10%

Oct

ober

30,

210

3

WRHA CompletenessProblem List compared with Billing

Collecting Data - Create Audit Tools

A spreadsheet format is ideal for record keeping.

• For those more comfortable with paper-based systems, a sheet of paper pre-printed with key points/questions to check in each chart serves well as an audit tool.

• Another sheet that compiles and summarizes your findings is also helpful.

• Appropriately report relevant data in form of a rate, percentage, mean, or other statistical measurement.

15

Reporting Findings

• Summarizing the data is a little more complex than just counting up all the data sheets. You must consider how the data will be used, and make sure the information is presented in a way it will be useable.

• The trickiest part is often determining the proper denominator to use for percentages. Inconsistencies here can lead to confusing or un-interpretable data.

Analyze and apply findings/results

• Compare and contrast to internal and external benchmarks • Explain rationale for proxy measures you may have

used• Justify why data is above or below pre-determined

benchmarks • Remember a rate of 100% or 0% is usually

unrealistic• There are usually explanations as to why clinical

data deviates from expectations/recommendations

16

Making Specific Recommendations

• Describe interventions to correct the ‘defect’

• What can be done with the current organizational system to make things “better” beyond mere ‘raising of awareness’?

• Be sure to involve stakeholders in the design of a proposed solution or course of action• What does ‘success’ look like for the next audit?• Is it REASONABLE and ACHIEVABLE?

• WHO needs to be tasked with follow-through?

If data quality is good the EMR will support efforts to leverage the EMR for your quality improvement “intervention”

Tasks to request individual phone calls to patients Letter generation to mail reminders to patients Reminders (in patient chart) to mention

requirements to patients when next in the office

Examples of How to Use your Queries

17

Taking action on query results

Taking this action will create tasks for the selected user with the note as entered for each patient in the query results

Taking action on query results (2) • You may also choose to create reminders in patient

charts. It will appear the next time you are in the chart.

18

And what about research?

What is CPCSSN?• CPCSSN - Canadian Primary Care Sentinel Surveillance

Network

• The first pan-Canadian primary care based multi-disease surveillance system

• Collaboration of Clinicians and Researchers

• A network of networks securely collecting health information from Electronic

Medical Records (EMRs) in the offices of primary care providers since 2008.

19

How can you get involved?

What is CPCSSN?

11 Primary-Care Based Research Networks across Canada

7 Provinces, 1 Territory

11 distinct EMR systems

MaPCReN is Manitoba’s network in the CPCSSN

How can you get involved?Sites Sentinels Patients

CPCSSN 217 1,189 1,500,000+

MaPCReN 48 251 287,961 As of Apr. 3, 2017

• Each local CPCSSN network recruits local primary care providers who consent to extraction of data from their EMRs.

• MaPCReN (Manitoba Primary Care Research Network)o currently extracting EMR data from practices utilizing

Accuro and JonokeMed

What is CPCSSN?

20

CPCSSN - FocusInitial focus:

• 5 chronic health conditionso Hypertensiono Osteoarthritiso Diabeteso COPDo Depression

• 3 neurologic conditionso Alzheimer’s and related dementiaso Epilepsyo Parkinson’s Disease

CPCSSN EMR Data• Provider characteristics• Practice characteristics• Patient demographics• Health conditions• Patient encounters• Risk factors• Medications• Diagnostic test results

Structured Database

21