PIs: Oppezzo, M; Brown-Johnson, C.

Assessment of Experience-Driven Online Medical Education (“Experience-Driven Online Med Ed”)

Co-PIs: Marily Oppezzo, MS, RD, PhD, and Cati Brown-Johnson, PhD; Post Doctoral Scholars with the Stanford Prevention Research Center in the Department of Medicine at Stanford University; Contact information: moppezzo@stanford.edu; catibj@stanford.edu Mentor: Judith Prochaska, PhD, MPH, Associate Professor and licensed clinical psychologist with the Stanford Prevention Research Center in the Department of Medicine at Stanford University Consultant: Daniel Lewis Schwartz, PhD, I. James Quillen Dean of the Stanford Graduate School of Education, Nomellini & Olivier Professor of Educational Technology Amount of funding requested: $19,900

PIs: Oppezzo, M; Brown-Johnson, C.

Rationale: Learners retain and apply knowledge better when they experience the situations addressed by the knowledge prior to obtaining the information 1. However, much of current online learning presents the knowledge only, with a teacher-centered pedagogy of lecture videos or narrated slides 2. Simultaneously, there is also a call to redesign the delivery of health professions’ education to incorporate more active-learning strategies, such as problem- and case-based learning 3. Finally, there is not enough rigorous, comparison-group testing of the effectiveness and pedagogy of e-learning courses. Of 177 studies reporting effectiveness of e-learning courses for nursing education, only 11 reported effectiveness on participants’ knowledge level with comparison groups (Lahti et al. 2014). Our proposal, Assessment of Experience-Driven Online Medical Education (“Experience-Driven Online Med Ed”) exchanges the current common pedagogical model for a learner-centered approach 4, which provides active context for the information, and offers practice opportunities for learned skills. To test effectiveness of Experience-Driven Online Med Ed, we will use a pilot randomized control trial design to compare learning and med student behavior adoption between a typical teacher-centered pedagogy (lecture-driven or LECT) and our learner-centered approach (Experience-Driven Online Ed or EXP), within the framework of a medical education mini-course on electronic cigarettes.

Pilot Data: 1) Experience-Driven e-learning: patients are motivated by e-learning scenarios and state that directing the time/location for learning is a positive aspect of EXP 5; 2) Being told procedures and concepts before problem solving can inadvertently undermine the learning of deep structures in physics 6, implying that experience-driven learning can increase transfer of knowledge.

Project Description/Anticipated Work Product: Our team has developed an online interactive “Day in the Life” of a family physician, weaving expert opinions and research into patient cases created to illustrate what is known, and not known, about e-cigarettes and health. A new and controversial topic on the rise 7, the current science on e-cigarettes is more nuanced than a simple harmful or helpful choice. In the EXP arm of our trial, the learner can search for knowledge needed to address each patient case by accessing video clips from experts and is motivated by specific patient problems. The learner navigates each patient case by selecting response options in a “choose your own adventure” framework, and at the end receives feedback about their choices coupled additional course information. To test the learning effects of this approach, we aim: AIM 1: To develop an online content-identical mini-course in a lecture-driven format (LECT) and pilot the implementation of tracking learner choices and paths within the EXP mini-course. AIM 2: To determine, via a randomized controlled pilot, the efficacy of our experience-driven online model on the following outcomes: declarative knowledge of the science of e-cigarettes and learning objectives (see educational objectives); practical application of knowledge to individualized patient scenarios; self-efficacy of providing smoking cessation counseling; application and retention of e-cigarette knowledge.

Table 1. Course comparison (bolded words are content-identical)

EXP ~ 90% video + interactive learning LECT ~ 70% video; NO interactive learning

Videos of interactive patient cases where patients present e-cigarette questions and learners choose from responses options.

Patient cases are presented after didactic material.

Expert video clips available in a resource library for learners. Learners must search out and gain their own knowledge.

Expert video clips presented sequentially as didactic material. Learners cannot opt out of knowledge presentation.

Expert patient advice given for each interactive case. Expert patient advice given as solution to cases.

Design: We will run an RCT comparing learning and behavioral outcomes from EXP and LECT e-cigarette mini-course. We will recruit 120 medical students, 30 within each year, using block randomization to assign 15

PIs: Oppezzo, M; Brown-Johnson, C. participants to each condition by med school year (similar sample sizes to Schwartz & Bransford, 1998). Participants will receive a $20 Coupa Cafe card as compensation for completing course and assessments.

Table 2. Learning and Behavioral Objectives/Plan for Evaluation

Learning Objective (LO) / Behavioral Objective (BO)

Measure Timing Hypothesis

BO1. To increase patient e-cigarette counseling self-efficacy.

1. Self-efficacy of Providing E-cig Counseling (Likert)

Pre / Post, 3 mo

EXP > LECT

LO1. To learn new information about e-cig risks and benefits; LO3. To reiterate best practices for supporting smoking cessation (e.g., Ask-Advise-Refer; first-line cessation medication + psychosocial support)

2. Declarative knowledge of Learning Objectives (multiple choice)

Post mini-course

EXP = LECT

LO1; LO2. To develop informed professional opinions about how to counsel patients on e-cig use; BO2

3. Knowledge comprehension and application (open-ended response to novel patient)

Mid mini-course

EXP > LECT

BO2. To counsel patients who are using or considering using e-cigarettes to attempt cessation with FDA-approved NRT or pharmacotherapy

4. Learner Behaviors (choices within EXP mini-course)

Throughout EXP mini-course

Later patient case choices will reflect BO2

LO1; LO2; BO1; BO2;

5. Delayed Retention and Near Transfer Knowledge Application (New patient Case)

1 week post mini-course

EXP > LECT

Table 3. Timeline and Implementation Plan

Activities Mar Apr May Jun July Aug Sept-Feb

IRB X Finish development EXP mini-course -> X HTML-5 programmer to embed learner tracking code in EXP (goal: track learner pathways through EXP cases) -> -> X

Create content-identical LECT mini-course -> -> X Create assessment items -> X Recruit medical students through listserves and courses -> -> X Collect and analyze data; write final report -> X

Plan for dissemination and anticipated impact of results on online medical education: Potential papers for dissemination of results in peer-reviewed journals include: 1) (Education Psychology) Efficacy of a Learner-Centered and Experience-Driven Medical Education Mini-Course on E-cigarettes; 2) (Tobacco Control) Effects of Virtual Nicotine Addiction Counseling on Medical Student Tobacco Intervention Self-Efficacy. Because this is a cross-disciplinary topic, results will be relevant to schools of education and medicine, and allied health care delivery. By not only comparing two pedagogical formats, but expanding typical declarative knowledge assessment measures to include new situation application, self-efficacy of behavior adoption, and virtual learning and response behavior in practice situations, this will be a novel and rigorous evaluation that can inform future online medical education course development.

PIs: Oppezzo, M; Brown-Johnson, C. !

1. Schwartz DL, Bransford JD. A time for telling. Cogn Instr. 1998;16(4):475-5223.

2. Breslow L, Pritchard DE, DeBoer J, Stump GS, Ho AD, Seaton DT. Studying learning in the worldwide classroom: Research into edX’s first MOOC. Res Pract Assess. 2013;8. http://search.proquest.com/openview/27e5ba80e54284de770427a71f05bf8d/1?pq-origsite=gscholar. Accessed March 1, 2016.

3. Prober CG, Heath C. Lecture halls without lectures—a proposal for medical education. N Engl J Med. 2012;366(18):1657-1659.

4. Norman DA, Spohrer JC. Learner-centered education. Commun ACM. 1996;39(4):24-27.

5. Brown-Johnson CG, Berrean B, Cataldo JK. Development and usability evaluation of the mHealth Tool for Lung Cancer (mHealth TLC): A virtual world health game for lung cancer patients. Patient Educ Couns. 2015;98(4):506-511.

6. Schwartz DL, Chase CC, Oppezzo MA, Chin DB. Practicing versus inventing with contrasting cases: The effects of telling first on learning and transfer. J Educ Psychol. 2011;103(4):759.

7. Ramo DE, Young-Wolff KC, Prochaska JJ. Prevalence and correlates of electronic-cigarette use in young adults: Findings from three studies over five years. Addict Behav. 2015;41:142-147. doi:10.1016/j.addbeh.2014.10.019.

!Non$Compensation$Items$

Description$of$Request$ Item$ Quantity$ Cost$per$Item$ Total$Cost$Learner!tracking!code!and!LRS!(learner!record!store)!database!creation!for!tracking!experience8driven!course!choices!(developed!in!HTML5!with!xAPI/TinCan)!

Programmer!contract!

40!hours! $110/hour! 4,400.00!

Create!online!mini8courses!(Med!IRT!8!Educational!Technology!group)!

MedIRT!Ed!Tech!Group!contract!

LECT!mini8!course!

! 5,000.00!

Participant!Incentives! Coupa!Café!Gift!Cards!

120! $20!/!card! 2,400.00!

Create!Online!Patient!assessment!(OSCE,!objective!structured!clinical!examination)!including!video,!patient!actor!!(Med!IRT!8!Educational!Technology!group)!

MedIRT!Ed!Tech!Group!contract!

OSCE!interactive!video!assessment!

! 2,000.00!

Statistical!consulting!for!structural!equation!modeling!(SEM)!for!learner!path!analysis!within!experience8driven!course!arm!

! 3!hours! $100!/!hr!! 300.00!

! ! ! ! 14,100.00$!Compensation$Items$

Description$of$Request$ Compensation$$ Total$Cost$Honoraria!#1!Marily!Oppezzo,!PhD,!MS,!RD! 2,900.00! 2,900.00!Honoraria!#2!Cati!Brown8Johnson,!PhD! 2,900.00! 2,900.00!! ! 5,800.00$

!! ! ! ! ! ! ! ! ! $$$$$Total$Expenses:$19,900.00$!!

OMB No. 0925-0001/0002 (Rev. 08/12 Approved Through 8/31/2015)

BIOGRAPHICAL SKETCH Provide the following information for the Senior/key personnel and other significant contributors.

Follow this format for each person. DO NOT EXCEED FIVE PAGES.

NAME: Oppezzo, Marily Ann

eRA COMMONS USER NAME (credential, e.g., agency login): MOPPEZZO

POSITION TITLE: Postdoctoral Research Fellow

EDUCATION/TRAINING

INSTITUTION AND LOCATION DEGREE Completion Date

MM/YYYY

FIELD OF STUDY

Santa Clara University, Santa Clara, CA B.S., B.A. 06/1999 Psychology (Health); Italian

San Jose State University, San Jose, CA M.S., R.D. 08/2003 Nutritional Science

Stanford University, Stanford, CA PhD 12/2012 Educational Psychology

Stanford University School of Medicine, Stanford Prevention Research Center, Stanford, CA

Postdoctoral Fellowship 09/2016 Prevention Research

A. Personal Statement My work, expertise, and experience intersect the areas of psychology, education, physical activity, behavior

change, and preventative medicine. Through these diverse interdisciplinary trainings, I have accrued a unique set of skills that will allow me to continue successfully contributing to health sciences and both behavioral and medical education. The goal of the proposed research is to empirically investigate the effectiveness of a problem-driven, learner-centered online medical education mini-course compared to a solution-driven, teacher-centered online medical education course. My doctoral training in psychological and educational theories of motivation and learning, with expertise in measure development, make me a strong candidate for Co-PI for this project. I helped design the experience-driven learner-centered module to incorporate learning theories such as the generation effect (e.g., generating knowledge helps facilitate learning and retrieval), contextualized learning (e.g., learning within the context one will apply the knowledge helps with later application), and problem-driven knowledge seeking (e.g., finding out problem constraints and seeking solutions prior to receiving expert solutions improves deeper learning). I have also used my learning theory knowledge and experience in study design to outline the comparison lecture-driven teacher-centered course. The online educational research we propose will not only compare pedagogical formats between these two courses, but will field-test knowledge assessment measures that are beyond the typical declarative knowledge assessments found in most online courses. These novel measures will include application of knowledge and knowledge seeking behavior in a new clinical situation, self-efficacy of behavior adoption, and virtual learning and response behavior in practice situations. My experience in designing educational assessments to test applicative and deeper knowledge, rather than declarative, will help inform this project’s measures. Outcomes from this novel research idea will inform future online medical education course development, and be relevant to schools of education and medicine that my expertise and training currently bridge.

B. Positions and Honors Positions and Employment 2000-2003 Dietetic Research Employee, General Clinical Research Center, Stanford University Hospital,

Stanford, CA 2003 Dietetic Intern, Dietetics, Palo Alto Veteran’s Affairs Hospital, Palo Alto, CA 2003-2005 Faculty Instructor, Department of Nutritional Sciences, San Jose State University, San Jose,

CA

2004-2005 Nutrition Instructor, Department of Nutritional Sciences, Mission College, Santa Clara, CA 2004-2005 Nutrition Instructor, Department of Nutritional Sciences, College of San Mateo, San Mateo, CA 2004 Nutrition and Health Educator, Dietetics, Camino Medical Group, Sunnyvale, CA 2004-2005 Nutrition and Health Educator, Dietetics, El Camino YMCA, Mountain View, CA 2004-2004 Research Assistant for Dr. Gene Speller, Los Altos Health Research Center 2005-2012 Research Assistant, Department of Education, Stanford University, Stanford, CA 2012-2014 Adjunct Faculty, Department of Biology and Department of Psychology, Santa Clara

University, Santa Clara CA Honors 1999 Dissertation Support Grant, Stanford University’s Graduate School of Education, $6000 1999 Phi Beta Kappa, Santa Clara University 1999 Magna Cum Laude with Honors, Santa Clara University 1999 Honors Program, Santa Clara University 1999 Phi Sigma Iota, Santa Clara University

Other Experience and Professional Memberships 1999 Phi Beta Kappa 2003- Registered Dietitian, CDR registered, #943792, California 2003- American Dietetic Association, (ADA), member 2003- San Jose Peninsula District Member of the California Dietetic Association (CDA), member 2013- Reviewer for Learning and Instruction 2015- Society of Behavioral Medicine member

C. Contribution to Science 1. Investigating the live interaction between physical movement and cognition. While there is an abundance of evidence suggesting that long-term, regular, aerobic exercise is good for the brain, there is a lack of research on how the brain works during concomitant physical movement. My dissertation investigated how walking improved creative ideas but did not change or help focused thinking, which shows it is not a global effect, but specific to certain types of thinking. Demonstrating that walking’s effect differs by what is held in mind during the walk provides a new perspective on the interaction of physical activity and cognition. Looking at how light physical movement affects cognition, I designed and ran a study showing that 6th grade students have more creative ideas when they are allowed to fidget and move in their chairs compared to when told to sit still and face forward. Finally, I designed and am currently running a study investigating the effects of walking on negotiation, with the idea that if walking enhances creative perspectives, more creative solutions will be allowed to be generated with negotiations. This work has broadened physical activity research beyond aerobic, longitudinal effects by showing that light physical activity can have immediate and concurrent effects. It also opens up new insights into the mechanisms of how walking can influence cognitive processes. I have also investigated the relationship between physical activity (as well as other health behaviors) and health-related quality of life in veterans. This link between physical activity participation and both mental and physical quality of life demonstrates the connection between movement and cognition at a coarser level of analysis.

• Oppezzo, M.A. & Schwartz, D.L. (2014). Give your ideas some legs: The positive effect of walking on creative thinking, Journal of Experimental Psychology: Learning, Memory, and Cognition, 40(4), 1142-1152.

• Oppezzo, M., Michalek, A., Delucchi, K., Baiocchi, M. (in press). Health-related quality of life among veterans in addictions treatment: identifying behavioral targets for future intervention. Quality of Life Research

2. Connecting theories of motivation, self-regulated learning, and behavior change. While educational psychology and behavior change theories are often siloed in separate disciplines and empirical studies, I have leveraged my interdisciplinary expertise to bridge them together in my work. My work with Daniel Schwartz, one of the leading experts in learning theories, focused on how to best motivate students to learn, as well as regulate themselves to maintain future learning behaviors. In one project which I helped conduct and theorize,

our lab used an interactive learning technology to allow students to teach either a computer agent or themselves by designing a concept map. We found students were more motivated to take time to read, preference learning choices over social chatting options, and learned more deeply about the concepts when teaching the computer agent then when learning for themselves, what we called “The Protégé effect.” I also co-authored a chapter that conceptually bridged the self-regulation of learning techniques to behavior change. In it I referenced my (pending publication) work where I taught theory-based motivation strategies to high school students to help motivate themselves to study difficult subjects and achieve a difficult behavior change (specifically, increase their fruit and vegetable intake). Interleaving these two previously disparate fields to draw informative parallels shows how the two disciplines not only complement each other, but also could benefit from each other.

• Chase, C.C., Chin, D.B., Oppezzo, M.A., & Schwartz, D.L. (2009). Teachable agents and the protégé effect: Increasing the effort towards learning. Journal of Science Education and Technology, 18, 334-352.

• Oppezzo, M.A. & Schwartz, D.L. (2013). A behavior change perspective on self-regulated learning with teachable agents. In R. Azevedo, & V. Alevan (Eds), International Handbook of Metacognition and Learning (pp. 485-500). New York: Springer.

• Chin, D.B., Dohmen, IM, Cheng, B.H., Oppezzo, M.A., Chase, C.C., & Schwartz, D.L. (2010). Preparing students for future learning with teachable agents. Educational Technology Research and Development, 58(6), 649-669.

3. Theory-based technology to facilitate behavior change and education. Another major component of my work involves the applications of motivation and educational theories to technologies for enhancing learning and behavior change. While a lot of technology research focuses on technology for technology’s sake, I strive to investigate the “why” it works. My doctoral co-emphasis is in Learning Sciences and Technology Design, and I helped intellectually contribute to and run studies using a learning-by-teaching software to investigate accountability and motivation in learning choices. Currently in my post-doc, and the purpose of this K-award, I designed and am leading an investigation using Twitter to increase physical activity based on empirical, psychological theories of social support, mindfulness, and goal-setting combined with visual markers of progress. Additionally, I am helping design an online continuing education unit for health professionals on e-cigarettes, and basing it in theories of learning such as having the users use resources to find answers rather than just telling them; having a contextualized learning situation; involving choice rather than passive reading; using applied assessment questions rather than declarative, fact-based assessments; allowing for learning to happen during the assessment by providing guided feedback and allowing users to return to assessments and self-correct.

• Chase, C.C., Chin, D.B., Oppezzo, M.A., & Schwartz, D.L. (2009). Teachable agents and the protégé effect: Increasing the effort towards learning. Journal of Science Education and Technology, 18, 334-352.

• Chin, D.B., Dohmen, IM, Cheng, B.H., Oppezzo, M.A., Chase, C.C., & Schwartz, D.L. (2010). Preparing students for future learning with teachable agents. Educational Technology Research and Development, 58(6), 649-669.

• Berlin, D., Person, M., Mittal, A., Oppezzo, M., Chin, D., Starr, B. Klein, T. Schwartz, D., Altman, R. (2010). DNATwist: A web-based tool for teaching middle and high school students about pharmacogenomics. Clinical Pharmacology and Therapeutics, 87(4), 393-5.

4. Tailoring evidence-based guidelines to education and behavioral interventions. Both education and health interventions can be driven by generic guidelines or evidence derived in specific populations. Being able to adapt to the community or classroom of interest is an important skill for both fields. In both my training in education and health research I have learned and been able to apply the necessary balance between evidence-based practice / guideline implementation and practice-based evidence in diverse settings. For example, my classroom-based intervention for self-regulated learning was developed through iterations of studies in high-school classrooms, adapting the empirical theories to fit the high-school classroom environment. When then applying it to middle school, it had to be again changed to only be focused on how to develop good goals rather than maintaining long term behavior, as middle-school children’s ability to prepare and forward-chain on long timelines was not yet developed. In my post-doctoral training, I adapted an

established nutrition behavior change intervention program, ProChange, for Alaskan natives; instead of the national guidelines advocating fresh fruit and vegetables, which are expensive and rare in Alaska, I changed the materials to encourage consumption of native heart healthy foods.

Complete List of Published Work in MyBibliography: http://www.ncbi.nlm.nih.gov/sites/myncbi/marily.oppezzo.1/bibliography/48747308/public/?sort=date&direction=ascending

D. Research Support

Ongoing Research Support Stanford Center for Continuing Medical Education (SCCME) Jackler (PI) 08/01/2015-01/01/2016 E-Cigarettes: Harmful or Harm-Reducing? Online Medical Education in Electronic Nicotine Delivery Products Funding to create an online continuing medical education module about e-cigarettes. Role: Consultant NIH Grant: T32 CA113710 9/01/2014-present Postdoctoral Training in Preventative Medicine Role: Postdoctoral Fellow Completed Research Support

Stanford University’s Graduate School of Education’s Dissertation Support Grant 6/2011-6/2012

OMB No. 0925-0001/0002 (Rev. 08/12 Approved Through 8/31/2015)

BIOGRAPHICAL SKETCH Provide the following information for the Senior/key personnel and other significant contributors.

Follow this format for each person. DO NOT EXCEED FIVE PAGES.

NAME: Brown-Johnson, Cati

eRA COMMONS USER NAME (credential, e.g., agency login): CATHJOHNSON

POSITION TITLE: Postdoctoral Research Fellow

EDUCATION/TRAINING

INSTITUTION AND LOCATION DEGREE

Completion Date

MM/YYYY

FIELD OF STUDY

Agnes Scott College, Decatur, GA B.A. 06/2000 English/French

University of Georgia, Athens, GA Ph.D. 12/2006 Linguistics

Center for Tobacco Control Research and Education, University of California, San Francisco

Postdoctoral Training 09/2013 Tobacco Control &

Public Health

Stanford Prevention Research Center, Stanford University School of Medicine, Stanford, CA

Postdoctoral Fellowship 07/2016 Prevention Research

A. Personal Statement My work centers on communication and technology in the context of tobacco-related health disparities and

diseases. The goal of the proposed research is to determine the impact of experience-driven pedagogy on medical student knowledge, attitudes, and behaviors with respect to electronic cigarettes and tobacco cessation counseling for their future patients. Outcomes from the overall trial will inform online medical education best practices.

Specifically, as Co-PI for this project I will design, implement, and assess the proposed randomized control trial. I will adapt our team’s existing online continuing medical education course, leveraging a previous $20,000 grant from the Stanford Center for Continuing Medical Education, to develop a mini-course for medical students, and create a second lecture + slides mini-course for comparison. Both courses will utilize multi-media components, i.e., expert interviews and videos of patients. The lecture-driven course will present these elements sequentially in the following format: 1) information, 2) expert interviews, 3) patient cases + best-practice physician responses. The experience-driven course will present largely identical elements, but learners will be responsible for accessing information based on their need, and learning will be driven by interactive patient cases with information and expert video reference libraries. Best-practice case responses will be presented AFTER med students make their interactive patient decisions in the experience-driven format.

I am well-suited to this project and have experience in development and evaluation of interactive technology-based training/interventions in the context of tobacco control, including a virtual reality lung cancer stigma intervention, and the previously-mentioned online continuing medical education course on e-cigarettes.

Additionally, the proposed research will also provide much-needed emphasis on e-cigarettes for medical students who will most certainly encounter questions about this topic during their practice. Best-case end results of this project include: 1) empirical data about pedagogical online approaches in a medical education setting, and 2) better understanding of student decision-making with respect to e-cigarette advice for patients. Ideally, outcomes will inform the development of a stand-alone online mini-course on e-cigarettes, complete with evidence of efficacy, student assessment tools, and a new e-cigarette OSCE.

a. Brown-Johnson, C.G., Burbank, A., Wassmann, A., Chieng, A., Rutledge, G. & Prochaska, J.J. (in

revision) Online patient-doctor consultations on e-cigarettes: perceptions of safety, harm and data gaps. American Journal of Preventive Medicine.

b. Brown-Johnson, C.G., Berrean, B. & Cataldo, J.K. (2015) Development and usability evaluation of the mHealth Tool for Lung Cancer (mHealth TLC): a virtual world health game for lung cancer patients. Patient Education and Counseling, 98(4), 506-11. NIHMSID: NIHMS658478

c. Sanders-Jackson, A., Brown-Johnson, C.G. & Prochaska, J.J. (2015) Applying linguistic methods to understanding smoking-related conversations on Twitter. Tobacco Control, 24(2), 136-8. NIHMSID: NIHMS596372

d. Brown-Johnson, C.G., Sanders-Jackson, A. & Prochaska, J.J. (2014) Online comments on smoking bans in psychiatric hospitals units. Journal of Dual Diagnosis, 10(4), 204-11. NIHMSID: NIHMS628980

B. Positions and Honors Positions and Employment 2001-2005 Research Assistant, the Tobacco Documents Project at the University of

Georgia, Department of Speech Communication, University of Georgia, Athens, GA 2002-2005 Research Assistant, CASPR (Computer Analysis of Speech for Psychiatric

Purposes), Artificial Intelligence Center, University of Georgia, Athens, GA 2004 Instructor of Record, Linguistics 2100, University of Georgia, Athens, GA 2005-2008 Manager of Linguistic Technology, H5 Technologies, San Francisco, CA 2008 2008-2011

Content Team Lead, SkyGrid, Sunnyvale, CA Senior Research Associate, See Change Inc., San Francisco, CA

2010-2012 Founding Advisory Board Member, Respect Institute, San Jose, CA 2011-2013 2013-

Postdoctoral Trainee, Center for Tobacco Control Research and Education, University of California, San Francisco, CA Postdoctoral Fellow, Stanford Prevention Research Center, Stanford School of Medicine, Stanford, CA

2015- Consultant, Oklahoma Tobacco Settlement Endowment Trust, Oklahoma City, OK Honors 2000 Magna cum laude, Agnes Scott College, Decatur, GA 2000 Agnes Scott Writer’s Festival poetry finalist (one of six) 2001-2003 Presidential Graduate School Assistantship, University of Georgia (competitive) Other Experience and Professional Memberships 2002-2006 Member, Linguistic Society of America 2003-2004 Vice President. University of Georgia Linguistics Society, 2003-2004 2004 Treasurer. UGA Linguistics Society Graduate Student Conference. The University of Georgia, Athens, Georgia, 21 February 2004 2007 Co-founder. Bay Area Industrial Linguists 2011-present Member, American Public Health Association 2012-2013 Postdoctoral Representative. Chancellor's Advisory Committee on Childcare. The University of

California, San Francisco ongoing Ad hoc reviewer for Lung Cancer, Journal of Psychoactive Drugs, Tobacco Control,

Occupational and Environmental Medicine, Behavioral Research Methods, Health Education and Behavior, JAMA Internal Medicine, Journal of Psychoactive Drugs

C. Contribution to Science 1. Treating tobacco in the context of communication and technology. Combining my background in linguistics

with an ongoing interest in tobacco and health disparities, I have used linguistic methods to explore tobacco-related communication, and develop tobacco treatment studies. These studies have included basic qualitative and quantitative approaches like content coding in addition to more advanced techniques such as semantic network analysis. Since I view language as data, I am able to explore attitudes towards tobacco in non-traditional technology-based settings, such as Twitter, New York Times comments, and online patient-provider forums. My major contributions in this space have been to 1) successfully integrate cross-disciplinary approaches and demonstrate the power of language, and specifically narrative, in motivation and engagement with uncomfortable interventions, and 2) develop tailored tobacco treatment initiatives based on qualitative data findings.

a. Prochaska, J.J.; Michalek, A.K.; Brown-Johnson, C.G.; Daza, E.J.; Baiocchi, M.; Anzai, N.; Rogers,

A.; Grigg, M. & Chieng, A. (in press). Smokers have a harder time finding work: results from a one-year observational study. JAMA Internal Medicine.

b. Brown-Johnson, C.G., Berrean, B. & Cataldo, J.K. (2015) Development and usability evaluation of the mHealth Tool for Lung Cancer (mHealth TLC): a virtual world health game for lung cancer patients. Patient Education and Counseling, 98(4), 506-11. NIHMSID: NIHMS658478

c. Sanders-Jackson, A., Brown-Johnson, C.G. & Prochaska, J.J. (2015) Applying linguistic methods to understanding smoking-related conversations on Twitter. Tobacco Control, 24(2), 136-8. NIHMSID: NIHMS596372

d. Brown-Johnson, C.G., Sanders-Jackson, A. & Prochaska, J.J. (2014) Online comments on smoking bans in psychiatric hospitals units. Journal of Dual Diagnosis, 10(4), 204-11. NIHMSID: NIHMS628980

2. Smoking and Lung Cancer Stigma. As a new postdoctoral scholar at the Center for Tobacco Control Research and Education, I was surprised at reports of the pervasive nature of smoking stigma and its negative downstream results. For instance, all lung cancer patients, regardless of smoking status, experience stigma related to their diagnosis and its connection to smoking. My research in smoking stigma and lung cancer stigma has reiterated that smokers and those with smoking-related conditions are demonized. I have also established that stigma related to smoking is a factor distinct from other psychological constructs (e.g., anxiety and depression) and other experiences of discrimination (e.g., stigma related to ethnicity or mental illness). Finally, my ongoing research explores the relationship between smoking stigma and cessation, with indications that smoking stigma may be a factor in quitting behaviors. Next steps in smoking stigma research include isolating which elements of smoking stigma, if any, may support quitting, and which deter it. a. Brown-Johnson, C.G., Cataldo, J.K., Orozco, N., Lisha, N., Hickman, N. & Prochaska, J.J. (2015)

Validity and reliability of the Internalized Stigma of Smoking Inventory: An exploration of shame, isolation, and discrimination in smokers with mental health diagnoses. The American Journal on Addictions, 24(5), 410-8.

b. Brown-Johnson, C.G. & Popova, L. (2015) Smoking stigma in non-smokers, smokeless tobacco users, conventional cigarette users and dual users. Health Behavior and Policy Review.

c. Brown-Johnson, C.G., Brodsky, J. & Cataldo, J.K. (2014) Lung Cancer Stigma, anxiety, depression and quality of life. Journal of Psychosocial Oncology, 32(1), 59-73.

d. Brown, C.G. & Cataldo, J.K. (2013) Explorations of lung cancer stigma for female long-term survivors. Nursing Inquiry, 20(4), 352-62. PMCID: PMC3683354

3. Tobacco industry documents. Another major component of my work has involved investigation of tobacco

industry documents. Prior to the existence of the UCSF Legacy Tobacco Documents Library, or even the release of the infamous internal Enron email corpus, with colleagues at the University of Georgia (Donald Rubin, PI) I helped collate the Tobacco Documents Corpus, a stratified sample of 1000 internal tobacco industry documents gleaned from documents released as part of the Tobacco Master Settlement Agreement between 48 Attorneys General and the tobacco industry in 1998. My previous tobacco documents work connected deception in tobacco industry documents to a lack of linguistic causal markers (“because” and “since”) and to verb choice. More recent industry document research represented collaborations between UCSF and the CDC, and highlighted tobacco industry marketing initiatives targeting low socioeconomic status women including use of couponing, selling with luxury, unisex branding and point-of-sale price reductions.

a. Brown-Johnson, C.G., England, L., Glantz, S. & Ling, P. (2014) Tobacco marketing to low

socioeconomic status women in the US. Tobacco Control, 23(e2), e139-46. NIHMSID: NIHMS560691 b. Brown, C.G. (2006) Ranking tobacco industry documents for corporate deception and fraud: a corpus

linguistic assessment. Athens, GA: University of Georgia Dissertations. c. Brown, C.G. & Rubin, D.L. (2005) Causal markers in tobacco industry documents: the pragmatics of

responsibility. Journal of Pragmatics, 37, 799–811.

d. Kretzschmar, W.A., Darwin, C., Brown, C.G., Rubin, D.L. & Biber, D. (2004) Looking for the smoking gun: principled sampling in creating the tobacco industry documents corpus. Journal of English Linguistics, 32(1), 31-47.

4. Linguistics in abnormal psychology. As part of the CASPR Group (Computer Analysis of Speech for Psychological Research) at the University of Georgia, I worked to identify language-based traits associated with schizophrenia and to develop automated language analysis tools for use in psychological research.

a. Brown, C.G., Snodgrass, T., Kemper, S.J., Herman, R. & Covington, M.A. (2008) Automatic

measurement of propositional idea density from part-of-speech tagging. Behavior Research Methods, 40(2), 540-545. PMCID: PMC2423207

b. Covington, M.A., Riedel, W.J., Brown, C.G., He, C., Morris, E., Weinstein, S., Semple, J. & Brown, J. (2006) Does ketamine mimic aspects of schizophrenic speech? Journal of Psychopharmacology, 21(3), 338-346.

c. Brown, C.G., Covington, M.A., Semple, J. & Brown, J. (2005) Reduced idea density in speech as an indicator of schizophrenia and ketamine intoxication. Schizophrenia Bulletin, 31, 187-8.

d. Covington, M.A., He, C., Brown, C.G., Naci, L., McClain, J.T., Fjordbak, B.S., Semple, J. & Brown, J. (2005) Schizophrenia and the structure of language: the linguist's view. Schizophrenia Research. 77, 85-98.

Complete List of Published Work in MyBibliography: http://www.ncbi.nlm.nih.gov/sites/myncbi/catherine.johnson.1/bibliography/47897950/public/?sort=date&direction=ascending

D. Research Support

Ongoing Research Support Stanford Center for Continuing Medical Education (SCCME) Jackler (PI) 08/01/2015-01/01/2016 E-Cigarettes: Harmful or Harm-Reducing? Online Medical Education in Electronic Nicotine Delivery Products Funding to create an online continuing medical education module about e-cigarettes. Role: Co-Course Director Tobacco-Related Disease Research Program 21BT-0018 Prochaska (PI) 07/01/2015-06/30/2018 The Total Improving eMployment Partnership: Alliance to Curb Tobacco [Total IMPACT] Project – An RCT Treating smoking for unemployed jobseekers in the Bay Area, this randomized control trial is based on pilot data showing that smokers have a harder time finding a job and remain unemployed for longer than non-smokers. We seek to determine whether treating smokers within unemployment settings with materials tailored for jobseekers results in higher quit rates, and greater employment, for participants. Role: Co-Investigator T32 HL007034 Gardner (PI) 07/01/2015-06/30/2016 Postdoctoral Training in Preventative Medicine Research from the National Heart, Lung and Blood Institute Role: Postdoctoral Fellow Completed Research Support R25 CA 113710-01A2 Glantz (PI) 07/01/2011-06/30/2013 Postdoctoral Training in Tobacco Control Role: Postdoctoral Scholar

March 1, 2016 RE: Support for Stanford Medicine Teaching and Mentoring Academy Grant - Assessment of Experience-Driven Online Medical Education! Dear Drs. Oppezzo and Brown-Johnson: I am in full support of your proposed pedagogical grant focused on innovations in medical education. This randomized controlled evaluation of different modes of online education will inform best practices with respect to online medical education with a focus on a timely clinical and public health concern: electronic cigarettes. I have participated in the refining of this proposal and as your faculty mentor, would be pleased to advise you in this research educational activity. My research expertise centers on randomized control trials designs and online behavior change interventions with a focus on tobacco and nicotine, including recent studies of e-cigarette use. I have developed, evaluated, and disseminated curricula for medicine focused on evidence-based tobacco cessation treatment (http://rxforchange.ucsf.edu) and will utilize my skill and experience in these areas to assist your success in this trial. With an experimental design, the outcome of the proposed trial of Experience-Driven Online Medical Education will be highly informative. If the lecture and slides format (LECT) performs best, we will know that Stanford and other medical institutions may save time and resources by providing high quality narrated slides online without the additional cost and time of patient scenario production. If the interactive and experiential patient case format (EXP) yields stronger outcomes, the utility of investing in online medical education modules with patient actors and more involved pedagogical design parameters would be supported. The research team is strong, multidisciplinary in expertise, and committed. I welcome the consulting support of Stanford’s Dean of Education, Dr. Daniel Schwartz, and input from School of Medicine colleagues, including former Assistant Dean of Continuing Medical Education, Dr. Robert Jackler. For this grant in particular, I will assist with design and randomization specifications. I enjoyed collaborating with you both during the development of an online continuing medical education module and believe this new project strongly capitalizes on that work. I will review progress, the analytic plan, results and manuscript drafts. The project is novel and timely in its content and methods. I wish you the best with the review. Sincerely, Judith J. Prochaska, PhD, MPH Associate Professor Stanford Prevention Research Center Stanford University

March 2016 Dear Drs. Oppezzo and Brown-Johnson: Thank you for the opportunity to collaborate with you on your Assessment of Experience-Driven Online Medical Education. I enjoyed consulting with your team on the initial development of what will become the “experience” arm of your trial. I am glad to know that you incorporated my idea of contextualizing the information in the situation where the learners will have to apply it (e.g. meeting patients in a clinic). As Dean of the Stanford School of Education and a cognitive psychologist, I am well-informed with respect to innovative and rigorously-tested pedagogical approaches, as well as an expert at study design and learning measure development. You are accurate in your assessment that a lecture-first pedagogical format has not been extensively tested against an experience-first pedagogical format in an online setting, nor have key characteristics of either been isolated. My prior research showing that having students try to look for deeper structure within a problem space before being told the solution is loosely related to your current comparison structure; the lecture condition being given expert solutions to the problem cases vs the experience condition being first situated in the problem case, having to learn and seek knowledge to solve the problem, and then being told the expert solution. While I expect that the experience-first format will outperform lecture in certain measures of knowledge, such as application and learning under new circumstances, your proposed research will be provide the data to back this up. I plan to consult with you on the development of the two conditions and learning measures. I believe this is an important topic that should be explored, and I look forward to your proposal being funded. Best regards,

Daniel L. Schwartz, PhD I. Quillen Dean Nomellini & Olivier Professor of Educational Technology daniel.schwartz@stanford.edu <AAALab.Stanford.edu> (650) 736-1514