Tech-based Behavioral Interventions: Poor Fit for Our Current Theories and Research

Methods

Audie Atienza, Ph.D.

Senior Behavioral Scientist

Science of Research & Technology Branch

Division of Cancer Control and Population Science

National Cancer Institute

Disclaimer

The views expressed are my own and do not necessarily reflect those of the Department of Health and Human

Services, the National Institutes of Health, or the National Cancer Institute.

http://articles.washingtonpost.com/2012-11-12/national/35504060_1_health-apps-app-developers-itunes-store

“We need to be sure that we're collecting the right evidence…The plural of anecdotes is not data.” Francis Collins

NCI Science of Research and Technology Branch Priorities

• Theory development, testing and application;

• Measure development and testing, particularly of antecedents to, changes in, and consequences of health behavior;

• Technology development and application;

• Methodological innovation, particularly in analytic approaches;

• Data harmonization and research synthesis; and

• Team science and cross-disciplinary approaches.

Chronic Disease Management • Problem: Chronic diseases are difficult and expensive to

manage within traditional healthcare settings

• Solution: CHESS: Disease self-management programs for asthma, alcohol dependence and lung cancer

• Information provided the user needs it

• Intervene remotely with greater frequency

than traditional care

– Real-time management

– More efficient triage

– Reduces acute care

David Gustafson, University of Wisconsin, NIAAA R01 AA 017192-04

ECG/ACC

ACC

Structure of Data Collecting Software

End-to-end

Encryption of

Sensitive Data

Device Manager

Local

Storage [User Configuration]

[Analyzed Data]

[Raw Data]

Transmitter [Encrypt/Decrypt]

Analyzer [Plug-in

modules]

GPS ACC ECG

Data Collector

Service Manager

Application with Graphic User Interface

Local Socket or IPC

•Problem: Overweight and Obesity among urban, minority youth •Solution: KNOWME networks personalized monitoring & feedback in real-time Immediate access to data allows nimble reactions to events, environments, & behavior User interface for health professionals, children & families

Donna Spruijt-Metz, PHD, USC, U54-CA-116848

Body Sensor Networks

Emerging Technologies and Assays for Adherence Monitoring

Xhale SMART “breathalyzer” for GRAS drug taggants

GlowCaps Proteus pill microchips and sensor

Drug (metabolite) concentrations via hair samples or dried blood spots

Implantable Biosensors

• Problem: Measurement of analytes (glucose, lactate O2 and

CO2) that indicate metabolic abnormalities

• Solution: Miniaturized wireless implantable biosensor that continuously monitors metabolism

– Inserted by needle subcutaneously

– Operated remotely using a PDA

– Multi-analyte sensor

– One month continuous monitoring

Diane J. Burgess, University of Connecticut NHLBI, R21HL090458

https://www.youtube.com/watch?v=J-QipXhQM5M

Theory-based Tech-Delivered Interventions

• Commercially-developed mobile health interventions do not adhere to treatment guidelines (Abroms et al, 2011),

• A substantial proportion of tech-based interventions in the published literature fail to indicate a theoretical basis for the intervention – Ritterband & Tate, 2009 for Internet

– Riley, Rivera, Atienza, et al., 2011 for mobile

“In theory, there is no difference between theory and practice, but in practice, there is.” Yogi Berra

Self-Efficacy

Behavior

Observational Learning

Mini-goals

O

O

O

Because Interventions Target Putative Mediators, not Behavior

WIRED MAGAZINE: Issue 16.07

http://www.wired.com/science/discoveries/magazine/16-07/pb_intro

The End of Theory: The Data Deluge Makes the Scientific Method Obsolete

Why Should We Care About Theory?

• Theory-based interventions appear to produce better results than non-theory based (Webb et al., 2010; Glanz & Bishop, 2010; Noar et al., 2007)

• but . . . – Some have failed to find this effect (Prestwich et al.,

2013) – It may be the result of more thoughtful intervention

development

• Predominant behavioral theories are dated and have numerous limitations, including limited variance explained (Ogden, 2003)

http://tech.fortune.cnn.com/tag/data-scientists/

Improving the Fit of Theory with Tech-based Technologies

• Technologies provide the ability for intensive longitudinal data within individuals over time – Real-Time Data Capture: ESM/EMA

– Sensor Technologies

• Yet our theories were developed primarily to explain differences between individuals, not within individuals over time (Dunton & Atienza, 2009)

A Control Systems Engineering Option

• Generate model of hypothesized systems of mediators and moderators affecting outcome

• Iterate and refine from rich longitudinal data sets of individuals

• Close the loop via proposed intervention

• Iterate and test in multiple individuals

Riley, Rivera, Atienza, et al, Transl Behav Med. 2011;1: 53–71

Control System Model of TPB

Barrientos, Rivera, and Collins (2010). A dynamical model for describing behavioral interventions for weight loss and body composition change. Mathematical and Computer Modeling of Dynamical Systems.

Technology Advances Leveraged to Advance Theory

• Use of intensive longitudinal data to model behavior changes over time

• Ability to isolate theory-based components • Marry modular software development with

theoretically-derived intervention components

• Tech interventions isolate intervention components with full fidelity

• Use of intervention optimization designs to not only optimize the intervention but also test theoretical components

EVALUATION OF TECH-DELIVERED INTERVENTIONS

Failure to Fit Part II

Randomized Controlled Trial: Standard for Intervention Evaluation • FDA: Safety and Efficacy Claims via RCT

– See FDA Mobile Medical Application Guidance (Sept. 25, 2013) http://www.fda.gov/downloads/MedicalDevices/.../UCM263366.pdf

• Meta-Analyses of Intervention Research – See Whitaker et al., Cochrane Database of Systematic Reviews, 2009,

CD006611 for example of mobile phone smoking cessation intervention review

• Practice Guidelines

• IOM and Other Scientific Consensus Reports

Technology Outpaces RCTs

2005 2006 2007 2008 2009 2010 2011

Grant

Submit

and

Award

Development and

Pilot Testing

Recruit and

Randomize

Follow-ups Analyze

and

Publish

iPhone Android YouTube iPad

Riley et al., 2013

Streamlining Tech-Based RCTs

• Speed or bypass development – Evaluate existing applications

– Open source or access (eliminate 1 off dev)

– Modular, component, “object” oriented development

• Leverage mobile technologies for research purposes – Remote Recruitment

– Embedded Outcome Assessment

– Automated RCTs

• Shorten follow-ups – Evaluate proximate outcomes

– Model or simulate more distal outcomes

Rapid, Responsive, Relevant Research (R3): Riley et al., 2013

Optimization Designs

• Which combination of components optimize outcome? – Factorial or Fractional Factorial

Designs

• See Collins et al., Psychol Methods, 2009; 14: 202-24

• What sequence of components optimize outcome? – Adaptive Trials

– Sequential Multiple Assignment Randomized Trial

• Murphy, Stat Med, 2005;:24:1455-81.

Collins et al., Am J Prev Med 2007; 32:S112-8

N of 1 – ABA Design

•Baseline

•Multiple assessments

•Need stability

•Prefer non-trending (see Salanas et al, Beh Mod, 2010;34-195-218 for baseline trend correction)

•Intervention

•Rapid effects

•Sustained during intervention

•Trending a plus

•Return to Baseline

•Withdraw treatment

•No lasting intervention effects

•Analysis

•Visual inspection

•Paired t-tests

•SD Bands (2 or 3 SDs)

•Variants

•Replication

•Staggered Replication

•Graph – Social Skills Intervention for Autism

•Gutman et al. Occup Ther Int 2010; 17:188-197

Interrupted Time Series •Use baseline data points to estimate CI of future data points

•Intervention with hypothesis that future data points will exceed CI

•Strengthened by control (as here) or staggered intervention

• RFID system to reduce healthcare wait times

•Kim et al, IEEE Trans Info Tech Biomed, 2010; 14: 935-940

Stepped Wedge Design

•Sequential roll-out by individuals or cohorts over multiple time periods

•Random assignment of intervention timing

•All eventually receive treatment

•Excellent example from Gambia Hepatitis Study

•Graph from: Brown & Lilford, BMC Med Res Method, 2006;6:54-62.

Regression Discontinuity Designs

•Assignment via baseline cutoff

•Regression of baseline measure on outcome measure

•Discontinuity (b and d) indicates intervention effect

Linden et al., J Eval Clin Pract, 2006;12, 124-131

Figure 1 Continuous evaluation of evolving BITs BIT, behavioral intervention technology

David C. Mohr , Ken Cheung , Stephen M. Schueller , C. Hendricks Brown , Naihua Duan

Continuous Evaluation of Evolving Behavioral Intervention Technologies

American Journal of Preventive Medicine Volume 45, Issue 4 2013 517 - 523

http://dx.doi.org/10.1016/j.amepre.2013.06.006

Rapid Learning Systems

• IOM Workshop on “Learning Healthcare Systems” (2006)

– “structural inability of evidence to keep pace with the need for better information to guide clinical decision making”

• Use EHR practice-based data to answer practice-based scientific questions

• Integrate mHealth into EHRs to enhance data set

• Extrapolate rapid learning to a mHealth platform

Automated RCTs for Commercialized mHealth Solutions

• Within subject method

– Stable baseline period before intervention components are initiated

– Each user serves as his own control

– Staggered N of 1s with Bayesian Estimates

• Between subject method

– Rolling out a new component or upgrade

– Automate random assignment at download

– Embed outcomes in intervention app

– Compare two conditions

THANK YOU