Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Session 4 Suicidal Ideation and Behavior:
Methodological and Technical Challenges
2013 Annual ISCTM Meeting Washington, DC
February 20, 2013
ISST-Plus Overview
• A modified version of the ISST (1997), was developed by Alphs and Lindenmayer in 2009
• Components – Part I: Suicidal Ideation—13 items (Time: Last 7 Days) – Part II: Suicidal Behavior (Time: Since last visit) – Part III: Global Suicidal ideation and behavior (Time: Now)
• Severity • Status if subject misses assessment
– Suicide Narrative (optional) (Time: Since last visit) • If the subject reports suicidal behavior (e.g., preparation,
injuries, accidents, passive or active attempt) in Part II, or if the subject completes suicide (documented in Part III), all available information will be reported in the Suicide Attempt Narrative section
• The ISST-Plus takes 15-20 minutes to complete confidential
Results of a Prospective Validation
for Mapping 3 SIB Assessment Instruments to the C-CASA
Larry Alphs, MD, PhD Janssen Scientific Affairs
February 21, 2013
• Employed by J&J • Stock in J&J • Developer of ISST-Plus
Disclaimer
4
• Valuable to have alternative scales to assess suicidal ideation and behavior (SIB)
• These scales must meet FDA requirements and map to the Columbia Classification Algorithm of Suicide Assessment (C-CASA) similar to the Columbia Suicide Severity Rating Scale (C-SSRS)
• Alternatives to the C-SSRS – InterSePT Scale for Suicidal Thinking–Plus (ISST-Plus) – Sheehan Suicidality Tracking Scale (S-STS)
Introduction
5
• To examine the concurrent validity for mapping symptoms of suicidal ideation, self-harm, and suicidal behavior to C-CASA (2010 and 2012) for – ISST-Plus – S-STS (clinician-rated) – S-STS (patient-rated) – C-SSRS
Objectives
6
• Subjects with full range of suicidality from “not present” to “extremely severe”
• Identified from inpatient and outpatient settings • Signed informed consent • Interviewed on the same day on the ISST-Plus, the S-
STS, and the C-SSRS • Independent raters administered the 3 scales in random
sequence for each subject (Figure below) • Time frame: Past 7 days • Separate consent for video recordings
Study Design
1st Interview Rater N
(Scale X)
2nd Interview Rater M (Scale Y)
3rd Interview Rater Q
(Scale Z)
7
• Mental health professionals from one academic site • Experienced in working with suicidal patients • Training provided by 3 scale authors
– semi-structured interviews – training slides and materials – standardized and uniform definitions for all critical SIB
terms • Trained to administer all 3 scales • Selected for participation on an “as available” basis
Raters
8
• Ratings from subjects who completed responses on all 3 instruments were mapped to the C-CASA (2010 and 2012)
– C-SSRS:
• Completed on paper by a trained individual
• Used predefined C-CASA mapping algorithms from the author
– ISST-Plus and S-STS:
• Direct data capture and mapping with computers
• Eliminates the possibility of missing values, double entries, and legibility problems
Instrument Mapping
9
Mapping of Assessment Instrument to C-CASA (2010)
CSSRS, ISST-Plus, S-STS
Completed suicide Suicide attempt Preparatory suicidal behavior Suicidal ideation Other, no deliberate self-harm Fatal, not enough information to classify Self-injurious act, no suicidal intent (NSSI) Self-injurious behavior, intention unknown Nonfatal, not enough information to classify
10
Mapping Algorithms C-CASA 2010 C-CASA
Code Number
C-CASA (Version 2010) Classification
ISST-Plus Map to
C-CASA* (Version 2012)
S-STS Map to
C-CASA (Version 2012)
C-SSRS Map to C-CASA
(Version 2012)
1 Completed Suicide Yes if: Part III coded as 6 A Yes response to 13 A Yes response to Completed
Suicide
2 Suicide attempt Yes if: Yes on Part II, Item 1.2 OR Yes on Part II, Item 1.3
A positive response to 10 or 16 OR A Yes response to 1b
A Yes response to Actual Attempt
3 Preparatory suicidal behavior Yes if: Part II, Item 1.4 = Yes
A positive response to 8 with at least one Level 1 on 12
A Yes response to Preparatory Acts or Behavior
4 Suicidal Ideation
Yes if: (Part I, Item 4 ≥1 OR Part I, Item 5 ≥1) AND Items 10 and 12 ≥1; AND Item 11 = 0
A positive response to 3 and 4 (and 6 or 7) AND A negative response to 5
A Yes response to Item 4 (Active Suicidal Ideation With Some Intent to Act, Without Specific Plan)
5 Other, no deliberate self-harm
Yes if: Part II, Item 1.1 = Yes; AND Part II, Item 1.2 = No
A positive response to 9 OR A positive response to 1a and a negative response to 1b
A Yes response to Has subject engaged in Nonsuicidal Self-injurious Behavior?
6 Fatal, not enough information to classify Yes if: Part III scored 8 A Yes response to 14 —
7 Self-injurious act, no suicidal intent (NSSI)
Yes if: Part II, Item 1.1 = Yes; AND Part II, Item 1.2 = No
A positive response to 9 OR A positive response to 1a and a negative response to 1b
A Yes response to “Has subject engaged in Nonsuicidal Self-injurious Behavior?”
8 Self-injurious behavior, intention unknown
Yes if: Part II, Item 1.1 = Yes; AND Part II, Item 1.2 = Unknown/NA
A positive response to 1a with 1b and 6 and 7 and 10 unanswered —
9 Nonfatal, not enough information to classify
Yes if: Part II, Item 1.0 = Unknown/NA OR Part II, Item 1.1 = Unknown/NA
Yes if question 17 is yes and C-CASA1-C-CASA13 are ALL No —
11
Mapping Algorithms C-CASA 2010 C-CASA
Code Number
C-CASA (Version 2010) Classification
ISST-Plus Map to
C-CASA* (Version 2012)
S-STS Map to
C-CASA (Version 2012)
C-SSRS Map to C-CASA
(Version 2012)
1 Completed Suicide Yes if: Part III coded as 6 A Yes response to 13 A Yes response to Completed
Suicide
2 Suicide attempt Yes if: Yes on Part II, Item 1.2 OR Yes on Part II, Item 1.3
A positive response to 10 or 16 OR A Yes response to 1b
A Yes response to Actual Attempt
3 Preparatory suicidal behavior Yes if: Part II, Item 1.4 = Yes
A positive response to 8 with at least one Level 1 on 12
A Yes response to Preparatory Acts or Behavior
4 Suicidal Ideation
Yes if: (Part I, Item 4 ≥1 OR Part I, Item 5 ≥1) AND Items 10 and 12 ≥1; AND Item 11 = 0
A positive response to 3 and 4 (and 6 or 7) AND A negative response to 5
A Yes response to Item 4 (Active Suicidal Ideation With Some Intent to Act, Without Specific Plan)
5 Other, no deliberate self-harm
Yes if: Part II, Item 1.1 = Yes; AND Part II, Item 1.2 = No
A positive response to 9 OR A positive response to 1a and a negative response to 1b
A Yes response to Has subject engaged in Nonsuicidal Self-injurious Behavior?
6 Fatal, not enough information to classify Yes if: Part III scored 8 A Yes response to 14 —
7 Self-injurious act, no suicidal intent (NSSI)
Yes if: Part II, Item 1.1 = Yes; AND Part II, Item 1.2 = No
A positive response to 9 OR A positive response to 1a and a negative response to 1b
A Yes response to “Has subject engaged in Nonsuicidal Self-injurious Behavior?”
8 Self-injurious behavior, intention unknown
Yes if: Part II, Item 1.1 = Yes; AND Part II, Item 1.2 = Unknown/NA
A positive response to 1a with 1b and 6 and 7 and 10 unanswered —
9 Nonfatal, not enough information to classify
Yes if: Part II, Item 1.0 = Unknown/NA OR Part II, Item 1.1 = Unknown/NA
Yes if question 17 is yes and C-CASA1-C-CASA13 are ALL No —
12
• Issue identified: – 5 subjects who had been injured in accidents, but
were not suicidal. • C-SSRS was not designed to categorize such subjects
and thus could not be used as a standard for them.
• Categorization of these 5 subjects by ISST-Plus and S-STS is compared separately
Instrument Mapping
13
• Baseline demographics and clinical characteristics described descriptively
• Levels of agreement between mappings to the C-CASA of the same subject from ISST-Plus, S-STS, and C-SSRS were summarized using diagnostic characteristics measures – sensitivity – specificity – positive predictive value [PPV] – negative predictive value [NPV] – area under the receiver operating characteristic [ROC] curve)
• Tested using McNemar’s test • Kappa coefficients were computed for each mapping variable
Statistical Analysis
14
• Sensitivity: When present according to the standard reference scale, how often does the diagnostic detect the condition? A/(A + C)
• Specificity: When absent according to the standard reference scale, how often does the diagnostic say the condition is absent? D/(D + B)
• Positive Predictive Value: When present according to the diagnostic, how often does the standard reference scale agree? A/(A + B)
Definitions
Standard Reference (C-SSRS)
Present Absent Diagnostic
Scale Present A B
Absent C D
15
• Negative Predictive Value: When absent according to the diagnostic, how often does the standard reference scale agree? D/(C + D)
• Area Under the ROC Curve: What is the probability that a randomly selected subject with the trait is more likely to be categorized as having the trait relative to a randomly selected subject without the trait? When both the predictor and the criterion variables are binary, this is calculated by (Sensitivity + Specificity)/2
Definitions
Standard Reference (C-SSRS)
Present Absent Diagnostic
Scale Present A B
Absent C D
16
Subject Characteristics
Characteristic Description
Sample size 45
Raters 5
Videotapes 40
Age 39.9 (15.0) years (range, 19-73)
Sex 65.6% female
Race 73.3% white, 24.4% black, and 2.2% mixed race
Age 1st psychiatric symptoms 16.6 (10.4) years (range, 5-42)
Age 1st psychiatric treatment 24.5 (13.3) years (range, 5-58)
Age 1st psychiatric hospitalization 30.8 (14.7) years (range, 10-68)
Number past suicide attempts Mean (SD) 2.1 (2.8) (range, 0-13)
17
Sensitivity of Each Scale by C-CASA (2010) on Classification Relative to the C-SSRS
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
2 3 4 7 8
Excellent
Good
Acceptable
ISST+ S-STSp S-STSc
18
Suicide Attempt Preparatory Acts Suicidal Ideation Self Injury, No Suicide Intent
Self Injury, Intent Unknown
Specificity of Each Scale by C-CASA (2010) on Classification Relative to the C-SSRS
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
2 3 4 7 8
Excellent
Good
Acceptable
ISST+ S-STSp S-STSc
19
Suicide Attempt Preparatory Acts Suicidal Ideation Self Injury, No Suicide Intent
Self Injury, Intent Unknown
NPV of Each Scale by C-CASA (2010) on Classification Relative to the C-SSRS
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
2 3 4 7 8
Excellent
Good
Acceptable
ISST+ S-STSp S-STSc
20
Suicide Attempt Preparatory Acts Suicidal Ideation Self Injury, No Suicide Intent
Self Injury, Intent Unknown
PPV of Each Scale by C-CASA (2010) on Classification Relative to the C-SSRS
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
2 3 4 7 8
Excellent
Good
Acceptable
ISST+ S-STSp S-STSc
21
Suicide Attempt Preparatory Acts Suicidal Ideation Self Injury, No Suicide Intent
Self Injury, Intent Unknown
Mapping to C-CASA 2010 % Disagreement with C-SSRS
22
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
CCA
SA 1
CCA
SA 2
CCA
SA 3
CCA
SA 4
CCA
SA 5
CCA
SA 6
CCA
SA 7
CCA
SA 8
CCA
SA 9
ISST
SSTS
• The ISST-Plus, the S-STS, and the C-SSRS are all potentially valuable instruments for identifying and categorizing SIB
• None of the 3 scales is clearly and consistently superior to the other 2
Discussion: C-CASA 2010 Mapping
23
• Differences in categorization observed for the 3 scales may be related to – Variability in patient reports – Differences in rater reliability – Differences in the instruments’ approaches to collecting
SIB data – The relatively low base rates of most categorical hits
• NPVs are relatively better than the PPVs. • Inherently tied to base rates • Would change (potentially by a substantial amount ) if the
frequency of the constructs of interest changed
Discussion: C-CASA 2010 Mapping
24
Mapping of Assessment Instrument to C-CASA (2012)
25
Mapping of Assessment Instrument to C-CASA (2012)
26
Additional highly desirable categories from C-CASA 2010
Mapping Algorithms
C-CASA Code Number
C-CASA (Version 2012) Classification
ISST-Plus Map to
C-CASA* (Version 2012)
S-STS Map to
C-CASA (Version 2012)
C-SSRS Map to C-CASA
(Version 2012)
1 Passive Suicidal Ideation
Yes if: Part I, Item 1 ≥1 A positive response to 2 A Yes response to Item 1 (Wish
to be Dead)
2 Active Suicidal Ideation: Nonspecific (no method, intent, or plan)
Yes if: (Part I, Item 4 ≥1 OR Part I, Item 5 ≥1) AND Part I Items 10, 11, 12 = 0
A positive response to 3 AND A negative response to 4 and 5 and 6 and 7
A Yes response to Item 2 (Nonspecific Active Suicidal Thoughts)
3 Active Suicidal Ideation: Method, but no intent or plan
Yes if: (Part I, Item 4 ≥1 OR Part I, Item 5 ≥1) AND Part I, Item 10 ≥1; AND Part I, Items 11, 12 = 0
A positive response to 3 and 4 AND A negative response to 5 and 6 and 7
A Yes response to Item 3 (Active Suicidal Ideation With Any Methods (no plan) Without Intent to Act)
4 Active Suicidal Ideation: Method and intent, but no plan
Yes if: (Part I, Item 4 ≥1 OR Part I, Item 5 ≥1) AND Items 10 and 12 ≥1; AND Item 11 = 0
A positive response to 3 and 4 (and 6 or 7) AND A negative response to 5
A Yes response to Item 4 (Active Suicidal Ideation With Some Intent to Act, Without Specific Plan)
5 Active Suicidal Ideation: Method, intent, and plan
Yes if: (Part I, Item 4 ≥1 OR Part I, Item 5 ≥1) AND Part I, Items 10, 11, and 12 ≥1
A positive response to 3 and 4 and 5 (and 6 or 7)
A Yes response to Item 5 (Active Suicidal Ideation With Specific Plan and Intent)
6 Completed Suicide Yes if: Part III coded as 6 A Yes response to 13 A Yes response to Completed
Suicide
7 Suicide Attempt
Yes if: Yes on Part II, Item 1.2 OR Yes on Part II, Item 1.3
A positive response to 10 or 16 OR A Yes response to 1b
A Yes response to Actual Attempt
8 Interrupted Suicide Attempt
Yes if: Part II, Item 2.3 ≥1
A positive response to 8 with at least one Level 3 on 12
A Yes response to Interrupted Attempt
27
Mapping Algorithms
C-CASA Code Number
C-CASA (Version 2012) Classification
ISST-Plus Map to
C-CASA* (Version 2012)
S-STS Map to
C-CASA (Version 2012)
C-SSRS Map to C-CASA
(Version 2012)
9 Aborted Suicide Attempt
Yes if: Part II, Item 2.2 ≥1
A positive response to 8 with at least one Level 2 on 12
A Yes response to Aborted Attempt
10
Preparatory acts toward imminent suicidal behavior—not counting Aborted or Interrupted Attempts
Yes if: Part II, Item 1.4 = Yes
A positive response to 8 with at least one Level 1 on 12
A Yes response to Preparatory Acts or Behavior
11 Self-injurious Behavior Without Suicidal Intent
Yes if: Part II, Item 1.1 = Yes; AND Part II, Item 1.2 = No
A positive response to 9 OR A positive response to 1a and a negative response to 1b
A Yes response to Has subject engaged in Nonsuicidal Self-injurious Behavior?
12 Self-injurious Behavior, Intent Unknown Yes if: Part II, Item 1.1 = Yes; AND Part II, Item 1.2 = Unknown/NA
A positive response to 1a with 1b and 6 and 7 and 10 unanswered
Cannot Evaluate —
13 Not enough information (fatal) Yes if: Part III scored 8 A Yes response to 14 Cannot Evaluate —
14 Not enough information (nonfatal)
Yes if: Part II, Item 1.0 = Unknown/NA OR Part II, Item 1.1 = Unknown/NA
Yes if question 17 is yes and C-CASA1-C-CASA13 are ALL No
Cannot Evaluate —
15 Other (accidental, psychiatric medical), no deliberate self-harm
Yes if: Part II, Item 1.0 = Yes; AND Part II, Item 1.1 = No
(A positive or blank response to 1 and a negative response to questions 2, 3, 4, 5, 6, 7, 8, 9, 10, 13, and 14) OR A positive response to 15
Cannot Evaluate —
28
Mapping Comparison • Not all categories depicted
– Completed suicide; fatal, not enough information; and nonfatal, not enough information, respectively) are not depicted here; these items did not occur in the subjects surveyed
– Therefore, 100% agreement
• The 5 subjects identified as “recent accident without suicide intent”
– On ISST-Plus and S-STS Clinician Form, all these subjects mapped with 100% agreement to Category 15 (other injury or overdose, no suicide intent)
Results: C-CASA 2012
29
Area Under ROC Curve for Each Scale by C-CASA (2012) Classification Relative to the C-SSRS
Excellent
Good
Acceptable
ISST+ S-STSp S-STSc
30
Ideation Passive
Ideation Nonspecific
Active Method
Method, Intent
Method, Intent, plan
Suicide Attempt
Interrupted Aborted
Self Injury Not suicide
Preparatory Acts
Area Under ROC Curve for Each Scale by C-CASA (2012) Classification Relative to the C-SSRS
Excellent
Good
Acceptable
ISST+ S-STSp S-STSc
31
Ideation Passive
Ideation Nonspecific
Active Method
Method, Intent
Method, Intent, plan
Suicide Attempt
Interrupted Aborted
Self Injury Not suicide
Preparatory Acts
Sensitivity of Each Scale by C-CASA (2012) Classification Relative to the C-SSRS
Excellent
Good
Acceptable
ISST+ S-STSp S-STSc
32
Ideation Passive
Ideation Nonspecific
Active Method
Method, Intent
Method, Intent, plan
Suicide Attempt
Interrupted Aborted
Self Injury Not suicide
Preparatory Acts
Specificity of Each Scale by C-CASA (2012) Classification Relative to the C-SSRS
Excellent
Good
Acceptable
ISST+ S-STSp S-STSc
33
Ideation Passive
Ideation Nonspecific
Active Method
Method, Intent
Method, Intent, plan
Suicide Attempt
Interrupted Aborted
Self Injury Not suicide
Preparatory Acts
NPV for Each Scale by C-CASA (2012) Classification Relative to the C-SSRS
Excellent
Good
Acceptable
ISST+ S-STSp S-STSc
34
Ideation Passive
Ideation Nonspecific
Active Method
Method, Intent
Method, Intent, plan
Suicide Attempt
Interrupted Aborted
Self Injury Not suicide
Preparatory Acts
PPV for Each Scale by C-CASA (2012) Classification Relative to the C-SSRS
Excellent
Good
Acceptable
ISST+ S-STSp S-STSc
35
Ideation Passive
Ideation Nonspecific
Active Method
Method, Intent
Method, Intent, plan
Suicide Attempt
Interrupted Aborted
Self Injury Not suicide
Preparatory Acts
Mapping to C-CASA 2012 % Disagreement with C-SSRS
36
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
CCA
SA2
-1
CCA
SA2
-2
CCA
SA2
-3
CCA
SA2
-4
CCA
SA2
-5
CCA
SA2
-6
CCA
SA2
-7
CCA
SA2
-8
CCA
SA2
-9
CCA
SA2
-10
CCA
SA2
-11
CCA
SA2
-12
CCA
SA2
-13
CCA
SA2
-14
CCA
SA2
-15
ISST
SSTS
• The ISST-Plus, the S-STS, and the C-SSRS remain potentially valuable instruments for identifying and categorizing SIB
• Reliability of mapping the C-SSRS, the ISST-Plus, and the S-STS scales to the C-CASA (2012) was considerably worse than reliability of mapping to the C-CASA (2010)
• The small sample size highlights the problems of ratios as psychometric descriptors
• The domains of the C-CASA (2012) are not empirically derived and may inadequately represent the reality of SIB
Discussion: C-CASA 2012 Mapping
37
• Greatest disagreement between the C-SSRS and the ISST-Plus and the S-STS
– At “intermediate levels” of active suicidal ideation (2, 3, and 4) (i.e., short of including method, with intent and plan)
– Agreement between the scales was high when the C-SSRS fails to endorse these 3 categories.
– Conversely, agreement between the 3 scales was low when the C-SSRS endorsed any these 3 categories
• ISST-Plus and the S-STS endorsed categories 2, 3 and 4, 0% to 15% of the time
• C-SSRS endorsed 55% to 70% of the time
Discussion: C-CASA 2012 Mapping
38
• Differences in categorization may be related to – Variability in patient reports – Differences in rater reliability – Differences in the instruments’ approaches to
collecting SIB data and navigating through possible alternatives
– The relatively low base rates of most categorical hits
• NPVs are relatively better than the PPVs. • Inherently tied to base rates • Would change (potentially by a substantial amount ) if
the frequency of the constructs of interest changed
Discussion: C-CASA 2012 Mapping
39
• The sample size of the study was modest
• Kappa scores affected by low base rates for endorsement
• Lack of variation in C-CASA Categories 6, 12, 13, and 14 precludes analysis of these categories.
• Emphasis of the C-SSRS on suicidal patients (without addressing of non-suicidal patients) precluded its use as a standard reference scale for C-CASA Category 15
• Assumes a unidimensional construct
Limitations
40
• Definitions of terms like “intent,” “plan,” “aborted attempt,” “interrupted attempt,” are inadequate: – Unclear, idiosyncratic, overlapping, not consistent with field or general
population usage and, so, lead to confusion • Unclear what information should be counted in each category • Domains like “intent,” “plan,” “method” are not mutually exclusive and are
not empirically derived. Important domains may be missing • Important domains of classification for clinical trials are missing • Even with the identified groupings 11 of 16 possible classifications are
missing • Assumes a simple uni-dimensional construct that does not reflect the
clinical situation • Problems in defining the baseline
Problems Identified with C-CASA 2012
41
Limitations: Combinations of Active Ideation, Method, Intent and Plan
42
16 combinations possible. Only 5 of these are covered by C-CASA and by C-SSRS.
Combin-ation
Passive Ideation
Active Ideation Intent Method Plan C-CASA 1
C-CASA 2 C-CASA 3 C-CASA 4 C-CASA 5
? covered by C-CASA
Can this occur?
1 0 0 0 0 0
Covered by a No to
1,2,3,4,5
2 0 1 0 0 0 X Covered in
Q 2 3 0 0 1 0 0 Not covered Yes 4 0 1 1 0 0 X x 5 0 0 0 1 0 Not covered Yes 6 0 1 0 1 0 X x 7 0 0 1 1 0 Not covered Yes 8 0 1 1 1 0 X x 9 0 0 0 0 1 Not covered Yes
10 0 1 0 0 1 X Not covered Yes 11 0 0 1 0 1 Not covered Yes 12 0 1 1 0 1 X x 13 0 0 0 1 1 Not covered Yes 14 0 1 0 1 1 X Not covered Yes 15 0 0 1 1 1 Not covered Yes 16 0 1 1 1 1 X x
Based on C-CASA 2012 Definitions
• The ISST-Plus, the S-STS, and the C-SSRS remain potentially valuable instruments for identifying and categorizing SIB
• Additional psychometric validation work is necessary – Assessment of inter-rater and intra-rater reliability – Sensitivity to change
• The C-CASA 2012 should be clarified to meet needs of clinical trials
Conclusions: C-CASA Mapping
43
• How do you distinguish one thought from another? • When does one thought end and another begin? • Can we distinguish between conscious and unconscious
thought or is that on a continuum? • Do unconscious thoughts count? Dreams? • Can you have more than one thought at a time? • Can you hold opposing thoughts at the same time? • Is the vocabulary being used by the rater understood in
the same way by the subject? • Is the vocabulary being used understood in the same
way by all subjects?
Methodological Questions
44
• David V. Sheehan, USF
• David Williamson, Janssen
• Lian Mao, Janssen
• Christopher Gray, Consultant
• Qin Li, Janssen
• Richard Shelton, UAB
• Roberta May, UAB
• Xiaohua Li, UAB
• Cheryl McCullumsmith, UAB
• Emily Hammond, UAB
Thank you
45