JOURNALOF

PSYCHIATRIC

Journal of Psychiatric Research 38 (2004) 497–502

www.elsevier.com/locate/jpsychires

RESEARCH

Improvement of anger at one week predicts the effects ofsertraline and placebo in PTSD

Jonathan Davidsona,*, Lawrence R. Landermana, Cathryn M. Claryb

a Department of Psychiatry and Behavioral Sciences, Duke University Medical Center, Trent Drive, Yellow Zone, Box 3812, 4th Floor,

Room 4082B, Durham, NC 27710, USAb Pfizer Inc., NY 10017 USA

Received 26 March 2003; received in revised form 18 December 2003; accepted 8 January 2004

Abstract

In previous work we demonstrated an early, robust and sustained effect for sertraline vs placebo on irritability and anger in

subjects with PTSD. In this report, we explore the same dataset to assess whether a clinician might usefully predict ultimate response

to sertraline, on the basis of its effect upon anger after one week.

Three hundred and eighteen subjects were assessed. Outcomewasmeasured by whether or not the score was reduced by at least 50%

from baseline. Ordinary least squares regression was used to estimate the effects of change in anger at one week. Logistic regression was

applied to estimate the effects on odds of a 50% drop in score. Cut points were developed for one-week change scores on anger for

sertraline and placebo. The best cut point was selected as predictive of non-response, i.e. a cue suggesting that treatment switch would

be in order. An increase in anger of 30% at one-week best predicted the likelihood of not responding to treatment in both the drug and

placebo groups. Twenty-five percent of all non-responders were incorrectly identified, while only 7% of all improvers were incorrectly

categorized as non-responders using this cutoff. Our findings imply that, for patients similar to those in this study, an increase in anger

after one week of treatment might be one factor to consider when making a decision about continuation of the medication.

� 2004 Elsevier Ltd. All rights reserved.

Keywords: Anger; Predictor; Response; Sertraline; Placebo; PTSD

1. Introduction

In a study which looked at the effects of sertraline and

placebo on individual symptoms of PTSD, we noted a

broad-spectrum effect on practically all symptoms, with

notable improvement of irritability/anger after oneweek, which was sustained throughout the remaining

period of treatment. The magnitude of this change was

significantly greater for sertraline than placebo at every

visit, starting at week one. In further exploring the early

improvement on anger, we found it explained some of

the subsequent improvements in other symptoms (Da-

vidson et al., 2002). The robust nature of this finding led

us to ask whether the early changes in anxiety/irritability

* Corresponding author. Tel.: +1-919-684-2880; fax: +1-919-684-

8866.

E-mail address: jonathan.davidson@duke.edu (J. Davidson).

0022-3956/$ - see front matter � 2004 Elsevier Ltd. All rights reserved.

doi:10.1016/j.jpsychires.2004.01.005

could perhaps serve as a predictor of likely response or

non-response over the ensuing weeks.

We therefore present a report which assesses the im-

portance of early change in anger after one week of

treatment. In particular, we wish to know whether, on

the basis of change in anger at one week, the cliniciancould determine whether or not the likelihood of an

unsuccessful outcome was strong enough that a change

in treatment might be considered. In other words, we are

concerned mostly with early detection of non-response

at 12 weeks, defined as a reduction of 50% or more in

PTSD symptoms.

2. Materials and methods

2.1. Study design and procedures

The samples were obtained from two almost identical

clinical studies of sertraline vs placebo in posttraumatic

498 J. Davidson et al. / Journal of Psychiatric Research 38 (2004) 497–502

stress disorder (Brady et al., 2000; Davidson et al.,

2001), differing in the length of placebo lead-in (one or

two weeks respectively). Subjects were male and female

outpatients aged at least 18 years, and met criteria for

PTSD, based upon the Diagnostic and StatisticalManual of Mental Disorders Revised, 3rd Edition

(DSM-IIIR) (American Psychiatric Association, 1987).

At least six months illness was required, along with a

minimum score of 50 on the Clinician Administered

PTSD Scale version that assesses symptoms over the

past week (CAPS-II) (Blake et al., 1995). Other inclu-

sion criteria, which are listed in full detail elsewhere

(Brady et al., 2000; Davidson et al., 2001) included ab-sence of psychotropic drugs for 2 weeks, non-pregnant

status, absence of lifetime bipolar, schizophrenia or

other psychotic disorder, or alcohol or substance use

disorders within six months, clinically unstable disorder

and use of cognitive behavioral therapy during the trial.

With respect to comorbid depression, investigators used

clinical judgment to determine which was primary (i.e.

at the time of study, was the most clinically salient).Such judgment was based on information obtained via

clinical history and the numerous structured rating

scales. It was necessary for PTSD to be the primary

disorder. Each study had received institutional review

board approval, and written informed consent was ob-

tained from all subjects.

Treatment was preceded by a one to two week single-

blind placebo lead-in, following which subjects wererandomized to receive sertraline or placebo for 12

weeks, on a double-blind basis. Among the various as-

sessments, the Davidson Trauma Scale (DTS) (David-

son et al., 1997) was one of the main self-rating scales

for assessing PTSD symptom severity.

2.1.1. Statistical methods

In the initial analyses we examine (1) whether declinein anger/irritability between baseline and week one

predicts a favorable outcome after 12 weeks; (2) whether

the impact of decline in anger is the same in the ser-

traline and placebo groups. Based upon findings in (1)

and (2), we estimate the sensitivity and specificity Insel

and Goodwin (1983) of various cut points for early

change in anger, and then present an empirically-based

threshold for initial (i.e. one week) change in anger,which can be used as an indication that the likelihood of

a successful outcome is sufficiently small that a change in

treatment could be considered.

2.1.2. Sample

Forty-three subjects were excluded due to insufficient

baseline anger and 24 were dropped for additional rea-

sons, as follows. From an original sample of 385, whosedemographic and clinical characteristics were described

elsewhere (Brady et al., 2000; Davidson et al., 2001), we

selected 342 respondents with a score of 2 or more on

the DTS anger item at baseline. These included 173 in

the treatment group, and 169 controls. 23.9% were male

and 76.1% female; 84.9% were white, 11.0% black, 2.6%

Asian and 2.5% other. 10.6% were between ages 18–24,

27% between 25–34, 31.8% between 35–44, 29.9% be-tween 45.64, 0.7% 65 or greater. Fifty-one (of 342)

sample members dropped out of the study prior to week

12. Eight of these provided no follow-up data on PTSD

symptoms, and were dropped from the analyses. For the

remaining 43, we imputed the week 12 summary symp-

tom measure by carrying the last observation forward.

In preliminary analyses, we regressed imputed and un-

imputed versions of the summary symptom measure on(baseline to wave 1) change in anger. Results were the

same across versions. An additional 8 respondents were

dropped because they were missing on summary symp-

tom data at baseline, and 8 more respondents were

dropped because they were missing on (baseline or week

1) measures of anger needed to construct change in

anger. We were left with an analysis sample of 318. Of

the 24 subjects dropped for missing data 19 (11%) be-longed to the treatment group, while 5 (3.1%) belonged

to the placebo group.

2.1.3. Measures

2.1.3.1. Change in anger. In initial analyses, we measured

change in anger as (baseline to week 1) raw change in

PTSD score on anger, and percent change on the samemeasure. For the analyses described below, results using

the measure of raw change were virtually identical to

those using percent change. We report results for the

percent change measure below. While models using both

measures had adequate fit (Hosmer and Lemeshow,

1989), those based on the percent change measure had

slightly better scores on this and other (Somers’s D,

Gamma) standard measures of model fit (Cox and Snell,1989; SAS Institute, 1990). Anger was assessed by

summing the total frequency (0–4) and severity (0–4)

scores for item 14 on the DTS. The scale asks subjects to

assess whether they have experienced outbursts of anger

or been irritable in the past week. The specific wording

for this question is as follows: ‘‘have you been irritable

or had outbursts of anger?’’ The frequency ranges from

not at all (0), once only (1), 2–3 times (2), 4–6 times(3), and every day (4). The severity choices are: not at

all distressing (0), minimally distressing (1), moderately

distressing (2), markedly distressing (3), and extremely

distressing (4). The total score thus ranges from 0–8.

2.1.3.2. PTSD score at week 12. For the analyses below,

the summary PTSD score was first recalculated with the

anger item excluded. Two dependent variables were then

created. The first is percent change in total PTSD (minus

the anger item, i.e. the 16 remaining items) between

baseline and week 12. Assuming the relationship between

J. Davidson et al. / Journal of Psychiatric Research 38 (2004) 497–502 499

change in anger and decline in other PTSD symptoms

after 12 weeks is continuous, this outcome most closely

mirrors the relationship under consideration, and pro-

vides optimal statistical power. The second measure,

whether or not a respondent reported a 50% decline inoverall PTSD symptoms at week 12, provides a measure

of clinicallymeaningful change. Both dependent variables

were used in the analyses to ensure that dichotomizing the

clinical measure did not bias the findings, and results for

both are reported below. As with change in anger, we

compared measures using percent and raw change in our

dependent variable in initial analyses, and results were the

same across measures.

2.1.3.3. Treatment. The sample included 173 respon-

dents treated with sertraline and 169 controls. As a

potential confounder, treatment is included as a control

in our statistical models. We also test whether the rela-

tionship between change in anger and our dependent

variables is the same across levels of treatment.

2.1.3.4. Statistical models. We used ordinary least

squares regression to estimate the effects of change in

anger and treatment on percent change in Summary

PTSD symptoms, and logistic regression to estimate

their effects on the odds of a 50% or greater drop in the

summary symptom measure. To protect against over-

fitting the sample data, we use bootstrap methods

(Efron and Gong, 1983) with logistic models used todevelop cut-points for clinicians. One hundred replicate

samples of n ¼ 318 were used in these analyses.

In each case, we estimated a model of the general form:

Y ¼ aþ b1ðchange in angerÞ þ b2ðtreatmentÞþ b3ðbaseline PTSD sxÞ þ b4ðbaseline angerÞ; ð1Þ

where b1 estimates the effect of change in anger andb2–b4 estimate the effects of controls (treatment, baseline

summary sx, baseline anger).

For each dependent variable, interactions of change

in anger with treatment were tested by initially adding a

treatment by change in anger product term to the model.

In preliminary analyses, we also tested for nonlinear

Table 1

Means and standard deviations (mean, sd) on the measures used in this stud

Total (n ¼ 318),

Mean (s.d.)

Treatm

Mean

Baseline (raw) anger score 4.88 (1.76) 4.82 (1

(Raw) anger score at week 1 4.04 (2.30) 3.62 (2

% change in anger (baseline-week 1) )15%b (50%) )23%

Baseline (raw) PTSD score 4.32 (1.48) 4.34 (1

(Raw) PTSD score at week 12 2.84 (1.85) 2.45 (1

% change in PTSD (baseline-week 12) )33% (40%) )44%Proportion with 50% decline in PTSD 0.35 (0.48) 0.45 (0

(baseline-week 12)a F -test difference in group means by treatment vs placebo.b Standard deviations not reported for percentages or proportions.

effects for baseline PTSD, baseline anger, and change in

anger by adding quadratic terms to relevant models. No

interactions or nonlinearities were present, and the in-

teraction and quadratic terms were dropped from the

model.

3. Results

3.1. Descriptive statistics

Table 1 presents descriptive statistics on the inde-

pendent and dependent variables for the sample as awhole and for the treatment and placebo groups sepa-

rately. The results in Table 1 indicate that sertraline is

significantly associated with a lower mean anger score

than is placebo at week 1 (3.62 vs 4.43) and with a

greater percent decline in anger between baseline and

week 1 ()23% vs )7%). Sertraline is also associated with

a lower overall mean DTS item score at week 12 (2.45 vs

3.21), a larger decline in PTSD symptoms from baselineto week 12 ()44% vs )23%) and an increased probability

of a 50% decline in PTSD symptoms during the period

of observation (0.45 vs 0.26). All but the last finding

have been reported in more detail elsewhere (Davidson

et al., 2002), and serve as the starting point for the issues

addressed below.

3.2. Initial change in anger and PTSD symptoms

Table 2 presents the relationship between initial

change in anger and change in the PTSD summary score

over 12 weeks. Table 2 shows that initial decline in anger

is a significant predictor of both percent decline in the

PTSD summary score and the likelihood of a 50% de-

cline in overall PTSD symptoms. In magnitude, the re-

gression effect of a 100% initial decline in anger results ina 23% decline in the PTSD summary score (Table 2,

Panel A), and multiplies the odds of a 50% decline by

3.28 (Table 2, Panel B). Both these effects are slightly

larger than the effects of treatment on these outcomes.

Both treatment by decline in anger product terms were

y

ent (n ¼ 154),

(s.d.)

Placebo (n ¼ 164),

Mean (s.d.)

F a P -value

.70) 4.95 (1.82) 0.64 0.522

.24) 4.43 (2.29) 3.18 0.002

(50%) )7% (49%) 2.83 0.005

.43) 4.29 (1.53) 0.29 0.774

.78) 3.21 (1.83) 3.76 <0.001

(36%) ).23% (42%) 4.76 <0.001

.50) 0. 26 (0.44) 3.77 <0.001

500 J. Davidson et al. / Journal of Psychiatric Research 38 (2004) 497–502

non-significant, indicating that the relationship between

initial change in anger and an outcome did not vary

across treatment groups, i.e. they applied alike to ser-

traline and placebo. Taken together, the findings in

Tables 1 and 2 indicate that while treatment is associ-ated with an initial decline in anger, the effect of this

initial decline is the same in the treatment and placebo

groups.

In supplemental analyses designed to ensure that the

results in Table 2 were not an artifact of how initial

change in anger was measured, we replicated the models

in Table 2 substituting raw change in anger, and average

change in anger from baseline to week 2. In both cases,results were the same.

3.3. Developing a cut-point based on initial change in

anger

Building on the results in Tables 1 and 2, we devel-

oped cut-points based on initial change in anger which

can be used to determine that the probability of im-provement (defined as a 50% decline in PTSD symp-

toms) is sufficiently minimal, that in some circumstances

it may prompt the clinician to consider an early change

of treatment. As noted above, our study sample was

obtained from two nearly identical clinical samples of

sertraline vs placebo-treated respondents. In the first

sample, we used logistic regression to estimate the

probability of non-response to treatment across levels ofinitial change in anger. As we wish to identify those who

do not improve, we reverse coded the dependent vari-

able and estimated the effect of initial change in anger on

the likelihood of non-response to treatment. Predicted

probabilities from this first clinical sample were used to

Table 2

The relationship between (baseline to week 1) change in anger and change i

Panel A: The effect of decline in anger on % change in the PTSD summary

ba

% decline in anger (baseline to week 1) )0.23��

treatment )0.17��

Baseline anger 0.00

Baseline PTSD score )0.02Treatment� decline in anger nsc

R2 0.15��

Panel B: The effect of decline in anger on the likelihood of a 50% decline in

ORb

% decline in anger (baseline to week 1) 3.28��

Treatment 0.16

Baseline anger 1.08

Baseline PTSD score 0.86

Treatment� decline in anger nsc

R2 (logistic) 0.12��

* Indicates p < 0:05.aOrdinary least squares (OLS) metric b coefficient measures change in PTbLogistic multiplicative coefficient measures change in the odds a 50% droc Treatment by decline in anger product term was non-significant and dro** Indicates p < 0:01.

develop cut-points which were then tested on the second

clinical sample. To protect against over-fitting the

sample data, we used bootstrap methods (Efron and

Gong, 1983) to estimate logistic models for the proba-

bility of no improvement. Because the predicted prob-abilities (for each level of change in anger) used to

calculate cut-points, sensitivities and specificities (re-

ported below) are based on mean values across 100

replicates, the likelihood that our results are based on

an unrepresentative sample is greatly reduced. Because

the relationship between change in anger and the

probability of no improvement was the same in the

first and second study samples, and because sensitivi-ties, specificities, and positive predictive values were the

same in both samples when calculated separately, we

present results for the combined sample below. The re-

sults of these final analyses are given in Table 3 below

where a ‘positive’ result refers to one of non-response to

treatment.

For different potential cut-points (levels of initial

change in anger), Table 3 gives the probability of notimproving at endpoint, together with the sensitivity,

specificity, and predictive value of a positive test. The

predicted probability of no improvement in Table 3

varies from 0.389 for a 100% decrease in anger to 0.919

for a 100% increase in anger. Given our purpose, we are

most concerned with not changing treatment among

those who will ultimately have a successful outcome. At

the same time, we seek a potential cut-point that willenable us to identify a sufficient number of those with a

poor prognosis to be useful as a screening device. The

results in Table 3 indicate that using a 30% (or greater)

initial increase in anger can meet both criteria. Results

using this cut-point are summarized in Table 4. Readers

n PTSD symptoms over 12 weeks

score

P<0.001

<0.001

0.812

0.276

PTSD Symptoms

P<0.001

<0.001

0.349

0.130

SD summary score per unit change in a predictor.

p in PTSD symptoms per unit change in a predictor.

pped from the model.

J. Davidson et al. / Journal of Psychiatric Research 38 (2004) 497–502 501

wishing to use a different cut-point can use the infor-

mation in Table 3 to do so.

The results in Table 4 indicate that using a cut-point

of a 30% increase in anger after 1 week enables us to

identify about 26% (53/206) of those who will not makea substantial improvement. An additional 8 respondents

who make improvement are incorrectly included in this

group resulting in a false positive rate of (8/112)¼ 7.1%.

In supplemental analyses, we attempted to improve our

predictive power (and especially sensitivity) by first us-

ing an increase in anger from baseline to two weeks as

our predictor, but sensitivity and predictive power gen-

erally were lower in these models. Results using changein anger averaged over 2 weeks (D¼ [week 1 change +

week change]/2) were about the same as those reported

in Tables 3 and 4.

4. Discussion

A number of interesting findings have emerged fromthis analysis. Firstly, we were able to demonstrate that

complete resolution of anger at week one is associated

with a 23% decline in the overall PTSD score (with

anger deleted) at endpoint, or increases the odds of a

50% reduction in DTS by over 3 times, each effect being

somewhat larger than the effects of treatment on out-

Table 3

Probability of no improvement, sensitivity, specificity, and positive predicted

% Change in

anger

Probability of no

improvement

True negative True positive R

100 0.919 0 10

80 0.869 1 4

70 0.852 0 2

60 0.845 0 1

50 0.824 2 8

40 0.805 0 1

30 0.780 5 27

20 0.756 8 11

10 0.748 0 3

0 0.708 24 50

)10 0.671 5 9

)20 0.653 8 17

)30 0.617 13 26

)40 0.589 4 7

)50 0.555 10 5

)60 0.523 1 7

)70 0.500 2 0

)80 0.468 2 1

)100 0.389 27 17

Total 112 206 3aDefining predicted positives as all sample members whose row probabilit

given row, the sensitivity is the proportion of true positives who are predicted

is (10+4)/206¼ 0.068.bDefining predicted positives as in note (a) above, the specificity is [1 – the f

negatives who are incorrectly included with the predicted positives. For thecDefining predicted positives as in note (a), the positive predicted value for a

predicted positives who are true positives. For the second row from the top

come. Our findings applied both to the sertraline and

placebo conditions, and in this regard are not unlike the

findings of early improvement as a predictor of final

outcome from sertraline and placebo in panic disorder

(Pollack et al., 2002). We suspect that in PTSD, bothactive pharmacotherapy and placebo set in motion some

form of common internal healing process, as has been

speculated earlier in the case of depression by Stassen et

al. (1993); what is most important, however, is that this

process is initiated more frequently (and perhaps more

successfully) by active drug than by placebo. The early

improvement in anger/hostility may be one of the first

manifestations of this process at work, and is consistentwith some of the depression literature (Katz et al., 1987).

In terms of the practical implications, the question

was posed as to whether the clinician could usefully

engage in decision making about whether to continue or

switch treatment after one week. We found that any

increase of anger by at least 30% over baseline (i.e. a

score which may have gone from two to three, or six to

eight on the DTS) fulfilled the two criteria we had setforth, i.e. (1) to seek a potential cut point that would

permit the identification of a sufficient number of those

with a poor prognosis as to be useful and (2) to avoid

changing treatment among those who would ultimately

have a good likelihood of response. This threshold was

met by 61 subjects or 19% of the sample. Eight of these

value across levels of initial (baseline to week 1) change in anger

ow frequency Sensitivitya Specificityb Positive predictive

valuec

10 0.049 1.00 1.00

5 0.068 0.991 0.933

2 0.078 0.991 0.941

1 0.083 0.991 0.944

10 0.121 0.973 0.893

1 0.126 0.973 0.897

32 0.257 0.929 0.869

19 0.311 0.857 0.800

3 0.325 0.857 0.807

74 0.568 0.643 0.745

14 0.617 0.598 0.737

25 0.694 0.527 0.730

39 0.820 0.411 0.719

11 0.854 0.375 0.715

15 0.879 0.286 0.693

8 0.913 0.277 0.699

2 0.913 0.259 0.694

3 0.917 0.241 0.690

44 1.00 0.00 0.648

18

y (of no improvement) is greater than or equal to the probability for a

positives For example, for the second row from the top, the sensitivity

alse positive rate], where the false positive rate is the proportion of true

second row from the top, the specificity is 1)(0+1)/112¼ 0.991.

given row is the number of true positives divided by (the proportion of

, the predicted positive value is 14/(14+1)¼ 0.933.

Table 4

Results of using an increase of 30% or more as a cut-point to identify (predict) response to treatment

Predicted response

Non-response (Positive)

(>30% " in anger)

Response (Negative)

(<30% " in anger)

Total n

Actual treatment response Non-response (Positive)

(<50% # in DTS score)

53 153 206

Response (Negative)

P50% " in DTS score

8 104 112

Total 61 257 318

specificity¼ [1)(8/112)]¼ 0.928

sensitivity¼ 53/206¼ 0.257

502 J. Davidson et al. / Journal of Psychiatric Research 38 (2004) 497–502

went on to show a good response for a positive predic-

tive value of 0.87.

In summary, initial decline in anger is a significant

predictor of ultimate treatment response, whether mea-

sured by percent decline in the DTS summary score or

as the likelihood of a 50% drop in score. The relation-

ship between early change in anger and ultimate out-

come was the same in both the sertraline and placebogroups. The probability of no improvement in the

combined sample ranged from 92% to 39%, according to

initial changes in anger score, with a 30% increase in

anger at one week providing a sensitivity of 26%, spec-

ificity of 93% and PPV of 87%. The practical implication

of our findings is that, for patients similar to the sample

studied here, an early increase in anger might be one

factor to consider when making a decision about med-ication continuation. Limitations of the study include

the relatively restricted nature of the study sample, as

compared to the general clinical population. For ex-

ample, males, non-Caucasians, those with primary de-

pression or substance/alcohol problems, and those

initiating cognitive therapy were excluded. Thus, our

findings might not apply to these and other PTSD

groups. The underlying significance of early treatment-induced anger remains unclear. We may perhaps spec-

ulate on the significance of the fact that rapid increase in

anger is associated with worse outcome. One possibility

is that it serves as a marker of more serious personality

disorder, which itself may be associated with more

chronic illness and worse outcome. It is also possible

that a certain percentage of the population can experi-

ence anger or hostility as a paradoxical effect of an-tidepressants (or even placebo), and that such dysphoric

response may increase the likelihood of a generally poor

outcome or early attrition from treatment.

Acknowledgements

Financial support from Pfizer was provided to theprincipal author during the course of the original clinical

trial.

References

Blake DD, Weathers FW, Nagy LM, Kaloupek DG, Gusman FD,

Charney DS, Keane FM. The development of a Clinician-

Administered PTSD scale. Journal of Traumatic Stress 1995;8:

75–90.

Brady KT, Pearstein T, Asnis G, Baker DG, Rothbaum BO, Sikes

CM, Farfel GM. Efficacy and safety of sertraline treatment of

posttraumatic stress disorder: a randomized-controlled trial. Jour-

nal of American Medical Association 2000;283:1837–44.

Cox DR, Snell EJ. The Analysis of Binary Data. Second ed. London:

Chapman and Hall; 1989.

Davidson JRT, Landerman LR, Farfel GM, Clary CM. Characteriz-

ing the effects of sertraline in posttraumatic stress disorder.

Psychological Medicine 2002;32:661–70.

Davidson JRT, Rothbaum BO, van der Kolk BA, Sikes CM, Farfel

GM. Multicenter, double-blind comparisons of sertraline and

placebo in the treatment of posttraumatic stress disorder. Archives

of General Psychiatry 2001;58:485–92.

Davidson JRT, Book SW, Colket JT, Tupler LA, Roth S, David D,

Hertzberg Mellman TA, Beckham JC, Smith RD, Davison RM,

Katz R, Feldman ME. Assessment of a new self-rating scale for

post-traumatic stress disorder. Psychological Medicine

1997;27:152–60.

Diagnostic and Statistical Manual of Mental Disorders, 3rd Edition,

Revised. Washington, DC: American Psychiatric Association;

1987.

Efron B, Gong G. A leisurely look at the bootstrap the Jackknife and

cross-validation. American Statistician 1983;37:36–48.

Hosmer Jr DW, Lemeshow S. Applied Logistic Regression. New

York: John Wiley and Sons; 1989.

Insel TR, Goodwin FK. Predictive value of a positive test. The

dexamethasone suppression test: promises and problems of diag-

nostic laboratory tests in psychiatry. Hospital and Community

Psychiatry 1983;34(12):1131–40.

Katz MM, Koslow SH, Maas JW, Frazer A, Bowden CL, Casper R,

Croughan J, Kocsis J, Redmond E. The timing, specificity and

clinical prediction of tricyclic drug effects in depression. Psycho-

logical Medicine 1987;17:297–309.

Pollack MH, Rapaport MH, Fayyad R, Otto MW, Nierenberg AA,

Clary CM. Early improvement predicts endpoint remission status

in sertraline and placebo treatments of panic disorder. Journal of

Psychiatric Research 2002;36:229–36.

SAS Technical Report P-200 SAS/STAT Software: CALIS and

LOGISTIC Procedures: Release 6.04. Cary NC: SAS Institute

Inc.; 1990.

Stassen HH, Delini-Stula A, Angst J. Time course of improvement

under antidepressant treatment: a survival-analytical approach.

European Neuropsychopharmacology 1993;3:127–35.