Development of the Paediatric Pain Profile: Role of Video Analysis and Saliva Cortisol in Validating a Tool to Assess Pain in Children with Severe Neurological Disability

276 Journal of Pain and Symptom Management Vol. 33 No. 3 March 2007

Original Article

Development of the Paediatric Pain Profile:Role of Video Analysis and Saliva Cortisolin Validating a Tool to Assess Pain in Childrenwith Severe Neurological DisabilityAnne Hunt, RSCN, PhD, Alison Wisbeach, Dip COT, Kate Seers, RGN, PhD,Ann Goldman, FRCPCH, Nicola Crichton, PhD, Leslie Perry, PhD, andKiki Mastroyannopoulou, Dip Clin PsycholDepartment of Nursing (A.H.), Faculty of Health, University of Central Lancashire, Preston; Great

Ormond Street Hospital for Children NHS Trust and Institute of Child Health (A.W., A.G.), London;

Royal College of Nursing Institute (K.S.), Oxford; Institute of Primary Care and Public Health (N.C.),

Southbank University, London; Department of Clinical Biochemistry (L.P.), Barts & The London

NHS Trust, London; and School of Medicine, Health Policy and Practice (K.M.), University of East

Anglia, Norwich, United Kingdom

AbstractThe Paediatric Pain Profile (PPP) is a 20-item behavior-rating scale designed to assess painin children with severe to profound neurological impairment. Three raters independentlyused the PPP to rate behavior of 29 children (mean age 9.6, SD 5.8) filmed during everydaymorning activities. The validation process included assessment of interrater reliability andexploration of the relationship of PPP scores with saliva cortisol concentration. There wassubstantial agreement between raters. The PPP showed strong association with global painassessments and differentiated between preselected high- and low-pain groups. PPP scoreshowed moderate correlation with saliva cortisol concentration, but a single child explainedthe strength of the relationship and overall, saliva cortisol concentrations appeared low. Thedata provide additional evidence that the PPP is a reliable and valid instrument for painassessment in neurologically impaired children. Cortisol levels are not a useful criterion forpain in this population and further study of cortisol response to stress/pain in children withsevere neurological impairments is needed. J Pain Symptom Manage 2007;33:276e289.� 2007 U.S. Cancer Pain Relief Committee. Published by Elsevier Inc. All rights reserved.

Key WordsChildren with disability, pain measurement, validation study, child behavior, hydrocortisone/saliva, videotape recording

The study was funded by Action Medical Researchwith full support from the National Lottery Commu-nity Fund.

Address reprint requests to: Anne Hunt, RSCN, PhD,Department of Nursing, Faculty of Health,

� 2007 U.S. Cancer Pain Relief CommitteePublished by Elsevier Inc. All rights reserved.

University of Central Lancashire, Preston PR12HE, United Kingdom. E-mail: [email protected]

Accepted for publication: August 18, 2006.

0885-3924/07/$esee front matterdoi:10.1016/j.jpainsymman.2006.08.011

mailto:[email protected]

Vol. 33 No. 3 March 2007 277Development of the Paediatric Pain Profile

IntroductionA large proportion of children with life-

limiting disease have severe to profound neu-rological disability. It is increasingly beingunderstood that such children are at risk ofpain from many sources.1e4 In the absence ofthe ability to verbally report their pain, chil-dren are dependent on their parents and pro-fessional caregivers accurately interpretingtheir behavioral pain cues. A number ofresearch groups, including our own, havebeen working recently to develop pain assess-ment tools for children with cognitiveimpairment.4e10

The present authors have developed thePaediatric Pain Profile (PPP). The PPP incor-porates a validated behavior-rating scale andis envisaged as a parent-held document, whichcan aid in documenting and measuring child-ren’s pain behavior and monitoring responseto treatments.4,6,9 Developers were particularlyinterested in developing a tool able to assesseveryday pains of children with severe and of-ten life-limiting conditions. Items for the scalewere originally elicited inductively through fo-cused interviews with parents (n¼ 20) and ex-perienced professional caregivers (n¼ 26),9

and a survey of parents whose children at-tended one of four health care settings(n¼ 121).6 Fifty-six pain cues were identifiedthat had been reported by more than 5% of in-dividuals in one or more of the above groups.Behavioral cues described included changes infacial expression, vocal sounds, posture andmovements, daily routines such as sleepingand eating, and mood. An analysis of the 56-item scale was performed through a series ofquestionnaires sent to a different group of par-ents.6 Parents (n¼ 45) used the scale to de-scribe their children ‘‘at best’’ or ‘‘on a goodday’’ (baseline ratings) and on up to two dayswhen the child was thought to have pain(pain ratings). The difference between thebaseline and pain ratings was significant(t¼ 8.07, P< 0.001 paired t-test). Four calcula-tions of Cronbach’s alpha were performed toassess internal consistency of the scale. Thesewere baseline (‘‘at best’’) ratings provided byeach of mother and father (or significantothers) and pain ratings from each parent. Al-pha scores ranged from 0.91 to 0.95. The highlevel of alpha suggested some redundancy

among the items. Nine items were identifiedthat had a low item-total (<0.20) correlation.These were considered for removal but ulti-mately each was retained, as they had been re-ported heavily by the original informants, butgrouped with similar items with which theyhad significant correlations. Three items wereeliminated as considered to be too nonspecificand the remaining items were grouped accord-ing to the correlations that occurred with sim-ilar items: the items ‘‘cheerful,’’ ‘‘sociable,’’and ‘‘responsive,’’ for instance, each showedsignificant (P< 0.01) correlations with theother and so were grouped to form the newitem ‘‘cheerful, sociable, or responsive.’’ Thescale was reduced from 56 to 20 items. These20 items formed the scale assessed in the pres-ent paper (see Table 3). More recently, a mini-mally adjusted 20-item scale was assessed inclinical settings.4

Pain is a feature of the daily lives of somechildren with severe neurological impair-ments.3,11 It is intended that the PPP shouldbe useable to document behavior associatedwith these recurring or routine pains and mon-itor the effectiveness of treatments. In this pa-per, we describe an evaluation of the PPP whenit was used to rate the behavior of childrenfilmed during their everyday morning activi-ties. A parent survey had suggested that ina subgroup of children (approximately 20%),pain could be severe and could occur daily.6

We considered it likely, therefore, that pain be-havior would be observed during the everydayactivities of some children. Validation woulddepend upon association between PPP scoresand parents’ and video raters’ global evalua-tions of pain. However, recognizing that toa large degree, parents and professionalsidentify and derive their global evaluation ofchildren’s pain through the children’s behav-ior,9,12,13 we sought an additional, more inde-pendent criterion through which the PPPmight be validated. We investigated the rela-tionship between PPP score and the child’s sa-liva cortisol concentration, hypothesizing thatsaliva cortisol concentration would positivelycorrelate with PPP score.

While not specific to pain, both plasma andsaliva cortisol have been shown to increase inthe presence of acute nociceptive stimulisuch as heelstick,14 inoculation15 and circum-cision.16,17 Although fewer studies have

278 Vol. 33 No. 3 March 2007Hunt et al.

investigated the cortisol response to nonacutepain, there also appears to be a relation-ship.18,19 However, it has been suggested that,in adults at least, the cortisol response mayhabituate as pain becomes more chronic.20

Recent work by Grunau et al.21 suggeststhat down regulation of the hypothalamic-pituitary-adrenal (HPA) axis can occur in pre-term neonates exposed to repeated proceduralpain events.

Saliva cortisol concentrations of healthy chil-dren are similar to those of adults.22e24 Al-Ansari et al. reported median unstimulatedmorning (09:00e10:30 AM) saliva cortisol con-centration of 105 healthy school childrenaged 4e11 years as 11 nmol/L, with 95% ofvalues lying within the range 4e25 nmol/L,22

while Tornhage24 reported median morning(08:00e09:00 AM) saliva cortisol concentra-tions of 386 children from 7 to 15 years as8.7 nmol/L (range, 1.0e53.9). A trend towardhigher cortisol with age was observed.24 Thediurnal rhythm associated with cortisol is re-ported to appear at around 3e6 months ofage.25 In comparing saliva cortisol in unim-paired and cognitively impaired children, Ter-stegen et al. found that the characteristicdiurnal pattern was not evident in a proportionof children from both groups. No significantdifference in saliva cortisol concentration wasfound between impaired and nonimpairedchildren.26

AimsThis study aimed to assess the validity and re-

liability of the 20-item PPP. To demonstrate in-terrater reliability, there should be substantialagreement between three observers indepen-dently using the PPP to rate the videotape.To demonstrate intrarater agreement, thereshould be substantial agreement betweenscores of a single rater coding at differenttimes. Evidence for the validity of the scalewould be provided by strong correlation be-tween video raters’ PPP scores and parents’and researchers’ global estimates of the child-ren’s pain (concurrent validity), significant dif-ference between PPP scores of children with‘‘high pain’’ and ‘‘low pain’’ (extreme groupvalidity), and positive correlation betweenPPP scores and cortisol concentration in salivasamples collected from children at time ofvideo recording (criterion validity).

MethodsPatient Sample

Children were recruited through four cen-ters in the UK; two hospices for children,a larger voluntary sector service providingshort- and long-term care to children, andthe neurodisability center of a tertiary referralhospital for children. Local research ethicscommittees responsible for each of these cen-ters gave ethical approval for the study. Parentsgave informed consent to inclusion of childrenin the study.

In an earlier phase of the development ofthe scale, 121 parents completed a question-naire about their perceptions of their child-ren’s pain.6 For the current study, two groupsof children were identified from these 121 re-plies, a high-pain group consisting of 40 chil-dren who were reported by parents to have‘‘moderate,’’ ‘‘severe,’’ or ‘‘very severe’’ pain,which occurred ‘‘all the time’’ or ‘‘some timeeach day,’’ and a low-pain group consisting of20 children reported to have ‘‘no pain’’ or‘‘mild’’ pain ‘‘some time each month’’ or‘‘less than once a month.’’ The total numberof questionnaires in which this criterion wasmet was small and limited the number withlow pain who could be invited to participate.

Information sheets describing the study indetail were sent to parents of these 60 children.A second letter was sent after two months if noreply had been received. Positive replies werereceived from the parents of 24/40 childrenin the high-pain group and 10/20 in the low-pain group. Three children died or were with-drawn due to illness prior to an initial visit tothe family. Two more children died betweenthe initial visit and the date arranged for film-ing. All were in the high-pain group. Ulti-mately, 29 children were filmed, 23 at home,two in long-term residential care and three attheir residential school. One child was admit-ted to hospital with a febrile convulsion and oti-tis media the night before filming was due,and, at the parents’ request, was filmedthere.

Videotaping, Editing, and AnalysisChildren were filmed while they followed, as

far as possible, their normal morning routine.Five activities were included: lying, beingdressed, transfer to chair, sitting in a chair, and


being fed. After each activity, pain assessmentswere recorded independently by the parent(s)(or if filmed in residential care, by the profes-sional caregiver) and the two investigators pres-ent, on a 0e10 numerical rating scale (NRS)and on a five-point verbal rating scale (VRS)(‘‘no pain’’ to ‘‘very severe pain’’). Approxi-mately 1½ hours of film were taken per child,with each visit lasting approximately 2½ hours.In addition to filming, assessments were madeof each child’s motor and functional ability us-ing the Disability Coding Scheme of the BritishAssociation for Community Child Health andDepartment of Health,27 the Chailey Levels ofAbility,28 and the Gross Motor Function Mea-sure (GMFM).29 These tools were chosen asthey allow for description and discriminationof physical abilities in children and young adultswith particularly low levels of physical ability.Children were also examined for the presenceof dislocated hip and spinal curvature. These as-sessments were carried out by the second au-thor, an occupational therapist experienced inthe care of children with neurodisability andtrained and certified in the use of the GMFM.

Edit-lists were produced from the videotapesmarking the beginning and end of each activ-ity. Discussion with clinical and research col-leagues suggested that pain was most likely tooccur at the margins of the activities, for in-stance, toward the end of feeding or as a childwas placed into the chair and, consequently,the last three minutes of each activity was iden-tified. Where this portion was inadequate forcoding, for instance, when the view of thechild was obscured, the nearest satisfactory ep-isode was identified. In three cases, one of thefive activities was not performed by the child(for instance, two children were tube-fed con-tinuously overnight rather than during theday). Ultimately, 142 three-minute episodesof film were available for analysis, including12e15 minutes per child and 71⁄4 hours offilm in total.

Three copies of the tapes were produced sothat they could be coded independently bythree raters to assess interrater reliability.One of the raters was the first author, a regis-tered nurse for sick children, who knew towhich of the pain groups children belonged.The second rater was an experienced regis-tered nurse for people/children with learningdisabilities who did not know to which of the

two pain groups children belonged. The thirdrater was an experienced registered nursefor sick children and was blind to the factthat there were two groups of children. Eachthree-minute episode of film was rated inde-pendently by each of the three raters usingthe 20-item PPP behavioral scale. Each itemin the scale was scored on a four-point scale(0e3) according to the extent of its occur-rence during any chosen time frame from‘‘not at all’’ to ‘‘a great deal.’’ The total scoreranged from 0 to 60. The video was played atnormal speed but raters were permitted toreplay the episode if they felt it was necessary.After viewing each episode, the raters assessedthe child’s pain during that episode on the nu-merical and verbal rating scales. The 20-itemscale was then completed. To allow for itemsthat could not be evaluated (for instance, ‘‘dif-ficult to feed’’ could not be rated when feedingwas not taking place), the total score was calcu-lated as a percentage of the maximum scorepossible, allowing for missing items (rawscore� 100)/(number of coded items� 3).To assess consistency of scoring over time,the first author recoded 30 episodes of film2e3 weeks after the initial coding.

Saliva Sampling and Cortisol AssayA complete set of saliva samples consisted of

1) demonstration sample (taken on the after-noon of the initial visit), 2) four reference sam-ples collected by parents on one day in theabsence of the investigators (two in the morn-ing as close to 8:30 AM and 9:30 AM as possibleand two in the afternoon as close to 2:30 PM

and 3:30 PM as possible), and 3) two samplescollected at the time of video recording (inthe morning). The samples collected on thereference day served as a control for the pres-ence of the investigators on the day of filmingand allowed investigation of the diurnal pat-tern of cortisol release.

Saliva samples were collected using a com-mercial saliva sampling kit, consisting of cot-ton dental roll and centrifugation tube(Salivette�, Sarstedt, Leicester, England). Nostimulant for salivation was used. To avoid con-tamination of the saliva sample with blood, thechild’s teeth were not brushed before collect-ing the samples, but the mouth was gentlycleaned with water and pink sponges thatwere supplied. The dental roll was placed in


the child’s mouth, left there for up to threeminutes, then removed with the help of a pieceof dental floss previously threaded through theroll. The roll was then replaced in the suppliedcentrifugation tube and stored in the family’sdomestic freezer until collection by theinvestigators on the day of filming. Times ofcollection were recorded, together with a de-scription of the child’s activities in the 20 min-utes prior to collection and an assessment ofthe child’s pain on the NRS. Samples werebatched and sent to the laboratory for assay.Samples were centrifuged, then analyzed by ra-dioimmunoassay using a modification of an in-house serum cortisol assay.22 The interassayimprecision was less than 10% in each of threeinternal quality control assessments. The assayis used regularly for analyzing clinical samples.

Statistical AnalysisTwo sets of analyses were performed, one us-

ing the video episode as the case (n¼ 142) andone using the mean of the child’s scores overthe five activities as the case (n¼ 29). A two-tailed significance level of <0.05 was necessaryto reject the null hypotheses. Parametric statis-tics were used to analyze normally distributeddata (age, weight, and PPP scores). Nonpara-metric statistics were used to analyze datathat were not normally distributed (GMFMand saliva cortisol concentration) or were cate-gorical or ordinal.

To establish reliability of the scale, inter- andintrarater reliability was assessed using intra-class correlation coefficient (ICC), model 3.30

This is a two-way mixed-effect model. In thismodel, raters, who have not been selected ran-domly, each rate all the subjects and aretreated as a fixed effect rather than a randomeffect. A repeated measures analysis of vari-ance examined the proportion of the varianceexplained by the children, by the activity inwhich the child was engaged, and by the raters.Cronbach’s alpha, a statistic that describes theaverage correlation of each of the items in thescale with each of the others, was used to assessinternal consistency of the scale.

Several tests were made to assess validity ofthe scale. The PPP scores of each video raterwere examined for their correlation with theratings on the NRS of (a) the raters themselvesand (b) parents and investigators at the timeof the video recording, using Pearson’s

correlation coefficient. The PPP scores of eachrater were also examined for their associationwith their ratings on the VRS, using analysis ofvariance for linear trend. Independent-samplet-tests were used to test the hypothesis that chil-dren in the high-pain group would have higherscores than the low-pain group on the 20-itemscale. Spearman’s rank correlation coefficientswere used to explore the association betweenthe average PPP score across the five activitiesand saliva cortisol concentrations on the morn-ing of filming. In addition, Mann-Whitney testswere used to examine differences in saliva corti-sol concentration between high- and low-paingroups. Power for the study was calculatedbased on studies by Al-Ansari et al. (St. Bartholo-mew’s Endocrinology Lab data),22 wheremedian saliva cortisol concentration was11 nmol/L (range, 4e25) in a group of 105children without pain, and by Price et al., wherethe mean saliva cortisol concentration was11.0 nmol/L, with a standard deviation (SD)of 6.0 nmol/L (range, 3.3e26.6), in childrenwithout pain.25 The SD in a group with pain islikely to be higher. With SD¼ 8 in the paingroup, a significance of 0.05 (two-sided) andpower of 0.85, a difference in the mean of8 nmol would be detected with two samplegroups of 15.

ResultsCharacteristics of Study Population

Characteristics of the population are sum-marized by pain group in Table 1. Childrenhad both static and progressive encephalopa-thy, including cerebral palsy, chromosomaland congenital disorders, Sanfilippo syn-drome, lissencephaly, Rett syndrome, and un-diagnosed degenerative conditions. Therewere no differences between pain groups inthe proportion of children who had beenborn prematurely or who had been deliveredby caesarean vs. normal delivery. A history ofpain problems was significantly more repre-sented (100%) in the high-pain group (Fish-er’s Exact Test, P¼ 0.015), though it was ofnote that 60% of children in the low-paingroup were also reported to have a significantpain history. A greater proportion of childrenin the high-pain group had had significantsurgical interventions.


Table 1Population Characteristics for All Children and by Pain Group

Pain Groups All High Pain Low Pain Pa

N 29 19 10Age (years) (mean� SD) 9.7� 5.0 9.9� 4.5 9.6� 5.8 0.859Gross motor function measure (% (median (quartiles)) 5.0 (0, 22.5) 2.0 (0, 17) 9.0 (6.5, 44.5) 0.056Sex (M/F) 16:13 11:8 5:5 0.714Method of delivery (normal:caesarean) 19:6 12:3 7:3 0.653Born prematurely (Y:N) 6:19 4:11 2:8 1.0Birth injury (Y:N) 9:17 6:10 3:7 1.0History of hospitalizations (Y:N) 14:12 11:5 3:7 0.105Pain history (Y:N) 22:4 16:0 6:4 0.014History of significant surgery (Y:N) 18:8 15:3 1:7 0.001Dislocated hip (Y:N) 12:17 10:9 2:8 0.126Spinal curvature (Y:N) 19:10 16:3 3:7 0.011

aStatistics calculated for difference between groups are the independent-samples t-test (age), Mann-Whitney test (GMFM), and Fisher’s Exact test.

Using the BACCH/DOH disability codingscheme, from 27 to 29 children (93%e100%)were profoundly disabled in terms of conti-nence, personal care, learning ability, and com-munication ability. The remaining childrenwere severely disabled. Using the same codingscheme, 24 (83%) had severe or profound dis-ability in hand function, 13 (45%) in vision,and four (14%) in hearing. Using the ChaileyLevels of Ability scheme, 20 (69%) were unableto raise themselves from a lying into a sitting po-sition or remain in a sitting position withoutsupport and 15 (52%) were unable to rollfrom supine to prone or vice versa. GMFMdata had a skewed distribution. The medianGMFM score was 5% (quartiles 0, 22.5) of nor-mal. There was a nonsignificant tendency forchildren in the high-pain group to have a lowerGMFM score than the low-pain group, suggest-ing that children in the high-pain group weremore impaired. Though dislocated hip wasequally distributed between the groups, spinalcurvature was significantly more representedin the high-pain group (Fisher’s Exact Test,P¼ 0.011) (Table 1).

Children were receiving an average of three(range, 0e9) different medications per day;18 received from one to four different anticon-vulsants, 13 received from one to three antire-flux medications, nine received a musclerelaxant (diazepam and/or baclofen or dantro-lene), and six received laxatives daily. Five chil-dren regularly received a conventionalanalgesic including strong opioid (n¼ 1),weak opioid (n¼ 1), nonsteroidal anti-inflam-matory drugs (n¼ 3), and acetaminophen(paracetamol) (n¼ 4). Four children were re-ceiving nebulized medication for respiratorydisease, including three who received nebu-lized budesonide. No child was receiving anoral corticosteroid.

PPP ScoresThe average PPP percent scores across the

four to five activities for the whole group andby pain group are presented in Table 2.Fig. 1 provides boxplots of PPP scores for thedifferent activities by pain group. The meanPPP percent scores across the four to five activ-ities of each child were normally distributed in

Table 2PPP Percent Score of Each of Three Raters for All Children and by Pain Group

Ratersa

PPP Score Mean (SD) (95% CI)

t-Valueb P (df¼ 27)All

High Pain Low Pain

n¼ 19 n¼ 10

1 12.7 (6.6) (10.1, 15.2) 14.7 (6.6) (11.5, 17.8) 8.9 (5.1) (5.2, 12.5) 2.42 0.0232 12.8 (5.5) (10.7, 15.0) 14.5 (5.7) (11.7, 17.3) 9.7 (3.7) (7.0, 12.3) 2.42 0.0233 13.5 (8.3) (10.4, 16.7) 16.3 (8.6) (12.2, 20.5) 8.2 (4.4) (5.0, 11.3) 2.81 0.009

CI¼ Confidence interval.aMean score across five morning activities.bDifferences between high- and low-pain groups (independent-sample t-test).


Fig. 1. Boxplot of average PPP score of three raters rating 142 video episodes of morning activities of 29 childrenwith severe neurological disability by pain group.

the case of each rater (Kolmogorov-SmirnovGoodness of Fit Test).

Reliability of the PPPA repeated measures analysis of variance was

performed to investigate the proportion of var-iance in the PPP scores attributable to subjects(n¼ 29), raters (n¼ 3) and the activity thatwas being filmed (lifting, lying, and so on)(n¼ 5). The analysis demonstrated a significanteffect of subject (F¼ 561.4, df¼ 28, P< 0.001)and a smaller but significant effect of the activityin which the child was engaged (F¼ 2.93,df¼ 4, P¼ 0.021). No effect was observed forthe rater (F¼ 0.24, df¼ 2, P¼ 0.789), support-ing the reliability of the scale.

ICC for the total PPP score of the threeraters was 0.62 (95% CI: 0.54e0.70) for a singlerater and 0.83 (95% CI: 0.78e0.88) for the av-erage of three raters. Agreement on individualitems was variable within a range of 0.21e0.71,with an average of 0.45. Intrarater agreement(ICC) was 0.90 (95% CI: 0.80e0.95) for the to-tal PPP score and ranged from 0.21 to 1.0 forindividual items, with an average of 0.72. Inter-and intrarater reliability quotients are listedagainst items in Table 3.

Because there was no significant differencebetween the raters, the mean score on each

item across the three raters was calculatedand Cronbach’s alpha used to assess internalconsistency of the scale. Four itemsd‘‘Had dis-turbed sleep,’’ ‘‘Was reluctant to eat/difficultto feed,’’ ‘‘Appeared hot and sweaty,’’ and‘‘Was hard to comfort or console’’dwere ex-cluded because they could only be assessedin a very limited number of episodes or hadno variance. Alpha for the 16 remaining itemswas 0.88. Correlation between items and totalscore are listed in Table 3.

Validity of the PPPdConcurrent ValidityCorrelation between raters’ PPP percent

scores and their NRS for each of the episodesvaried from 0.76 to 0.90 (Pearson’s r, P<0.001). Correlation of PPP percent scoresacross raters was rather less, varying from0.58 to 0.78 (Pearson’s r, P< 0.001). Correla-tion of the average PPP percent score givenby each video rater with the numerical painratings given by the parent and investigatorsat home when the child was being filmed wasalso moderate, varying from 0.33 to 0.61(P¼ 0.002 to P< 0.001, respectively). Fig. 2shows the average PPP score (raw score) ofthe three raters plotted against their averagescore on the NRS and allows examination ofrelationship between the modes of scoring.


Table 3Internal and External Reliability

ItemItem-Total

CorrelationaInterrater

Agreement (ICC)Intrarater

Agreement (ICC)

1. Cheerful/sociable/responsive (reversed) 0.606 0.461 0.8432. Appeared withdrawn or depressed 0.804 0.196 0.5253. Was hard to console or comfort d 0.652 d4. Cried/moaned/groaned/screamed or whimpered 0.689 0.709 0.8555. Tended to touch or rub certain areas 0.363 0.407 0.8206. Was restless/agitated or distressed 0.769 0.553 0.4867. Pulled away or flinched when touched 0.708 0.441 0.7588. Had involuntary movements/was jumpy/startled or had seizures 0.166 0.434 0.8009. Bit self or banged head 0.509 0.508 d

10. Flexed inwards or drew legs up towards chest 0.369 0.483 1.00011. Tensed/stiffened or spasmed 0.569 0.489 0.73012. Twisted and turned/tossed head/writhed or arched back 0.609 0.233 0.72113. Resisted or was fearful of being moved 0.717 0.213 0.21114. Was reluctant to eat/difficult to feed d 0.674 d15. Had disturbed sleep d d d16. Grimaced/screwed up face/screwed up eyes 0.810 0.639 0.79517. Frowned/had furrowed brow/looked worried 0.737 0.271 0.50918. Looked frightened (with eyes wide open) 0.321 0.252 1.00019. Ground teeth or made mouthing movements 0.392 0.528 0.77220. Appeared hot and sweaty d d d

Correlation of each item with the total when the item is omitted (item-total correlation) and inter- and intrarater agreement (ICC).ICC calculated using model 3, mixed-effect model (see Ref. 30).aThe items ‘‘Was hard to comfort or console,’’ ‘‘Had disturbed sleep,’’ ‘‘Was reluctant to eat/difficult to feed,’’ and ‘‘Appeared hot and sweaty’’were omitted as they could only be assessed in a very limited proportion of episodes.

There is excellent correlation (Pearson’sr¼ 0.91) between PPP and NRS scores. In ad-dition, analysis of variance demonstrated a lin-ear relationship between each rater’s PPPscore and their global evaluation of pain onthe VRS (P< 0.001). Using Bonferroni posthoc tests, PPP percent scores for video epi-sodes rated as showing ‘‘mild pain’’ were signif-icantly higher than for ‘‘no pain’’ and PPPpercent scores for video episodes rated asshowing ‘‘moderate pain’’ were significantlyhigher than for ‘‘mild pain.’’ Two of the ratershad coded only two episodes as showing ‘‘se-vere pain,’’ and in these cases the PPP percentscores were not significantly different fromthose showing ‘‘moderate pain.’’ The PPP per-cent scores of the third rater did show signifi-cant difference in PPP percent scoresbetween episodes graded ‘‘severe pain’’ and‘‘moderate pain.’’

Extreme Group ValidityTable 2 gives the average PPP percent score

across the four to five activities for the wholegroup and by pain group. In the case of eachrater, PPP percent scores were significantlyhigher in the high pain than the low-pain group

(independent-sample t-test, P¼ 0.023e0.009),demonstrating extreme group validity of thePPP.

Criterion ValiditydAssociation Between PPPScores and Saliva Cortisol Concentration

The dental swabs used to collect saliva sam-ples were well tolerated by most of the children

Fig. 2. Plot of mean PPP score against mean nu-merical pain scale scores of three raters rating 142episodes of film from 29 children with severe neuro-logical disability. (Pearson’s r¼ 0.91).


and in most cases it was possible to collect suf-ficient saliva (1e2 ml) within three minutes.However, occasionally children did not chewon the dental roll but allowed it to just sit intheir mouths, in which case the roll absorbedvery little saliva. Across sampling times, sam-ples could not be obtained from four to ninechildren, including four on the morning offilming. In one child with very hypertrophiedgums, parents reported that on the referenceday the child’s gums had bled when chewingthe cotton roll. High concentrations of cortisol(19.2 nmol/L and 33.6 nmol/L) suggestingpossible contamination with blood were pro-duced on assay and those results were dis-carded. During the video session, care wastaken to remove the swab from the child’smouth before any bleeding occurred.

The saliva cortisol concentration data forthe whole group and by pain group is summa-rized in Table 4. In seven (8%) samples, salivaconcentration was below the minimal detect-able concentration and where this was thecase, the minimal detectable concentration of0.5 nmol/L was used in the analysis. Data foreach collection were skewed and nonparamet-ric tests have been used to test for statistical sig-nificance. There was no significant differencebetween saliva cortisol concentration in sam-ples collected on the mornings of filmingand reference day, or between samples col-lected on the afternoons of the demonstrationvisit and reference day. Stability of saliva corti-sol concentration for individual childrenacross different days was low to moderate, atr¼ 0.431 for morning samples and r¼ 0.212for afternoon samples. Cortisol release will nat-urally vary with daily events and concentration

data were, therefore, aggregated across days, assuggested by Gunnar and Vazquez,31 resultingin a median of 4.7 nmol/L (quartiles 2.9, 8.6)for morning samples and 2.2 nmol/L (quar-tiles 1.7, 3.6) for afternoon samples. Medianratio of morning to afternoon samples was2.38 (range, 0.44e10.05). Children in thehigh-pain group had significantly higher corti-sol concentration on the morning of filmingand in the morning and afternoon when con-centrations were aggregated across days.

Fig. 3 presents boxplots of cortisol concentra-tions in saliva samples collected on the refer-ence day. Concentrations are plotted at times8:30 AM, 9:30 AM, 2:30 PM, and 3:30 PM, thoughthere was some variation around these times.The plots show a decrease in cortisol concentra-tion over time in line with diurnal variation, withmost of the data falling within fairly narrowbands. There were occasional outliers and ex-treme values. The two children with extremevalues in the afternoon were both reported bytheir parents/carers to have pain rated on theNRS as 6 at these times. Fig. 4 is a boxplot of sa-liva cortisol concentrations on the morning offilming by pain group showing that cortisol con-centrations in the low-pain group are withina narrow band, whereas there is greater varia-tion in the high-pain group. The outlier onthe graph is the child with otitis media filmedin hospital, having been admitted the day be-fore following a febrile convulsion (this childdid not normally have seizures). The motherof this child rated her pain on the NRS as 3and VRS as 2 (moderate pain). The mean PPPpercent score of video raters for this child was21.0, just below the optimum cut-off point formoderate to severe pain.

Table 4Saliva Cortisol Concentration (nmol/L) in Children with Severe to Profound Neurological Impairments

Day and Time

N All High Pain Low Pain

PHigh:Low Median (quartiles) Median (quartiles) Median (quartiles)

MorningReference day 14:8 5.1 (2.6, 9.7) 5.8 (2.8, 12.0) 3.1 (2.4, 8.4) 0.267Day of filming 15:10 5.6 (2.6, 10.2) 7.9 (4.3, 14.4) 4.6 (1.3, 5.6) 0.008Aggregated days 11:8 4.7 (2.9, 8.6) 8.1 (4.6, 10.7) 3.7 (2.3, 6.7) 0.037

AfternoonReference day 13:7 1.9 (1.4, 4.5) 2.3 (1.6, 4.9) 1.6 (0.5, 3.4) 0.183Demonstration 17:7 2.8 (1.3, 4.5) 3.00 (1.8, 5.3) 1.7 (0.5, 3.2) 0.166Aggregated days 11:4 2.2 (1.7, 3.6) 2.6 (1.9, 3.7) 1.4 (1.2, 2.1) 0.048

Differences between high- and low-pain groups (Mann-Whitney test).


Some children who appeared to have paindid not have raised cortisol, in particular an8-year-old boy with cerebral palsy whose long-standing pain from a dislocated hip was poorlyrelieved despite receiving sustained-releasemorphine (MST) 300 mg three times a day.His saliva cortisol concentration on the after-noon of the demonstration day was3.6 nmol/L despite a numerical pain ratingof 7. Saliva cortisol concentrations were rela-tively low (average 2.6 nmol/L) on the morn-ing of filming despite an average NRS scoreof 6.3 and PPP score of 33 during one of theactivities (being lifted).

Depending on video rater, correlation be-tween PPP score and saliva cortisol concentra-tion was from rs¼ 0.378e0.451. The childadmitted to hospital and found to have an oti-tis media had a large influence on the correla-tion. With exclusion of this child from theanalysis, correlations ranged from r¼ 0.158 tor¼ 0.297 and were not significant. We had an-ticipated that children reported to have severepain daily might have pain at the time of ourvisit to film. From the films, we identified 17 in-stances in 13 children (all in the high-paingroup) in which the average PPP percent scoreof the raters was over 24. While there was a sig-nificant difference in cortisol concentrationbetween the a priori identified pain groups,the difference between children with PPP per-cent scores above 24 (median concentration

Fig. 3. Boxplot of saliva cortisol concentrationfrom 13 children who each provided four saliva sam-ples on the reference day.

7.6 nmol/L, range, 1.0e30.7 nmol/L) andthose below 24 (median concentration5.2 nmol/L, range, 0.5e15.0 nmol/L) wasnot significant (P¼ 0.298, Mann-Whitney test).

Sensitivity and Specificity of the PPPA positive pain state during a filmed activity

was assumed when the average verbal pain rat-ing of the three video raters for an episode was2 or more, that is pain of moderate or greaterintensity. Of 142 episodes of film, 10 episodes(in seven children) were rated as indicatingmoderate or worse pain. All these childrenwere in the high-pain group. The PPP scorefor each of these episodes was at 24.3% orabove. At a cut-off point of 24% (raw score14.4), sensitivity and specificity of the PPPscale for moderate or worse pain were 1.0and 0.95, respectively. Thirty-nine episodes(in 15 children) were rated on the VRS as indi-cating that the child had mild or worse pain.PPP scores in this case were 12.8% (raw score7.5) or above. Sensitivity was 1.0 and specificity0.88 for mild or worse pain at a cut-off point of12.5%.

DiscussionThe data presented suggest that the PPP has

both concurrent and extreme groups validity.To summarize the findings, the PPP

Fig. 4. Boxplot of saliva cortisol concentrations ofchildren in low- and high-pain groups who providedsamples on the day of filming. Cortisol was signifi-cantly higher in the high-pain group (P¼ 0.008,Mann-Whitney test).


differentiated between predetermined high-and low-pain groups and was shown to correlatevery well with the numerical pain ratings madeby the video raters and moderately well with nu-merical ratings made by investigators and par-ents at the time of video recording. Inaddition, significant differences were seen inPPP scores with different levels of pain on theverbal pain rating scale. The PPP was also shownto have high sensitivity and specificity when us-ing the average global pain ratings of videoraters as indicators for moderate or worsepain. The cut-off point of 24% (raw score14.4) on the PPP was in line with previously pub-lished findings.4 Although interrater reliabilityfor the summated scale score was adequate(ICC 0.62), intrarater reliability was better(ICC 0.90), suggesting that in routine practice,assessments might be more consistent whenperformed by the same person. As aparent-held document, this would, for themost part, be the child’s parent(s). The avail-ability of a behavior-rating scale will make it eas-ier to track back differences in scoring betweenindividuals when they do occur. Internal consis-tency of the scale was good (alpha 0.88), con-firming that it would be appropriate to use thePPP as a single-dimensional scale.

In the absence of a ‘‘gold standard’’ for painin this group, the standard used in assessingconcurrent validity and estimating sensitivityand specificity of the tool were observers’(proxy) global rating of pain on an NRS orVRS. Alongside context, it is the child’s behav-ior that leads adults to presume that the childhas pain and its intensity9 and it is not surpris-ing, therefore, that there will be a good relation-ship between them. However, proxy ratingshave not always been found to be representativeof the child’s32 or adult’s pain experience.33

While observed behaviors and self-reports ofpain intensity are related, and both are valu-able, they may not necessarily provide thesame information.32,34

In seeking criteria that might provide a dif-ferent and more independent standard, we ex-plored the use of saliva cortisol concentrations.Despite 13 children (all in the high-paingroup) having behavior strongly suggestive ofpain on the morning of filming (PPP percentscore above 24), the relationship between thePPP score (representing pain behavior) andsaliva cortisol concentration was weak and

largely explained by a single child who hadpain from diagnosed otitis media on the dayof filming. The ability to demonstrate a strongassociation may have been limited by the tim-ing of saliva samples. With the exception ofthe child with otitis media, painful eventstended to appear relatively brief, for example,occurring when the child was dressed or lifted,with behavior returning to normal when thechild was settled. Samples were not necessarilycollected immediately following the occur-rence of a painful event.

Overall, saliva cortisol concentrationstended to be lower than those published fornonimpaired children. For example, medianmorning saliva cortisol concentrations ofhealthy school children were reported as11 nmol/L and 8.7 nmol/L in 105 children22

and 386 children,24 respectively. Terstegenet al.26 are the only other group to have docu-mented saliva cortisol in children with cogni-tive impairments. They found no significantdifference between children with and withoutimpairments but found that a significant pro-portion in both groups did not show the ex-pected diurnal rhythm.26 In the currentstudy, in all but three of the 20 children whocontributed both morning and afternoon sam-ples, concentration of saliva cortisol in morn-ing samples was higher than afternoon. Intwo of the three cases where afternoon concen-trations were higher than morning, parentsreported behavior suggestive of pain. In thethird case, the morning cortisol concentrationwas relatively low rather than the afternoonconcentration being exaggerated. Terstegenet al.26 do not comment on the behavior ofchildren around the time of saliva collection,so it is not known whether any of them mighthave had pain around the time of sampling.

There are several possible reasons why salivacortisol concentration might be low in thisgroup of neurologically impaired children.Children were receiving multiple medications.Respiratory disease is common in childrenwith severe neurological disability and whilechildren receiving oral glucocorticosteroidswere not included in the study, three childrenwere receiving nebulized budesonide, and it ispossible that others had in the past receivedoral or inhaled steroids. The evidence is con-flicting as to whether inhaled steroids affectadrenal function.35,36 A child whose incident


pain was not well relieved by significant dosesof morphine appeared to have notably low cor-tisol concentrations. Opioids have been shownto lower the cortisol response, an effect thatmay be independent of pain relief, as it is ob-served both in individuals with pain37 and inthose without pain but addicted to heroin.38

Over 60% of children in this study received an-ticonvulsants and it is suggested that these,too, may interact with the HPA axis effect.39

Some studies have identified such an effect,40

while in others none is reported.41 Tornhage24

found that as well as age, the stage of pubertywas a factor in the cortisol concentration.Many children with severe neurological im-pairments do not have pubertal changes inline with their age, and stage of puberty couldhave been a factor in the low cortisol concen-tration of children in this study; we did not re-cord pubertal development of children in thisstudy. Other factors that could explain therather low saliva cortisol concentrations of chil-dren in this study are early exposure to painfulevents,42 the prolonged pain history of somechildren, or the central nervous system diseaseitself. Whatever the cause, a dampening of thecortisol response may have implications forchildren who undergo surgery or who sufferother stressful events. Further study of thestress/pain response of children with signifi-cant neurological impairments is warranted.Comparison with other groups of childrenwith chronic illness and/or chronic pain prob-lems may be helpful.

Further research with the PPP is warranted.Though there is satisfactory reliability for the to-tal PPP score, agreement on individual items isvariable. Further research might investigatethe way nurses and other clinicians make theirdecisions on scoring individual items. Furtherwork might also investigate the appropriatenessof the PPP for adults with severe physical andlearning disabilities. Current work is assessingthe acceptability, feasibility, and usefulness ofthe tool when implemented in community set-tings as a parent-held document.

ConclusionThis study contributed to the development

and testing of the PPP and provides further ev-idence of the tool’s reliability and validity and

robust psychometric properties. Pain can bea significant problem for children with severeneurological impairments and the availabilityof a reliable and valid pain assessment toolshould make it easier to document pain behav-ior, communicate between parents and profes-sionals about a child’s pain problem, monitorthe effects of treatment, and provide moreeffective pain relief for children. Data arepresented that suggest that saliva cortisol con-centration is generally low and may not bea good indicator of pain in this group of chil-dren. Down regulation may occur for a numberof reasons and further investigation iswarranted.

AcknowledgmentsThe authors would like to thank Vivienne

Moffat and Joan Anderson who independentlyrated the video-films and all the children andparents who so generously participated inthis study.

References1. Hunt A, Burne R. Medical and nursing prob-

lems of children with neurodegenerative disease.Palliat Med 1995;9:19e26.

2. Oberlander TF, O’Donnell ME, Montgomery CJ.Pain in children with significant neurological im-pairment. J Dev Behav Pediatr 1999;20(4):235e243.

3. Houlihan CM, O’Donnell M, Conaway M,Stevenson RD. Bodily pain and health-related qual-ity of life in children with cerebral palsy. Dev MedChild Neurol 2004;46(5):305e310.

4. Hunt A, Goldman A, Seers K, et al. Clinical val-idation of the paediatric pain profile. Dev MedChild Neurol 2004;46(1):9e18.

5. Collignon P, Giusiano B. Validation of a painevaluation scale for patients with severe cerebralpalsy. Eur J Pain 2001;5(4):433e442.

6. Hunt A. Towards an understanding of pain inthe child with severe neurological impairment. De-velopment of a behaviour rating scale for assessingpain. Unpublished PhD thesis. Manchester: RoyalCollege of Nursing Institute, University of Manches-ter, 2001.

7. Breau LM, McGrath PJ, Camfield CS, Finley GA.Psychometric properties of the non-communicatingchildren’s pain checklist-revised. Pain 2002;99(1-2):349e357.

8. Stallard P, Williams L, Velleman R, et al. The de-velopment and evaluation of the pain indicator for


communicatively impaired children (PICIC). Pain2002;98(1-2):145e149.

9. Hunt A, Mastroyannopoulou K, Goldman A,Seers K. Not knowing-the problem of pain in chil-dren with severe neurological impairment. Int JNurs Stud 2003;40(2):171e183.

10. Terstegen C, Koot HM, de Boer JB, Tibboel D.Measuring pain in children with cognitive impair-ment: pain response to surgical procedures. Pain2003;103(1-2):187e198.

11. Stallard P, Williams L, Lenton S, Velleman R.Pain in cognitively impaired, non-communicatingchildren. Arch Dis Child 2001;85(6):460e462.

12. Carter B, McArthur E, Cunliffe M. Dealing withuncertainty: parental assessment of pain in theirchildren with profound special needs. J Adv Nurs2002;38(5):449e457.

13. Hadden K, von Baeyer C. Pain in children withcerebral palsy: common triggers and expressivebehaviors. Pain 2002;99(1-2):281e288.

14. Gunnar MR, Porter FL, Wolf CM, Rigatuso J,Larson MC. Neonatal stress reactivity: predictionsto later emotional temperament. Child Dev 1995;66(1):1e13.

15. Lewis M, Thomas D. Cortisol release in infantsin response to inoculation. Child Dev 1990;61(1):50e59.

16. Gunnar MR, Malone S, Vance G, Fisch RO. Cop-ing with aversive stimulation in the neonatal period:quiet sleep and plasma cortisol levels during recov-ery from circumcision. Child Dev 1985;56(4):824e834.

17. Kurtis PS, DeSilva HN, Bernstein BA, Malakh L,Schechter NL. A comparison of the Mogen andGomco clamps in combination with dorsal penilenerve block in minimizing the pain of neonatalcircumcision. Pediatrics 1999;103(2):E23.

18. Kidd S, Lone N, Midgley P, et al. Salivary hor-mones as markers of pain in children. 5th Inter-national Symposium on Paediatric Pain; 2000;London; 2000 (Abstract).

19. Kidd S, Reiss J, McIntosh N, Stephens R, SmithJ. Measuring subacute pain in the newborn. 5th In-ternational Symposium on Paediatric Pain; 2000;London; 2000 (Abstract).

20. Moore RA, Evans PJ, Smith RF, Lloyd JW. In-creased cortisol excretion in chronic pain. Anaes-thesia 1983;38(8):788e791.

21. Grunau RE, Holsti L, Haley DW, et al. Neonatalprocedural pain exposure predicts lower cortisoland behavioral reactivity in preterm infants in theNICU. Pain 2005;113(3):293e300.

22. Al-Ansari AAK, Perry LA, Smith DS, Landon J.Salivary cortisol determination: adaptation of a com-mercial serum cortisol kit. Ann Clin Biochem 1982;19:163e166.

23. Kiess W, Meidert A, Dressendorfer RA, et al.Salivary cortisol levels throughout childhood andadolescence: relation with age, pubertal stage, andweight. Pediatr Res 1995;37(4 Pt 1):502e506.

24. Tornhage CJ. Reference values for morning sal-ivary cortisol concentrations in healthy school-agedchildren. J Pediatr Endocrinol Metab 2002;15(2):197e204.

25. Price DA, Close GC, Fielding BA. Age of appear-ance of circadian rhythm in salivary cortisol valuesin infancy. Arch Dis Child 1983;58:454e456.

26. Terstegen C, Koot HM, Koudijs SM, et al. Sali-vary cortisol in children with cognitive impairment.Dev Med Child Neurol 2003;45(2):139e140.

27. British Association for Community ChildHealth and Department of Health. Disability inchildhood. Towards nationally useful definitions.Report of the Working Group on Definitions of Dis-ability in Childhood. London: British Associationfor Community Child Health and Department ofHealth, 1994.

28. Pountney TE, Cheek L, Green E, Mulcahy C,Nelham R. Content and criterion validation of theChailey Levels of Ability. Physiotherapy (London)1999;85(8):410e416.

29. Russell DJ, Rosenbaum PL, Gowland C, et al.Manual for the gross motor function measure.Hamilton: McMaster University, 1993.

30. Shrout PE, Fleiss J. Intraclass correlations: usesin assessing rater reliability. Psychol Bull 1979;86:420e428.

31. Gunnar MR, Vazquez DM. Low cortisol anda flattening of expected daytime rhythm: potentialindices of risk in human development. Dev Psycho-pathol 2001;13(3):515e538.

32. Manne SL, Jacobsen PB, Redd WH. Assessmentof acute pediatric pain: do child self-report, parentratings, and nurse ratings. Pain 1992;48(1):45e52.

33. Jensen MP. The validity and reliability of painmeasures in adults with cancer. J Pain 2003;4(1):2e21.

34. Labus JS, Keefe FJ, Jensen MP. Self-reports ofpain intensity and direct observations of pain behav-ior: when are they correlated? Pain 2003;102(1-2):109e124.

35. Shapiro J, Perez M, Warden MJ. The impor-tance of family functioning to caregiver adaptationin mothers of child cancer patients: testing a socialecological model. J Pediatr Oncol Nurs 1998;15(1):47e54.

36. Kannisto S, Korppi M, Remes K, Voutilainen R.Adrenal suppression, evaluated by a low dose adre-nocorticotropin test, and growth in asthmatic chil-dren treated with inhaled steroids. J ClinEndocrinol Metab 2000;85(2):652e657.

37. Palm S, Moenig H, Maier C. Effects of oral treat-ment with sustained release morphine tablets on


hypothalamic-pituitary-adrenal axis. Methods FindExp Clin Pharmacol 1997;19(4):269e273.

38. Cami J, Gilabert M, San L, de la Torre R. Hyper-cortisolism after opioid discontinuation in rapid de-toxification of heroin addicts. Br J Addict 1992;87(8):1145e1151.

39. Putignano P, Kaltsas GA, Satta MA,Grossman AB. The effects of anti-convulsant drugson adrenal function. Horm Metab Res 1998;30(6-7):389e397.

40. Arvat E, Giordano R, Grottoli S, Ghigo E. Ben-zodiazepines and anterior pituitary function. J En-docrinol Invest 2002;25(8):735e747.

41. Evans PJ, Walker RF, Peters JR, et al. Anticonvul-sant therapy and cortisol elimination. Br J Clin Phar-macol 1985;20(2):129e132.

42. Porter FL, Grunau RE, Anand KJ. Long-term ef-fects of pain in infants. J Dev Behav Pediatr 1999;20(4):253e261.

Documents

Development of the Paediatric Pain Profile: Role of Video Analysis and Saliva Cortisol in Validating a Tool to Assess Pain in Children with Severe Neurological Disability