Disc Self Appraisal Inventories

7/30/2019 Disc Self Appraisal Inventories

1/4


2/4

June 1986Volume 2(3)

SELF-APPRAISAL PERSONALITY INVENTORI ES continued

respondent selects 'attractive' as the mostdescriptive this will count as + 1 towards 'Influence'and if least descripti ve - 1. Some items have only a+1 or -1 weighting and In the case of the PPA oneIte m 'Trusting' contributes to two categories,positively on Steadiness and negat ively on InfluenceIn the PAL the former weighting has been removed.In the PPA one item 'Eager' does not contribute toany category whereas In the PAL it IS a 'Dominance'item This unusual scoring procedure hascons iderable bear ing on the structure of the test,w hich will be discussed later.

The scoring mechanism on the PPA is based oncolour coding the items on the reverse page as eitherpurple, red, black or green with combinat ions of fullyenclosed box (for + 1 and -1). line above (+ 1) andline below (-1) Th is results in considerableconfusion and possible scor ing errors. The author ofthe PAL, who, as a previous dist ributor of PP Aexperienced this problem, revised the scoring byhaving a single carbonized sheet for each category.Reponses are transposed by carbon on to theappropriate sheet below t he question page. Thisworks we l l for the f irst two scoring sheets, butunless the respondent has pressed very firmlyindeed, his/her responses are scarcely visible on thethi rd and fo urth. We we re told that this had beengreatly improved In more recent cop ies of the test.

Validity an d reliabilityA question we have to ask about a part icular test ISwhether it is a useful measure of the psychologicalcha racteristic it is attempt ing to assess? Thisquest ion ISusually asked in two parts Firstly, IS thetest reliable and seco ndly, IS the test val id. Reliabi lityis the exten t to which a person 's score on a test ISaffected by factors whic h are extraneous to it, andt hus In t roduce measurement error. Validity is theapprop riateness or relevance of in ferences drawnfrom test scores or other forms of assessment.

The issue of reliability needs to be addressed first.If a tes t is not reliable, it cannot be valid. Clearly, if aperson's test score IS most ly due to measuremen terror, the test cannot be measuring what it isattempting to measure and thus cannot be valid.

We can test reliabili ty by (Js inq stat isticaltechniques to measure Interna l consistency of theitems, i.e. to see If items which are des igned tomeasure the same dimension show a high degree ofcorrelat ion.

Val id ity, on the other hand, needs to be examinedin a di ffe rent way. The construct validity of a test ISthe extent to which it actually measures what itpurports to measure. One way of demonstrating thisis to correlate the test with other val idated testswhich measure the same or closely relateddimensions. Obviously this method of derr,onstratingconstruct validity is only as good as the eX lstinqpsychological tests against which they aremeasured .

The British Psycholog i cal Society

To demonstrate predictive validi ty of a test isdifferent from construct val idity. In a selectionsituation we may wish to know whether a test is agood predictor of job performance or that itdiscriminates between criterion groups such as moreor less effect ive Job Incumbents. These questionswill be examined In the follow ing section.

Evaluating the PPA and PPSThe cia lms made on behalf of the PPA and PPS areempir ical by their nature. Ma intaining 87 per centaccuracy or suggesting that 40 per cer.t of theselection decision should be based on test scoresconstitute quantitive empirical claims. Evensuggesting that an instrument (test or not) IS aneffective system for Improving select ion decisions'IS a claim which requires at least substantive if notdirect empirica l support. The best means ofevaluating empirical cla ims is by emp irical methods.A 'techn ical manual' was written in 1958 In the USfor the Cleave r tes t. Although Thomas and PALprovide this manual with their tests, it is notlegit imate to use the data reported in this manual toevaluate t he PPS and PPA which are different Inmany respects The present report IS based on PPAand PAL results on 9 19 job applicants.

One of the first prob lems to be mentionedconcerning the PPA and PPS IS that the items usedto measure the four dimens ions are notindependent. Testees are required to indicate whichfour adject ives most and least describe themselves.The dec ision that one of the four dimensions mos tdescr ibes themselves, implies that one of theremaining three dimensions least describesthe m selves. The respondent is thus forced toindicate that he/she has a lower score on theremaining three dimensions, when in reality thetestee could rate high on each of the fourdimensions. This results in the dimens ions beingtotally Interdependent In a way which is impossib leto disentang le.

The PPS and PPA tests assume that sco res,derived from those items which respondents see asleast and most describing themselves, together w iththe composite of these two sco res (most-leas t),prOV ide measures of different characteristics. In thecase of PPA this IS 'how others see you' , 'behaviourunder pressure' and 'how you see yourself'. PPSdescribes 'how others see you' as 'personalityoutward (mask)'. The hypothesis that least and mostmeasure different characteristics is fairlyimplaus ible . Nowhere do Thomas Internat ional orPPS provide evidence to support it. What seemsmore plausible is to assume that 'least scores'measure the reverse end of the dimension measuredby 'most scores ' which is con f irmed by the scoring.So being least decisive is the same as being mostindeCisive. Th e two tes t scores (most and least)obtained for each of the characteristics can beviewed as alternative halves of a test which


3/4

SELF-APPRAISAL PERSONALI TY INVENTORIES continued

measures the oppos ite ends of these bipolardimensions.This is demonstrated, in part, by the correlations

presented in Table 1 between the most and leastscores for the few factors in the PPS In=452) andthe most and least scores in the PPA (n==467) testsThe results suggest that the most and least scoresfactors D, I and S in the PPS and PPA tests aremeasuring opposite ends of the same dimension aswould be predicted from the above argument.

This does not seem to be the case for factor C asthis result Indicates that the most and least scoresmay not be treated as alternate halves of the sametest. The Spearman-Brown split-half rel iab il itycoefficient was ca lculated for each of the four factorsof these two tests. These results are presented InTable 3 and suggest that factors D and S aremeasured rel iably by the PPS and PPA and Iapproaches an acceptable level of reliability for thePPS Factor C is on the other hand not measuredreliably by either PPS or the PPA and I on the PPA isbelow the acceptable level of rel iabil ity.Table 1. Correlations between the most and leastscores on the four factors of the PPS and PPA tests

PPS In=226)PPA (n=314)

D S C-0 '748 -0 '57 -0'646 -0'35-0 '696 -0,45 -0'564 -0,14

Table 2. Split-ha lf reliability coeff iCi ents for thefour factors In the PPS and PPA tests

PPS In= 100)PPA In=l 00)D084070

I0'64046

S0730,71C0,55

0,46Tables 3 and 4 present the correlat ions betweenthese four factors for the PPS and PPA respect ively.These correlations suggest that for both the PPS and

"rable 5. Relationship PPA and 16PF In =; 771Dominance In fluenceA Partic ipating -0-06 0288 Bright 0-23 -0 , 16C Ca lm Stable 01 8 0-27E Ase rtive 055 026F Enthusiastic 024 053

G Conforming - 0 -17 006H Socia lly Bold 0-31 0 -65I Tender-m inded - 003 0-04L Sceptical - 0,24 - 005M Imaginat ive 032 - 0-19N Diplomat ic - 0-23 0080 Apprehensive - 0'17 - 0-4301 Experimenting 0 45 013Q2 Self-sufficient -0 06 -0-3003 Controlled - 0'20 004Q4 Tense Driven 002 - 0'34

the PPA factors Sand D are measuring oppos iteends of the same dimenSion. The correlationbetween Influence and Compliance on both the PPAand PPS are highly elevated, suggesting that thesetwo categories have much in common.Table 3. Intercorrelation matrix PPS (n==452)

D S CD -0 ' 11 -0'73 -0 ,46I 1 -0'30 -0 '50S -0'34CTable 4. IntercorrelatlOn matr ix PPA In=467)

D S CD -0'14 -0'71 - 0'39I -0 '26 - OAlS -0'27CIn orde r to explore the valid ity of the present tests ,the D, I, Sand C of the PPA and PPS were correlatedwi t h the dimenSions measured by the 16PF and OPOrespect ively. Any associations with comp lia nce mus tbe viewed w ith extreme caut ion due to the lowreliab ility of th iS factor on both the PPA and PPS.

Table 5 presents the correlat ions between factorsD, I, Sand Con the PPA and the subscales of the16PF In=77). As Table 5 demonstrates factor D(Dominance) In the PPA IS most highly correlatedWith factor E (Assertive) on the 16PF. Th is finding ISconsistent w ith the hypothesis that factor Dmeasures something akin to dominance.

Factor S (Steadiness) wh ich appears to measurethe reverse of factor D, IS negatively (and moresign ificantly) correlated With factor E of the 16PFproviding further evidence to suggest that factor S ISIn fa ct measuring the re verse of factor D, something

Steadi ness Comp liance MR-0 '16 004 040- 0-12 - 0-01 0 19003 000 028- 0-60 - 033 0-63-OAl - 0 -4 1 06 1

0.Q3 020 026-0 -60 -035 0- 77- 0-09 0-10 0 19- 0'20 - 010 026-0-22 0-02 036023 016 0-320-32 0-14 05 1-0-36 -035 0-450-07 008 0300-10 027 0-31012 010 032

Table 6. Re lationship PP Sand OPO ~ O C T A G O N ) (n = 286)Domina nce Influence Steadiness Co mpliance MRAssertive 043 028 -0 '43 -0 , 28 050Empathy -0 '43 -0 ' 11 048 014 0-49Gregarious -0 ,06 054 -012 _0 ", ' , 056Abstract 011 - 0,03 -0'10 -0'02 013Method ical -0 '24 -0,37 022 044 048Relaxed -0-09 -0 -03 017 - 0,05 021Se lf-control -0,28 - 033 0-33 029 042Vigour 038 -0 ,04 -0 31 - 0,1 4 039

The British Psychological Society

June 1986

Vol ume 2(3 )


4/4

June 1986

Volume 2(3)

6

SELF-APPRAISAL PERSONALITY INVENTORIES continued

more akin to submissiveness than steadiness.In addition to the above correlation factors 0 and Swere also correlated to a lesser extent with factorsQ1. (Experimenting) and H (Socially Bold) on the16PF. While these traits are different fromdominance, the notion of an assertive,experimenting and venturesome person IS broadlyconsistent with the idea of a dominant, socially boldpersonality. Factor I (Influence) was highly correlatedwith factors H (Socially Bold). F (Enthusiastic) and 0(Self-assured) on the 16PF. suggesting that factor l isin fact measuring something akin to a self-assured,enthusiastic, venturesome approach to life.

If we turn to the OPO relationships with the PPS.we find that again Dominance and Steadinesscorrelate with the same factors; in this case withAssert ive and Empathy. The correlation withAssertive could be seen as supportive of the validityof D although the equally high relationship withEmpathy is more!n line with the high S description.I correlates with Gregarious and nega tively withMethodical.Multiple correlations from PPA and PPS with eachof the 16PF and OPO factors reveal the extent towhich factors measured by the OPO and 16PF aremeasured by the best weighted combination of D. I.Sand C. These are listed in Tables 5 and 6respectively. 16PF factors that are well predictedinclude Socially Bold (H). Assertive (E). Enthus iastic(F). Self-assured (0) and Experimenting (01) whichhas about 20 per cent in common with PPAcategories. 16PF factors C (Calm Stable). 04 (TenseDriven). B (Bright). G (Conforming). I (Tenderminded). L (Sceptical). M (Imaginative) and N(Diplomatic) have less than 10 per cent in commonWith the PPA factors.OPO factors Assertive. Empathy. Gregarious andMethodical have over 20 per cent in common withPPS factors whereas Abstract and Relaxed shareless than 5 per cent. This is broadly In agreementwith the 16PF result suggesting that neither thinkingstyle nor emotional stabil ity are measured by the PPSor PPA.

ConclusionsThe present results suggest that there is no reasonto believe that the most. least and combined scoreson the PPA and PPS describe respectively how aperson is seen by others or his behaviour underpressure. A logical analysis of the test suggests thatthe most and the least scores for each of the fourfactors are in fact measuring the opposite ends ofeach of these factors. Correlations between themost and least scores suggest that th is is the casefor the factors D. I and S and that the most and leastscores are best seen as alternate halves of acomposite test. The Spearman-Brown split-halfreliability coefficients suggested that the compositescores for factors D and S on both tests and I on thePPS formed fairly reliable subscales. Factor C onboth the PPS and PPA is unreliable as is factor I onthe PPA.The British Psychological Society

Correlations between the four factors suggestedthat factors D (Dominance) and S (Steadiness) weremeasuring opposite ends of the same scale. Thisfinding was further supported by a validity studywhich correlated factors D. S and I of the PPA andPPS tests with the factors of the 16PF and OPOrespectively. These results suggested that factor Dwas measuring a dominant, v ~ r . t u r e s o m e andexperimenting attitude towards life and that factor S(Stead iness) was measuring a passive. timid andconservative approach to life. These resultstherefore suggest that factor S should bereconceptualized as measuring something more akinto submissiveness than steadiness. The validity datafurther suggested that factor I (Influence) wasmeasuring something more akin to a bold.enthus iastic and confident. approach rather thansocialmfluenceThe present results have therefore demonstratedthat the PPA and PPS fail to live up to their ownclaims or the demands of empirical method. Factor Chas proved to be very unreliable for both the PPA andthe PPS and factors I and particularly S appear to bemeasuring constructs which are different from thosedescribed by the authors, This finding suggests thatthe analysis which IS used to obtain thumb-nailsketches of the candidate's personality is probablyinvalid. This procedure places great emphasis uponthe Importance of interactions between the test'sfour factors, suggesting that various combinations ofthese four factors produce characteristic personalityorientations which are more than the Simplecombination of each of the factors. If the test's fourfactors do not. however. have the psychometricproperties they purport to have. then it seems highlyunlikely that personality descriptions or 'personalitytypes'. wh ich rely upon profiles based uponpsychometrically invalid dimensions, could possiblybe valid.Consequently there seems to be little evidence tosuggest that either the PPA, or the PPS can providethe deta iled analysis of personality which the authorsclaim of their system of profile analysis. In reality, thePPA and the PPS approximate the 16PF and OPOwith regard to two of its factors by offering analternative measure of dominance and sociability.It is worth noting that sociability is one of the mostreliably rated personal attributes from Interview(Ulrich & Trumbo. 1965). It is left to the recruiter orcounseller to decide whether these two dimensionsare of sufficient relevance to make the exerciseworthwh ile.ReferencesCronbach, L.J (1949). Essentials of Psychological Testing . London

Harper & RowCleaver. J P (1965) Self-description: A Technical Manual. Princeton.NJ: Cleaver,Ulrich. L, & Trumbo. D. ( 1 9 6 The selection interview since 1949.Psychological Bulletin, 63 (2L 100-116,

PAL. PPA, OPQ. 16PF, Per/ormax. McQuaig. Life-L ine. Cleaver,Slough. Bucks: NFER.

Laurence Paltiells an Occupational Psychologistwith Psytech International Ltd

Documents

Disc Self Appraisal Inventories