Development of a measure of Generic Work Competencies

The Development and Validation of aMeasure of Generic Work Competencies

Ioannis NikolaouDepartment of Management Science and Technology

Athens University of Economics and Business

Competency management has attracted much attention, especially between businessconsultants and human resources professionals. Nevertheless, the lack of a unifiedframework of generic work competencies has been a significant obstacle for the fur-ther development of the field, both in research and in practice. This article discussesthe development, validation, and psychometric properties of a measure of genericwork competencies assessing three types of generic work competencies.

The assessment of work competencies and especially the development of valid andreliable measures for their assessment have attracted limited attention among re-searchers in the field of personnel psychology. On the contrary, human resourcespractitioners have used the competency approach at length, especially during the1980s and 1990s. Boyatzis (1982) first defined competencies as the certain charac-teristics or abilities of the person that enable him or her to demonstrate the appro-priate specific actions, thereby leading to effective work performance. Thus, on anindividual level, competencies are the capabilities the employee brings to the job.Sparrow (1997), in a review of the use of organizational competencies in personnelselection and assessment, defined competencies as those behavioral repertoires(sets of behavioral patterns) that some people can carry out more effectively thanothers, including all those behaviors that employees bring into the organization toperform well.

Boam and Sparrow (1992), describing the major changes occurring in humanresources management, argued that organizations are looking for an approachthat will enable them to bring changes “by describing global issues in a way thatis sensitive to the local context … drawing upon the language of line managers

INTERNATIONAL JOURNAL OF TESTING, 3(4), 309–319Copyright © 2003, Lawrence Erlbaum Associates, Inc.

Requests for reprints should be sent to Ioannis Nikolaou, 39 Euripides St., Piraeus, Greece 185 32.E-mail: [email protected]

and facilitating long-term changes in individuals’ behaviour” (p. 11), suggestingthat competency-based approaches can offer this alternative. Sparrow andBognanno (1993) claimed that the strategic change in organizations during the1990s reinforced the increased use of competencies. The realization that successin the new competitive environment depends largely on the ability to learn fasterthan competitors, and subsequently reconstruct and adapt the organization, hasfocused attention on internal resources and the capabilities of the organization,such as employees’ competencies, and the integration of human resources poli-cies and practices with business strategies.

Another factor in the development of the competency-based approach may havebeen the recession in the early 1980s, which was followed by the globalization ofbusiness in the 1990s. Traditional personnel practices proved no longer effectivefollowing the wave of change in western European and North American econo-mies. The established job descriptions tailored along the lines of employees andmanagers, who heretofore retained their positions until retirement, ceased to exist,probably forever. Expressions (e.g., leadership potential, innovation, creativity,strategic vision) started appearing as qualifications in job advertisements clearlyseeking a new type of employee.

The competencies approach may be a very influential part of the human re-sources strategy of any organization. Because the appropriate competencies havebeen identified, they can be applied in many organizational activities, includingpersonnel selection and assessment, training, career development, and perfor-mance management. Sparrow and Bognanno (1993) claimed that the creation of acore set of effective behaviors such as competencies may be used in a large set ofassessment settings encouraging mutual behavioral reinforcement across humanresources policy areas.

Feltham (1992) claimed that the competency-based approach contributed to theeffectiveness of selection and assessment methods in three ways: the process, theimplementation, and the evaluation of the approach. The main benefits in adoptinga competency approach in selection and assessment, according to Feltham (1992),are creating shared understanding among various levels of the hierarchy of thekind of personnel needed for new systems; creating more informed human re-sources options; agreeing on standards and enlisting more systematic and scien-tific recruitment processes; writing realistic job previews; and identifying the mostappropriate assessment methods.

Another application of the competency approach, according to Craig (1992), isin career planning and development. He also argued that competencies in careerdevelopment are important in two stages: prior to promotion and toward the middleof the job grade when both organization and the individual should be preparing formovement toward the next grade. He described the following flow:

• Competencies set standards for progression.

310 NIKOLAOU

• Competencies provide the referent for assessment.• Competencies, expressed in terms of strengths and development needs, set

the referent for development.• Competencies set the standards for progression to the next level.

The use of competencies in training resulted from the need to establishcost–benefit links between training and improved job performance(Antonacopoulou & FitzGerald, 1996). Applying competencies in training may bevery appropriate in deciding what to train (Weightman, 1994) and in identifyingspecific competencies that require further improvement at an individual, depart-mental, or organizational level.

Another area where competencies have been widely used is in competence-re-lated pay and performance-related pay schemes (Torrington & Blandamer, 1992).The suitability of competencies in those schemes is justified because they can clar-ify and identify objectives and goals that have to be reached, both for the appraiserand the appraisee, thereby facilitating the appraisal process. In addition, it offers acommon language between them, as well as between personnel management andtrade unions. Armstrong and Brown (1998) claimed that the use of competenciescan help to address traditional failings of performance-related pay, being particu-larly appropriate in sectors where employees’ skills and behaviors are consideredkey to competitive success.

Kurz and Bartram (2002), describing a theoretical model of competency and indi-vidual performance, incorporated competency assessment as an integral part of theirmodel for the World of Work, and Tett, Guternamn, Bleier, and Murphy (2000) devel-oped a taxonomy of 53 competencies clustered under nine general areas: traditionalfunctions, task orientation, dependability, open-mindedness, emotional control, com-munication, developing self and others, occupational acumen, and concerns.

A thorough examination of the competency management literature could notidentify a well-researched and documented model or an instrument measuring ge-neric work competencies across occupations and job levels. The Hay Group, a con-sulting company, has developed a model of competencies that, although its practi-cal application is extensive worldwide, it is not well supported with researchevidences regarding its psychometric properties. All of the researchers or practitio-ners who had worked in the field used organization-specific measures, sometimesdeveloped after a job analysis. Although significantly, organizations did developtailor-made measures assessing a company’s core competencies, some felt that theexistence of a simple, reliable, valid, and easy-to-use measure of generic workcompetencies would facilitate research and practice in the field of competencymanagement. Personnel psychology researchers will be able to use the new instru-ment in criterion-related studies, and human resources practitioners will have a ba-sis to use when developing their own competencies measures. Therefore, the cur-

GENERIC WORK COMPETENCIES 311

rent study describes an attempt to develop and validate a measure of generic workcompetencies.

METHOD

Based on a detailed literature review, experts’ratings analyzed with verbal protocolanalysis (Green, 1995), and a small study carried out in the United Kingdom(Nikolaou, 1999), four major clusters (task orientation, the action–leadership orien-tation, the people, and the communication skills orientation) were identified with 40items. In the task-orientationcluster theemployee isactivelyengagedandorientatedtoward accomplishing his or her duties or tasks, working systematically toward it.The employee’s major concern is successfully executing the job duties. In the ac-tion–leadership orientation the employee is a leading figure within the work envi-ronmentworkingactivelyanddynamically to inspireandmotivatecolleagues. In thepeople- orientation cluster the employee is sensitively and positively orientated to-ward his or her colleagues, taking into account their opinions and suggestions. Theemployee respects her or his colleagues, showing consideration for their problems.Finally, in the communication skills cluster the employee has good oral and writtencommunication abilities. During oral presentations he or she communicates in asimple and comprehensible way, clearly and fluently. The employee writes clearlyand concisely, using correct grammar, style, and language.

The participants were managers in various Greek firms, and they were asked to ratetheir immediate subordinates on each of the 40 items on a scale ranging from 0 to 7.The higher the value the employee received, the better the employee compared on thisparticular competency to his or her colleagues. Participants were also asked to com-plete a short (6-item) questionnaire assessing overall job performance, which would beused as a separate index of the criterion-related validity of the competencies measure.This measure was successfully used in numerous studies (e.g., Robertson, Baron, Gib-bons, MacIver, & Nyfield, 2000; Robertson, Gibbons, Baron, MacIver, & Nyfield,1999) as an overall job performance measure eliciting internal consistency reliabilityof .86. It consists of 6 items where the supervisor has to indicate whether he or sheagrees or disagrees with the behavior described on a 5-point scale. Sample items in-clude: “S/he achieves the objectives of the job, demonstrates expertise in all aspects ofthe job, fulfils all the requirements of the job,” and so forth. An overall job perfor-mance score was calculated by averaging the raw scores of the 6 items.

RESULTS AND CONCLUSIONS

Study 1

Twenty-four managers participated providing useful data for 107 subordinates.The majority of the managerial sample consisted of men (58%) between ages 35

312 NIKOLAOU

and 45 (67%) who were university graduates (46%). Table 1 shows the descriptiveresults of the four scales of the competencies clusters and the job performance in-dex along with their intercorrelations. For ease of comparison, the scale scores arethe average of the individual items of each scale.

Table 1 clearly shows that the competency scales are highly intercorrelated. Thefirst two clusters, the task and the action–leadership orientation, particularly have avery high correlation coefficient of .92. Also, the job performance index has statis-tically significant correlations with all four clusters. According to the managers ofthe sample, a strong positive relation exists among all four types of work compe-tencies and job performance, although the strength of this relation varies amongclusters of competencies.

The reliability of the competencies measure was very high. The Cronbach’s al-pha was .98 and the Guttman’ s split-half was .97 (corrected for length). All the in-dividual scales had also very satisfactory Cronbach’s alpha ranging from .92 up to.96. Similar results came up for the job performance index (α = .95). The high in-ternal consistency reliability is attributed to the high homogeneity and similarity ofthe competency items.

To explore the adequacy of the sample for carrying out the factor analysis, theKaiser-Meyer-Olkin measure was estimated, reaching a value of .93, which is con-sidered very acceptable (Norussis, 1994). Moreover, another index of the samplesuitability, the Bartlett’s test of sphericity, reached a statistically significant level.Following Gorsuch’s (1983) recommendations, these are two basic requirementsto proceed with the exploratory factor analysis because they explore whether theintercorrelation matrix is suitable for performing the factor analysis.

An initial exploratory factor analysis using principal components and obliminrotation was carried out. The extraction method of principal components was cho-sen to extract linear combinations of the variables explaining the majority of thevariance (Kline, 1994), whereas oblimin rotation was applied because it was ex-pected that the resulted factors would be highly intercorrelated. The latter was in-deed the case as Table 1 shows. The results of the factor analysis yielded four fac-tors explaining 76% of the variance; the people-orientation cluster and the


TABLE 1Descriptives and Intercorrelations of the Scales (Study 1)

Scales N M SD α 1 2 3 4

Task orientation (1) 107 4.64 1.61 .92Action–leadership orientation (2) 107 4.25 1.60 .96 .92People orientation (3) 107 4.86 1.45 .96 .64 .72Communication skills (4) 107 4.81 1.38 .93 .82 .78 .62Job performance (5) 107 3.62 0.99 .95 .76 .75 .44 .60

Note. All correlations are significant at the .01 level (two-tailed).

communication skills orientation emerged very clearly but the remaining itemswere distributed between the remaining two factors. The first point that should bemade here is that the participants could apparently distinguish the concepts of peo-ple orientation and communication skills easier than the other two competenciesclusters (i.e., the task orientation and the action–leadership orientation). Subse-quently, these two clear-cut factors (people and communication skills) were in-cluded in the subsequent analyses and that the focus would turn to the interpreta-tion of the remaining two factors.

Numerous exploratory factor analyses were conducted, repetitively excludingitems with low factor loadings, trying to identify a simple, coherent, and interpret-able factor structure, but all the results were similar. Four factors were always com-ing up; two of them were the people and communication factors, whereas the othertwo were combinations of the task and action–leadership items. Subsequently, be-cause the problem mainly concerned the task and action–leadership clusters, oneof the clusters was dropped because the participants could not distinguish betweenthese two groups of competencies. Thus, the action–leadership cluster was keptand the task- orientation items were omitted. This was considered more appropri-ate because it would be the only way of measuring leadership potential, which wasof more interest to the participant-managers, who also received feedback for theirresponses, compared to the task-orientation competencies.

A hierarchical regression analysis (Cohen & Cohen, 1983) was also conducted toexamine whether the remaining items could predict the dropped items of the task-ori-entation cluster. The three scales (people, communication, and action–leadership ori-entation) were entered as independent variables in the equation with the task scale asdependent. Table 2 summarizes the results of the regression analysis, which shows thattwo of the remaining three scales (the action–leadership orientation and the communi-cation skills) predict the task orientation very well with a multiple R = .932 (R2 = .869,F(2,104) = 345.877, p < .000). Dropping the task-orientation items from the subse-quent analyses should not be considered inappropriate because the remaining threecompetencies clusters explained almost 87% of its variance.

The first exploratory factor analysis, using principal component analysis andoblimin rotation, including all but the task-orientation items resulted again in a4-factor solution, with the action–leadership items divided in two groups. Butwhen a 3-factor solution was requested, a very clear and easily identified solutioncame out, explaining 72.237% of the total variance. Each competency scaleemerged very clearly corresponding to each of the three factors. The first factor,where the action–leadership items were loaded, explained the majority of the vari-ance (56.71%). To make the measure more efficient, a few more items, particularlythose with smaller loadings on the perspective factors, were omitted. Six itemswere chosen for the action–leadership and the people-orientation clusters respec-tively, and five for the communication skills cluster. Table 3 shows the resultingfactor pattern matrix, explaining a total variance of 79.5%.

314 NIKOLAOU

315

TABLE 2Hierarchical Regression Analysis of the Competencies Scales (Study 1)

Steps Variables Multiple R R2 Adjusted R2 F p

1Action–leadership .92 .84 .84 573.37 .00

2Action–leadership

communication.93 .87 .87 345.87 .00

B SE of B β T Sig of T

1Action–leadership .93 .04 .92 23.94 .00Constant .71 .18 4.05 .00

2Action–leadership .73 .06 .72 12.65 .00Communication .29 .06 .25 4.37 .00Constant .14 .20 0.70 .48

Note. Dependent variable: the “dropped” task orientation scale.

TABLE 3Exploratory Factor Analysis of the Competencies Scale (Study 1)

Factor

Variables Items 1 2 3

Action–leadership Looks for new activities within his or her workenvironment

.92 –.02 .00

Takes the role of the leader in group activities .89 –.04 –.12Looks for stimulation at work .84 .00 .12Motivates his or her colleagues at work .81 –.05 .08Is willing to commit himself or herself to new tasks .77 –.17 .02Behaves dynamically at work .59 –.16 .21

People Displays kindness toward his or her colleagues .02 –.94 –.05Shows consideration for his or her colleagues .08 –.92 –.05Has good relationships with most of his or her colleagues .03 –.90 .03Shows friendly behavior within the organization .04 –.86 .10Respects his or her colleagues .16 –.85 –.07Shows positive feelings toward his or her colleagues .04 –.77 .07

Communication Uses correct grammar in writing –.00 .02 .95Uses correct spelling in writing .08 .02 .88Uses an appropriate style in writing .30 .16 .78Uses suitable language both in writing and speaking .41 .11 .67Speaks clearly –.15 –.37 .67

Note. Numbers in bold represent XXXXXXXXXXX. Extraction method:principalcomponentanalysis; rotation method: oblimin with Kaiser normalization.

Following that, the relation of these three scales with job performance was ex-plored. Thus, a hierarchical regression analysis was conducted to examine howwell the job performance index is predicted by the three competency clusters. Ta-ble 4 summarizes the results of this regression analysis, which shows that the bestpredictor of the job-performance index was the action–leadership competenciescluster and not the other two clusters. The action–leadership was included in theequation with a multiple R = .706 (R2 =. 499, F(1,105) = 104.629, p < .000). Subse-quently, one can argue that the participants of the study do not consider good com-munication skills or friendly behavior toward others within the organization as be-ing associated with higher job proficiency, at least not as much as the competenciesincluded in the action–leadership orientation cluster.

Study 2

Following the development of the measure, we decided to conduct a second studyto examine the criterion-related validity of the new instrument. The sample con-sisted of 57 managers, the majority of whom were men (72%) between the ages of35 and 45 (53%) and were university graduates (49%). The participants were em-ployed in 22 Greek organizations that completed the competencies and the overalljob performance measures for their subordinates (N = 218). The majority of thefirms operated in the services sector (72%) whereas 18% were industries and 10%were commercial companies. Table 5 presents the descriptive characteristics andthe intercorrelations of the study’s variables, which all show satisfactory internalconsistency with action–leadership competencies showing the highest correlationwith job performance ratings.

An exploratory factor analysis of the competencies measure, using principalcomponent analysis and oblimin rotation, yielded results similar to the first studyestablishing the construct validity of the measure (see Table 6). The three expectedfactors emerged very clearly explaining a total variance of 84.9%. A hierarchicalregression analysis was also conducted with overall job performance as the de-

316 NIKOLAOU

TABLE 4Hierarchical Regression Analysis of the Three Competencies Scales

(Study 1)


1 Action–leadership 0.71 .50 .49 104.62 .00


Action–leadership 0.39 .04 .70 10.23 .00Constant 2.04 .17 12.06 .00

Note. Dependent variable: overall job performance.

pendent variable and the three competency scales as the predictors, eliciting simi-lar results with the results of the first study (see Table 7).

The results of these studies provide evidence for the construct and criterion-re-lated validity of a new measure of generic work competencies. The measure cap-tures three types of competencies: action–leadership, people orientation, and com-munication skills. The existence of a valid and reliable measure of generic work


TABLE 5Descriptives and Intercorrelations of the Variables (Study 2)

Scales N M SD α 1 2 3

Action–leadership orientation (1) 218 4.74 1.58 .93People orientation (2) 218 5.33 1.72 .97 .76Communication skills (3) 218 5.13 1.62 .95 .66 .68Job performance (4) 218 4.00 0.77 .93 .57 .39 .39

Note. All correlations are significant at the .01 level (two-tailed).

TABLE 6Exploratory Factor Analysis of the Competencies Scale (Study 2)

Factor

Variables Items 1 2 3

People Displays kindness toward his or her colleagues 1.02 –.01 –.06Respects his or her colleagues .97 .08 –.09Shows positive feelings toward his or her colleagues .93 .00 .02Shows friendly behavior within the organization .91 .01 .03Has god relationships with most of his or her

colleagues.88 .01 .05

Shows consideration for his or her colleagues .81 –.01 .16Communication Uses correct spelling in writing .05 .96 –.09

Uses correct grammar in writing –.03 .95 .03Uses suitable language both in writing and speaking .06 .92 –.02Uses an appropriate style in writing –.04 .92 .08Speaks clearly .10 .68 .14

Action–leadership Takes the role of the leader in group activities –.03 –.02 .93Motivates his or her colleagues at work .05 –.08 .90Behaves dynamically at work .02 .08 .81Looks for new activities within his or her work

environment.12 .03 .79

Looks for stimulation at work –.04 .25 .74Is willing to commit him or herself to new tasks .41 .03 .45

Note. Numbers in bold represent XXXXXXXXX. Extraction method: principal component anal-ysis; Rotation method: oblimin with Kaiser normalization.

competencies in Greece is a very important first step for the development of com-petency management as part of Greek human resources management.

Our current methodology approached competencies in a “psychologi-cal–psychometric” way in terms of conception and assessment to differentiatefrom “broad” approaches in competencies in the field of human resources manage-ment, which include in the definition of competencies concepts such as motives,traits, attitudes or values, content knowledge, or cognitive or behavioral skills(Hooghiemstra, 1992). Adapting this approach allowed us to build a simple-to-usebut simultaneously effective instrument both for academics researching compe-tence at work as well as human resources professionals in performance and com-petency management.

REFERENCES

Antonacopoulou, E. P., & FitzGerald, L. (1996). Reframing competency in management development.Human Resource Management Journal, 6, 27–47.

Armstrong, M., & Brown, D. (1998). Relating competencies to pay: The UK experience. Compensa-tions and Benefits Review, 30, 28–39.

Boam, R., & Sparrow, P. (1992). The rise and rationale of competency-based approaches. In R. Boam &P. Sparrow (Eds.), Designing and achieving competency (pp. 3–15). London: McGraw-Hill.

Boyatzis, R. (1982). The competent manager. New York: Wiley.Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sci-

ences. London: Lawrence Erlbaum Associates, Inc.Craig, S. (1992). Using competencies in career development. In R. Boam & P. Sparrow (Eds.), De-

signing and achieving competency. A competency-based approach to developing people and organi-zations (pp. 111–127). London: McGraw-Hill.

Feltham, R. (1992). Using competencies in selection and recruitment. In R. Boam & P. Sparrow (Eds.),Designing and achieving competency. A competency-based approach to developing people and or-ganizations (pp. 89–103). London: McGraw-Hill.

Gorsuch, R. L. (1983). Factor analysis (2nd ed.) Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.Green, A. (1995). Verbal protocol analysis. The Psychologist, 8, 126–129.

318 NIKOLAOU

TABLE 7Hierarchical Regression Analysis of the Three Competencies Scales

(Study 2)


1 Action–leadership 0.57 .32 .32 104.59 .00


Action–leadership 0.28 .02 .57 10.22 .00Constant 2.67 .13 19.44 .00

Note. Dependent variable: overall job performance.

Hooghiemstra, T. (1992) Integrated management of human resources. In A. Mitrani, M. Dalziel, & D.Fitt (Eds.), Competency based human resource management. London: Kogan Page.

Kline, P. (1994). An easy guide to factor analysis. London: Routledge.Kurz, R., & Bartram, D. (2002). Competency and individual performance: Modelling the world of

work. In I. T. Robertson, M. Callinan, & D. Bartram (Eds.), Organizational effectiveness. The role ofpsychology, pp. 227–255. Chichester, England: Wiley.

Nikolaou, I (1999). The five-factor model of personality and work behaviour in Greece. Unpublisheddoctoral dissertation, University of Manchester Institute of Science and Technology, England.

Norussis, M. J. (1994). SPSS professional statistics 6.1. Chicago: SPSS Inc.Robertson, I. T., Baron, H., Gibbons, P., MacIver, R., & Nyfield, G. (2000). Conscientiousness and

managerial performance. Journal of Occupational and Organisational Psychology, 73, 171–180.Robertson, I. T., Gibbons, P., Baron, H., MacIver, R., & Nyfield, G. (1999). Understanding manage-

ment performance. British Journal of Management, 10, 5–12.Sparrow, P. R. (1997). Organizational competencies: Creating a strategic behavioural framework for se-

lection and assessment. In N. Anderson & P. Herriot (Eds.), International handbook of selection andassessment (pp. 343–368). Chichester, England: Wiley.

Sparrow, P. R., & Bognanno, M. (1993). Competency requirement forecasting: Issues for internationalselection and assessment. International Journal of Selection and Assessment, 1, 50–58.

Tett, R. P., Guternamn, H. A., Bleier, A., & Murphy, P. J. (2000). Development and content validation ofa “hyperdimensional” taxonomy of managerial competence. Human Performance, 13, 205–251.

Torrington, D., & Blandamer, W. (1992). Competence, pay and performance management. In R. Boam& P. Sparrow (Eds.), Designing and achieving competency. A competency-based approach to devel-oping people and organizations (pp. 137–145). London: McGraw-Hill.

Weightman, J. (1994). Competencies in action. London: IPD.


Documents

Development of a measure of Generic Work Competencies