75
©2019 Leading Psychometric solutions, LLC Technical Validation Report for the REACH Quotient (RQ)

Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

©2019 Leading Psychometric solutions, LLC

Technical Validation Report

for the

REACH Quotient (RQ)

Page 2: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 2

Executive Summary The REACH Quotient (RQ), as a concept and as a measurable characteristic, is intended to reflect

the recognition of one’s primary motivational factors, the agility to leverage diverse interpersonal

styles, and the competency to support an engaged culture amidst continual and unforeseen

change. The concept can be deployed through three psychometric assessments at the individual,

team and organizational levels, including:

• Leading Dimensions Profile (LDP) with REACH Quotient (the combined instrument is branded as

REACH PROFILES)

• REACH Quotient 360-degree Feedback Survey (RQ 360)

• REACH Quotient Culture Survey (RQ Culture)

The enclosed research describes the initial validation effort for the RQ, involving a sample of over

3,000 participants from the US, Canada, South Africa, New Zealand and Australia. While the

primary focus of the research involved the REACH PROFILES, all three assessments are discussed

herein. The most salient findings emerging from this study include:

• RQ reflects a competency map of 16 job-related, skill-based constructs that have been shown to

correlate with higher job performance

• RQ is influenced by interpersonal style preferences, although it is not contained within a single

personality trait

• RQ exhibits very high reliability as a psychometric measure, although it is malleable via training,

development and coaching

• RQ measures facets of emotional intelligence shown to influence individual and team performance

• RQ growth contributes to a culture of increased employee engagement, with the likelihood of less

voluntary turnover.

These findings support the conclusion that RQ provides a reliable, valid and productive measure of

emotional intelligence. Based on the results described in this report, leaders who cultivate RQ may

anticipate stronger job performance, enhanced

engagement and the resilience to gain competitive advantage amidst a rapidly changing

environment.

Page 3: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 3

Table of Contents

Introduction .......................................................................................................................................... 4

Acknowledgements .............................................................................................................................. 5

Statement of Purpose .......................................................................................................................... 6

Framework ........................................................................................................................................... 7

Normative Findings ............................................................................................................................ 17

Construct Reliability ........................................................................................................................... 21

Construct Validity ............................................................................................................................... 23

Criterion-related Validity..................................................................................................................... 31

Transformation of Raw Scores .......................................................................................................... 72

User Agreement ................................................................................................................................. 73

References ........................................................................................................................................ 74

Page 4: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 4

Introduction

Conceptually, REACH is a unified model for organizational development based on the notion that

individuals can relate to others and achieve goals most effectively through targeted training and

coaching. Practically, the REACH model leverages individual and organizational assessments to derive

a metric based on observable traits and skills known to influence organizational health. This metric is

known as the REACH Quotient (RQ). RQ can be utilized to support or inform a variety of applications,

including:

• coaching

• individual and group training

• leadership development

• performance feedback

• succession planning

• teambuilding

The findings presented in this report are intended to provide conceptual and practical support for the

RQ model. The findings contained herein were derived from research conducted or reported from

2016-2018, with an emphasis on validation efforts for the REACH PROFILES, the foundational

component of the unified model. Contributors to this body of research include LDP publishers,

affiliates, scholars, and third-party consultants. As such, this report offers a compilation of research

conducted in workplace settings and academic environments, with participants representing diverse

demographic and occupational populations.

Questions regarding the material contained in this report may be directed to:

Leading Psychometric Solutions, LLC.

1201 6th Avenue West

Suite 100

Bradenton, Florida USA 34205

Page 5: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 5

Acknowledgements

Publisher

Leading Psychometric Solutions, LLC

Editors

Paul Findlay Michael P.

McCoy

R. Douglas Waldo, DBA

Karen Winfield

Editorial Advisory Panel

Hassan Abdullah, PhD

Shennen Bowman, PhD

Cihan Cobanoglu, PhD

Kizzy M. Dominguez, PhD

Rean du Plessis, PhD

Karen Jacobs, PhD

Emily Littlefield, CPA, PhD

Georgia Malan-Rush, PhD

Robert B. Wharton, PhD

Page 6: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 6

Statement of Purpose

The purpose of this report is to provide users and other interested parties with information to

demonstrate the sufficiency, applicability and appropriate utilization of the RQ model. To that end,

this report provides information that is patterned after the requirements for Technical Validation

Reports, as described in the Principles for the Validation and Use of Personnel Procedures (Society

for Industrial and Organizational Psychology [SIOP], 2003). The information contained in this report

covers the array of research conducted on the assessments leveraged within the RQ model by its

publishers, users and third-parties. The information is presented as it was received, and it is true

and correct to the best knowledge of its authors at the time of publication.

Page 7: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 7

Framework As stated in the Introduction, the RQ is a unified model for organizational development. It is based

on the notion that individuals, teams and organizations will be better positioned to thrive in an

environment of volatility, uncertainty, complexity and ambiguity (VUCA) to the extent that their

leaders:

• REcognize the primary motivational factors that drive how team members relate to others and

achieve goals

• demonstrate Agility in leveraging diverse interpersonal styles, to exercise the most contextually-

productive influence over their members

• cultivate the competencies necessary to support an engaged culture amidst continual and

unforeseen CHange

In practice, the RQ leverages individual and organizational assessments to derive a metric based on

observable traits and skills known to influence organizational health. As a unified model, the RQ can

be utilized in a variety of applications, including:

• coaching

• individual and group training

• leadership development

• performance feedback

• succession planning

• teambuilding

Three assessments are included within the RQ model, each incorporating the RQ competency map:

• REACH PROFILES, a psychometric assessment of two primary trait-based influence constructs

(derived from the Leading Dimensions Profile (LDP)) and 16 skillbased constructs within the RQ

competency map. The REACH PROFILES shares a factor structure with the LDP (original version,

published in 2009) emphasizing measures of Relational Drive and Achievement Drive, and their

respective dimensions. The addition of the RQ model within the REACH PROFILES allows for the

measurement of skill-based constructs alongside the trait-based constructs already included in the

original LDP. The REACH PROFILES provides comprehensive insights and recommendations, aimed

at strengthening interpersonal influence and individual performance.

• RQ 360, a multi-rater feedback survey eliciting observations of leadership efficacy within the RQ

model. The RQ 360 observations are based on the same 16 skill-based constructs leveraged in the

REACH PROFILES. In fact, when an individual completes the REACH PROFILES, they also will have

completed the self-rating portion of the RQ 360. The online administration portal integrates

responses from both

assessments, the REACH PROFILES and RQ 360, into a single record within its database structure.

The RQ 360 provides a mechanism for leaders to gain insight and perspective from superiors,

peers and direct reports, aimed at strengthening leadership efficacy and team performance.

Page 8: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 8

• RQ Culture Survey, an organizational culture survey designed to measure general climate and

individual engagement within the RQ model. The RQ Culture Survey leverages the same 16 skill-

based constructs, asking participants to rate the extent to which RQ is demonstrated within their

organizational setting. The RQ Culture Survey provides a mechanism for organizational leaders to

gather feedback from internal stakeholders, aimed at promoting engagement, boosting retention

likelihood and strengthening organizational performance.

Each assessment can be used individually or collectively within the RQ model. The assessments are

published by Leading Psychometric Solutions, LLC and are administered via

www.reachquotient.com. The basis and structure for the RQ model are described briefly as follows.

Psychometrics

Note: The REACH PROFILES is the next generation version of the original LDP (originally published in

2009). The REACH PROFILES shares a common factor and reporting structure with the original LDP.

Given this commonality, the following section includes excerpts from the Technical Validation

Report of the Leading Dimensions Profile (LDP), published in 2018 and available from Leading

Psychometric Solutions, LLC.

The REACH PROFILES framework is intended to reveal a pattern of personality characteristics and

workplace behaviors that describes how individuals exercise influence over others. Referred to as a

profile, this pattern of influence can be applied to diverse settings, such as leadership, sales,

negotiations, learning, conflict-handling, team-building and so on. Based on an exhaustive

literature review of studies dating back more than 50 years and supported by repeated exploratory

and confirmatory analyses, the publishers developed a framework emphasizing two primary

factors: RElational Drive and ACHievement Drive. Ultimately, these factors formed the basis for

the REACH concept, as measured by RQ.

Relational Drive Defined

Relational Drive describes the extent to which an individual engages emotionally in common

circumstances. At opposite ends of the Relational Drive continuum are two primary styles: guarded and

expressive.

• The guarded style may be described as reserved, private and distant in their interactions with others.

Often considered quiet or shy by others, guarded individuals are typically very careful about confiding

in others. They are inclined to maintain a formal and distant approach in most personal interactions

until others gain their confidence and trust. Guarded individuals often prefer to work alone rather than

collaborating with others and they may be considered impatient or disinterested when working within

a team setting.

Page 9: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 9

• The expressive style may appear more outgoing, gregarious and collaborative in their interactions.

Outgoing in nature, expressive individuals are drawn to personal interactions and opportunities to

affiliate with recognized groups. They are often considered very approachable by others, and they will

likely prefer teamwork over individual effort. Expressive individuals are often seen as sensitive and

cooperative in their approach, and they tend to influence others based on emotional persuasion rather

than impersonal facts or direction.

The Relational Drive factor may be similar to or associated with constructs measured by other

instruments, such as: Agreeableness and Interpersonal Sensitivity. Relational Drive may be

demonstrated by five supporting dimensions:

• Affiliation

• Openness

• Consideration

• Status Motivation

• Self-protection

More detailed information regarding the psychometric structure and validation of

Relational Drive and its supporting dimensions can be found in the Technical Validation Report of the

Leading Dimensions Profile (LDP), available from Leading Psychometric Solutions, LLC.

Achievement Drive Defined

Achievement Drive describes the focus and intensity with which an individual approaches common

activities as well as long-term goals. At opposite ends of the Achievement Drive continuum are two

primary styles: methodical and urgent.

• The methodical style may be described as approaching tasks and goals in a cautious, measured and

contemplative manner. Rarely impulsive, methodical individuals are typically very deliberate in their

actions and prefer to consider all possible outcomes before choosing a specific course. They are

inclined to seek clarification and order so that they fully understand both needs and consequences

within the circumstances they face. Others may view methodical individuals as very practical and

consistent in decision-making, preferring order over randomness in reaching conclusions.

• The urgent style may be described as spontaneous, competitive and adaptive. Spontaneous in

nature, urgent individuals are typically very comfortable with ambiguity and do not shy away from

taking action, even without a clear plan.

Their desire for recognizable accomplishments and need for change may cause them to work

at a faster pace than their peers. Urgent individuals are often very concerned with “what’s

next”, and they may be seen by others as very intense and confident in approaching

most circumstances.

Page 10: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 10

The Achievement Drive factor may be similar to or associated with constructs measured by other

instruments, such as: Dominance and Extraversion. Achievement Drive may be demonstrated by

five supporting dimensions:

• Intensity

• Assertiveness

• Risk Tolerance

• Adaptability

• Decision-making

More detailed information regarding the psychometric structure and validation of

Achievement Drive and its supporting dimensions can be found in the Technical Validation Report of

the Leading Dimensions Profile (LDP), available from Leading Psychometric Solutions, LLC.

Profiles Defined

Within the REACH PROFILES framework, the primary factors interact to generate four distinct

profiles:

• The combination of methodical Achievement Drive and expressive Relational

Drive is referred to as the Counselor Profile or Counseling Style. The

Counselor Profile is most recognized for its combination of thinking-orientation and people-

focus.

• The combination of urgent Achievement Drive and expressive Relational Drive is referred to

as the Coach Profile or Coaching Style. The Coach Profile is most recognized for its

combination of acting-orientation and people-focus.

• The combination of urgent Achievement Drive and guarded Relational Drive is referred to as

the Driver Profile or Driving Style. The Driver Profile is most recognized for its combination of

acting-orientation and task-focus.

• The combination of methodical Achievement Drive and guarded Relational Drive is referred

to as the Advisor Profile or Advising Style. The Advisor Profile is most recognized for its

combination of thinking-orientation and task-focus.

These profiles are used to describe the pattern with which individuals may tend to influence one

another in leadership, sales, negotiations, learning, conflict-handling, team-building and in other

related applications.

Page 11: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 11

Figure 1 - 2x2 REACH PROFILES matrix

This framework was operationalized by an initial version of the LDP, whereby only measures of the

two primary factors were generated. Participants’ results were reported on a 2x2 matrix (shown in

Figure 1), where Relational Drive is plotted on the vertical axis and Achievement Drive is plotted on

the horizontal axis. The matrix was divided into four quadrants, labeled as follows:

• Counselor Profile: upper left quadrant, comprised of methodical Achievement

Drive (on the lower extreme, ranging from 0-49%) and expressive Relational Drive (on the

higher extreme, ranging from 50-100%)

• Coach Profile: upper right quadrant, comprised of urgent Achievement Drive

(on the higher extreme, ranging from 50-100%) and expressive Relational Drive (on the

higher extreme, ranging from 50-100%)

• Driver Profile: lower right quadrant, comprised of urgent Achievement Drive (on the higher

extreme, ranging from 50-100%) and guarded Relational Drive (on the lower extreme,

ranging from 0-49%)

• Advisor Profile: lower left quadrant, comprised of methodical Achievement

Drive (on the lower extreme, ranging from 0-49%) and guarded Relational Drive (on the

lower extreme, ranging from 0-49%)

After further data collection and factor analytical procedures, it was determined that the two

primary factors may be comprised of, or related to, smaller constructs (smaller in terms of the

number of items used). Repeated analyses confirmed that between six and ten constructs may

exist within the two-factor framework, in addition to Relational Drive and Achievement Drive. Over

time, these additional constructs became known as Relating Dimensions and Achieving

Dimensions. They have since been leveraged to describe how individuals relate to others and

Page 12: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 12

achieve tasks, providing additional depth and clarity in evaluating the Relational Drive and

Achievement Drive factors, respectively.

The LDP framework has been deployed primarily in two forms:

• a 95-item Form A; the primary and most common form of the LDP and REACH PROFILES.

Within this form, two response formats have been utilized: a single, true-false format; and,

a double true-false format in which each question is answered via an “actual” and a

“perfect” response. Although a focus of scholarly research on assessment faking and

participant perceptions, the double true-false format was deployed on a limited basis after

2017.

• a 46-item Form B; an abbreviated form of the LDP and REACH PROFILES in which only the

two primary factors are measured (no dimensions are reported via this form).

These forms leverage the items and algorithms required to generate the 2x2 matrix, on which the

four distinct profiles are presented. In addition to the 2x2 presentation of the four profiles, the 95-

item Form A provides measures of ten trait-based constructs. Referred to as Relating Dimensions

and Achieving Dimensions, these constructs describe an individual’s likely approach in relating to

others (Relating Dimensions) and in achieving goals (Achieving Dimensions).

While each of the supporting dimensions helps to explain how an individual’s Relational Drive and

Achievement Drive may be observed, these are not necessarily considered psychometric facets of

the two primary factors. The dimensions do share a number of common items with the primary

factors, but only some were derived solely from factor analytical procedures involving Relational

Drive and Achievement Drive. The remaining dimensions emerged as the authors discovered scales

outside of the two-factor structure, with sufficient practical value to be reported as independent

constructs.

In presenting abstract personality constructs, there is often a tendency for users to overemphasize

a lower or higher label on a given dimension. Practically speaking, users often assume that a lower

score may be less preferred than a higher score when evaluating results, when in reality this label

has to do with placement of the individual’s outcome on a distribution of all outcomes. For

example, lower would indicate outcomes below the average outcome for all participants on a

normative distribution. Generally speaking, neither label (on the left or on the right of a continuum)

on any dimension would be considered more valued or desirable.

Page 13: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 13

Multidimensional Scaling

Multidimensional scaling is a technique utilized to graphically support inferences made regarding

potential associations between theoretical constructs. In other words, the technique provides a

picture or map of how constructs may be similar or dissimilar. The relative distance between points

in the multidimensional space reflects the potential correlation between the constructs measured

(Vecchione, Alessandri, Barbaranelli, and Caprara, 2011). In short, the more the constructs appear

to cluster on the map, the more the constructs may tend to relate to each other in revealing an

underlying domain of behavior.

The figure below provides a theoretical map of how the trait-based constructs within the REACH

PROFILES may be associated within the two-factor model. Figure 2 - Multidimensional

scaling of REACH PROFILES dimensions

A similar technique was leveraged to observe the alignment of the 16 skill-based constructs within

the RQ model. A study of 1,700 participants found that each of the profiles reported by the REACH

PROFILES and its predecessor (LDP) was associated with strengths in certain skill-based constructs. In

other words, a participant preferring the Counselor profile would be more likely to show strength in

Page 14: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 14

assimilating new team members within the group when compared to their efficacy in exercising

control over the team (a strength typically found among participants preferring the Driver profile).

The study found that while participants may exhibit strength or weakness in any of the 16

competencies, their respective profile did provide some indication as to the types of skills that a

participant may be more or less comfortable in performing. The figure below provides a theoretical

map of how the skill-based constructs may be associated within the two-factor model.

Figure 3 – Visual plotting of REACH PROFILES skill-based constructs

Based on this alignment, four clusters emerged, with participants tending to indicate stronger efficacy in

certain competencies according to their preferred primary profile:

Participants preferring the Counselor profile tended to show more strength in competencies such as:

• Assimilating team members

• Identifying personal needs

• Cultivating team spirit

• Recognizing others’ efforts

These were designated as Counseling Competencies that, when leveraged effectively, may contribute

to a team culture in which members feel most supported and encouraged to engage emotionally

with others.

Page 15: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 15

Participants preferring the Coach profile tended to show more strength in competencies such as:

• Building rapport

• Easing tensions

• Finding synergy

• Rallying others

These were designated as Coaching Competencies that, when leveraged effectively, may contribute

to a team culture in which members feel most inspired to pursue a compelling vision together.

Participants preferring the Driver profile tended to show more strength in competencies such as:

• Establishing expectations

• Exercising control

• Evaluating performance

• Guiding change

These were designated as Driving Competencies that, when leveraged effectively, may contribute to

a team culture in which team members feel directed within a controlled and accountable setting.

Participants preferring the Advisor profile tended to show more strength in competencies such as:

• Addressing quality concerns

• Designing team structure

• Aligning resources

• Integrating diverse ideas

These were designated as Advising Competencies that, when leveraged effectively, may contribute

to a team culture in which teams feel consulted regarding their perspectives, ideas and provisions.

These skill-based constructs, and their alignment as depicted above, form the basis for the RQ model

as a score. The model is scored based on respondents’ ratings of each competency, which are then

averaged to derive the overall RQ score. Although the respondent type may change based on the

assessment (such as self-ratings on the REACH PROFILES, multi-raters on the RQ 360 and

organizational ratings on the RQ Culture), this underlying alignment is consistently applied within the

RQ framework.

Page 16: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 16

Figure 4 – Conceptual map of the RQ model

Page 17: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 17

Normative Findings The following tables convey normative information regarding participants who completed the

REACH PROFILES from 2017-2018. Norms are presented for the skill-based constructs and trait-

based constructs (factors and dimensions) conveyed within the RQ model.

Norms were gathered at the time of publication for this report. The norms reveal tendencies and

variability based on the prevailing populations within which the REACH PROFILES was administered

prior to publication. As such, the norms may change over time as population proportions change with

the expanding global distribution of the instrument.

Table 1 - Norms for REACH PROFILES skill-based constructs

Constructs Mean StDev Min Max

Reach Quotient (RQ) 3.62 0.75 1.00 5.00

Counseling Competencies Cluster 3.76 0.84 1.00 5.00

Assimilating New Members 3.81 0.98 1.00 5.00

Cultivating Team Spirit 3.80 1.01 1.00 5.00

Identifying Personal Needs 3.57 1.01 1.00 5.00

Recognizing Others’ Efforts 3.87 1.00 1.00 5.00

Coaching Competencies Cluster 3.61 0.87 1.00 5.00

Building Rapport 3.67 1.06 1.00 5.00

Easing Tensions Among Members 3.53 1.07 1.00 5.00

Finding Opportunities For Synergy 3.55 0.98 1.00 5.00

Rallying Others Around A Cause 3.69 1.05 1.00 5.00

Driving Competencies Cluster 3.75 0.86 1.00 5.00

Setting Clear Expectations 3.68 1.03 1.00 5.00

Evaluating Individual Performance 3.59 1.04 1.00 5.00

Controlling Processes 3.51 1.02 1.00 5.00

Approaching Complex Issues 3.43 0.99 1.00 5.00

Advising Competencies Cluster 3.54 1.01 1.00 5.00

Addressing Quality Concerns 3.58 1.00 1.00 5.00

Aligning Resources 3.48 1.02 1.00 5.00

Designing Team Structure 3.50 1.04 1.00 5.00

Integrating Diverse Perspectives 3.58 0.96 1.00 5.00

n=858

* p<.05; ** p<.01

Page 18: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 18

Table 2 - Norms for RQ Culture Survey skill-based constructs Constructs Mean StDev Min Max

Reach Quotient (RQ) 3.22 0.62 1.00 5.00

Counseling Competencies Cluster 3.24 0.78 1.00 5.00

Assimilating New Members 3.14 0.10 1.00 5.00

Cultivating Team Spirit 3.27 1.00 1.00 5.00

Identifying Personal Needs 3.32 0.91 1.00 5.00

Recognizing Others’ Efforts 3.21 0.10 1.00 5.00

Coaching Competencies Cluster 3.31 0.67 1.00 5.00

Building Rapport 3.56 0.78 1.00 5.00

Easing Tensions Among Members 3.08 0.95 1.00 5.00

Finding Opportunities For Synergy 3.24 0.87 1.00 5.00

Rallying Others Around A Cause 3.36 0.93 1.00 5.00

Driving Competencies Cluster 3.23 0.75 1.00 5.00

Setting Clear Expectations 3.18 0.93 1.00 5.00

Evaluating Individual Performance 3.01 1.11 1.00 5.00

Controlling Processes 3.53 0.89 1.00 5.00

Approaching Complex Issues 3.21 0.94 1.00 5.00

Advising Competencies Cluster 3.10 0.79 1.00 5.00

Addressing Quality Concerns 3.12 0.95 1.00 5.00

Aligning Resources 3.09 1.15 1.00 5.00

Designing Team Structure 2.84 1.12 1.00 5.00

Integrating Diverse Perspectives 3.36 0.88 1.00 5.00

n=91

* p<.05; ** p<.01

Page 19: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 19

Table 3 - Norms for RQ 360 skill-based constructs Constructs Mean StDev Min Max

Reach Quotient (RQ) 3.47 0.59 1.00 5.00

Counseling Competencies Cluster 3.54 0.66 1.00 5.00

Assimilating New Members 3.55 0.79 1.00 5.00

Cultivating Team Spirit 3.58 0.84 1.00 5.00

Identifying Personal Needs 3.46 0.80 1.00 5.00

Recognizing Others’ Efforts 3.58 0.87 1.00 5.00

Coaching Competencies Cluster 3.48 0.68 1.00 5.00

Building Rapport 3.51 0.86 1.00 5.00

Easing Tensions Among Members 3.34 0.84 1.00 5.00

Finding Opportunities For Synergy 3.53 0.83 1.00 5.00

Rallying Others Around A Cause 3.51 0.83 1.00 5.00

Driving Competencies Cluster 3.46 0.72 1.00 5.00

Setting Clear Expectations 3.62 0.91 1.00 5.00

Evaluating Individual Performance 3.31 0.80 1.00 5.00

Controlling Processes 3.48 0.89 1.00 5.00

Approaching Complex Issues 3.41 0.85 1.00 5.00

Advising Competencies Cluster 3.39 0.60 1.00 5.00

Addressing Quality Concerns 3.40 0.79 1.00 5.00

Aligning Resources 3.49 0.80 1.00 5.00

Designing Team Structure 3.36 0.74 1.00 5.00

Integrating Diverse Perspectives 3.30 0.72 1.00 5.00

n=181

* p<.05; ** p<.01

Page 20: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 20

Table 4 – Percentile norms for REACH PROFILES trait-based constructs Constructs Mean StDev Min Max

Relational Drive 48.96 29.93 1.00 100.00

Relating Dimensions

Affiliation 48.93 35.68 1.00 100.00

Consideration 48.84 29.89 1.00 100.00

Openness 58.38 33.82 2.00 100.00

Status Motivation 52.78 25.10 0.00 100.00

Self-protection 57.79 29.05 0.00 100.00

Achievement Drive 50.22 27.73 0.00 100.00

Achieving Dimensions

Intensity 65.75 30.98 2.00 100.00

Assertiveness 61.31 29.14 3.00 100.00

Risk Tolerance 56.35 29.97 0.00 100.00

Adaptability 52.23 29.33 4.00 100.00

Decision-making 68.18 24.14 13.00 100.00

n=858 (factors) n=765 (dimensions)

* p<.05; ** p<.01

Summary of Normative Findings

Based on the tables reported, only minimal norm differences were observed within the categories

presented. In no instance did norm differences exceed one standard deviation from the mean. Even

where statistically significant differences were noted, this would not suggest that one category

would be advantaged or disadvantaged based on their norms. A fair outcome can be assumed for

the trait-based constructs (meaning, no adverse impact based on population characteristics), since

the factors and dimensions are not preferential in nature.

Page 21: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 21

Construct Reliability

The psychometric reliability of the RQ was examined using the Cronbach’s Alpha method. The

Cronbach’s Alpha coefficient focuses on content sampling and content homogeneity. Arguably the

most common analysis utilized for psychometric assessments, Cronbach’s Alpha provides an

accurate evaluation regarding the consistency among participants’ responses (Biddle, 2005). This

method leverages specialized correlation formulas to specifically measure the homogeneity

(sameness) of the questions asked by the assessment (that is, the degree to which questions point

to a single construct or dimension) (Cooper and Emory, 1995). For example, Cronbach’s Alpha would

evaluate the extent to which questions within the REACH PROFILES Assertiveness dimension point to

the personality construct of confidence in social and workplace settings. The Cronbach’s Alpha

method was applied to all RQ scorings collected between 2017 and 2018, the results of which are

reported in the table below.

Table 5 – Reliability coefficients for RQ skill-based construct clusters Constructs Items Alpha

Reach Quotient (RQ) 16 0.95

Counseling Competencies Cluster 4 0.86

Coaching Competencies Cluster 4 0.86

Driving Competencies Cluster 4 0.86

Advising Competencies Cluster 4 0.83

n=858

* p<.05; ** p<.01

Page 22: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 22

Table 6 - Reliability coefficients for REACH PROFILES trait-based constructs Constructs Items Alpha

Relational Drive 21 0.82

Relating Dimensions

Affiliation 10 0.87

Consideration 10 0.74

Openness 9 0.79

Status Motivation 14 0.78

Self-protection 9 0.83

Achievement Drive 25 0.86

Achieving Dimensions

Intensity 8 0.78

Assertiveness 11 0.81

Risk Tolerance 8 0.83

Adaptability 8 0.77

Decision-making 8 0.65

n=87

* p<.05; ** p<.01

Summary of Reliability Findings In the field of personnel psychology, reliability refers to the consistency and stability of a psychometric

assessment, both within the assessment (meaning, its items relate to a similar underlying construct) and

between administrations of the same assessment (meaning, its results are fairly consistent over a period of

time). While less important than validity in evaluating an assessment’s effectiveness, reliability is more readily

measurable and worthy of consideration (Cooper and Emory, 1995). A reliable assessment is one that offers

consistent results, providing users with confidence in the decisions made based on those results. This is

especially important when decisions involve workplace implications.

While the Uniform Guidelines for Employment Selection Procedures (USEEOC, 1978) do not provide

a minimum acceptable level of reliability, the US Department of Labor (DOL) suggests that reliability

coefficients of 0.70 and higher generally offer adequate reliability for employment assessment

purposes (Biddle, 2005; DOL, 1999). The DOL indicates that assessments should not be accepted or

rejected based on reliability coefficients, although coefficient values below 0.70 may offer limited

applicability (DOL, 1999). According to Cascio (1998), reliabilities as low as 0.70 are proven useful,

with lower reliabilities offering potential applications for research purposes.

Page 23: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 23

Construct Validity

Construct validity evidence is concerned with examining the extent to which the factors and

dimensions conveyed by the RQ model measure specific constructs of theoretical and/or practical

interest. Such evidence is presented herein according to common sources of construct validity

evidence: factor analytical procedures, multidimensional scaling, scale correlation, internal consistency

and correlations with other instruments (convergent/divergent validity evidence).

As construct validity evidence supports the user’s interpretation and utilization of the RQ model, it

is important to view the evidence in total. In other words, for factors or dimensions observed,

evidence should provide a broad understanding of the constructs estimated. While specific

relationships between constructs may differ based on samples, settings or methods, the

accumulation of evidence is of most relevance. As suggested by Anastasi and Urbina (1997), the

generalizability of the constructs may vary based on the context within which the RQ is researched

or utilized.

Page 24: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 24

REACH PROFILES and LDP (Original)

A sample of participants completed the original LDP and subsequently completed the REACH

PROFILES approximately three weeks later. Participants’ results on the trait-based constructs for

both instruments were compared using bivariate correlation analysis. As expected, there was a high

degree of association between results from the two versions of the original LDP framework.

Table 7 - Correlation matrix: comparing trait-based constructs from REACH PROFILES and

LDP (Original)

Constructs Correlation (r)

Relational Drive 0.74**

Relating Dimensions

Affiliation 0.63**

Consideration 0.74**

Openness 0.77**

Status Motivation 0.77**

Self-protection 0.51**

Achievement Drive 0.87**

Achieving Dimensions

Intensity 0.62**

Assertiveness 0.85**

Risk Tolerance 0.87**

Adaptability 0.48**

Decision-making 0.50**

n=31

* p<.05; ** p<.01

Based on the results reported herein, the primary factors are considered to be substantially similar

between the REACH PROFILES and the original LDP. As such, the psychometric framework is

consistently reflected within the two instruments.

Page 25: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 25

REACH Profiles and Schutte Emotional Intelligence Test Managers, trainers and educational professionals (n=87) completed the REACH PROFILES and later completed

the 33-item Schutte Self-Report Emotional Intelligence Test (SSEIT) (Schutte, Malouff, Hall, Haggerty, Cooper,

and Golden, 1998). The design of the SSEIT reflects an alignment with the Salovey and Mayer (1990) model,

including emotional appraisal, recognition and utilization. Researchers gathered assessment scores for

purposes of evaluating the alignment of REACH PROFILES constructs to a measure of general Emotional

Intelligence. The tables below reflect linear correlation statistics for REACH PROFILES RQ constructs and the

SSEIT.

Table 8 - Correlation matrix: REACH PROFILES skill-based constructs and Schutte Self-Report

Emotional Intelligence Test

Constructs SSEIT

Reach Quotient (RQ) 0.40**

Counseling Competencies Cluster 0.50**

Assimilating New Members 0.42**

Cultivating Team Spirit 0.43**

Identifying Personal Needs 0.42**

Recognizing Others’ Efforts 0.50**

Coaching Competencies Cluster 0.39**

Building Rapport 0.41**

Easing Tensions Among Members 0.32**

Finding Opportunities For Synergy 0.24*

Rallying Others Around A Cause 0.36**

Driving Competencies Cluster 0.29**

Setting Clear Expectations 0.34**

Evaluating Individual Performance 0.28**

Controlling Processes 0.16

Approaching Complex Issues 0.24*

Advising Competencies Cluster 0.25*

Addressing Quality Concerns 0.29**

Aligning Resources 0.12

Designing Team Structure 0.14

Integrating Diverse Perspectives 0.28**

n=87

* p<.05; ** p<.01

Page 26: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 26

Table 9 - Correlation matrix: REACH PROFILES trait-based constructs and Schutte Self-Report

Emotional Intelligence Test

Constructs SSEIT

Relational Drive 0.46**

Relating Dimensions

Affiliation 0.42**

Consideration 0.43**

Openness 0.42**

Status Motivation 0.50**

Self-protection 0.39**

Achievement Drive 0.11

Achieving Dimensions

Intensity 0.01

Assertiveness 0.21

Risk Tolerance -0.05

Adaptability 0.07

Decision-making 0.12

n=87

* p<.05; ** p<.01

Page 27: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 27

REACH Profiles and Fisher Temperament Inventory

Undergraduate psychology and graduate management students (n=113) completed the

REACH PROFILES and later completed the 56-item Fisher Temperament Inventory (JohnsonVickberg and

Christfort, 2017). Researchers provided assessment scores for the purpose of evaluating the alignment of

REACH PROFILES constructs to four biological systems. The tables below reflect linear correlation statistics for

REACH PROFILES constructs and the Fisher Temperament Inventory.

Table 10 - Correlation matrix: REACH PROFILES skill-based constructs and Fisher Temperament

Inventory

Constructs D S T E/O

Reach Quotient (RQ) 0.17 0.08 0.15 -0.12

Counseling Competencies Cluster 0.07 0.07 -0.03 0.03

Assimilating New Members 0.14 0.09 0.07 -0.02

Cultivating Team Spirit 0.09 0.10 -0.01 0.00

Identifying Personal Needs -0.02 -0.01 -0.08 0.04

Recognizing Others’ Efforts 0.04 0.05 -0.08 0.08

Coaching Competencies Cluster 0.18 0.08 0.12 -0.13

Building Rapport 0.16 -0.00 0.06 -0.14

Easing Tensions Among Members 0.04 0.13 0.15 -0.15

Finding Opportunities For Synergy 0.24* 0.02 0.13 -0.12

Rallying Others Around A Cause 0.17 0.14 0.08 -0.05

Driving Competencies Cluster 0.11 0.09 0.19* -0.20*

Setting Clear Expectations 0.16 0.06 0.10 -0.06

Evaluating Individual Performance 0.02 0.03 0.16 -0.25**

Controlling Processes 0.07 0.14 0.21* -0.20*

Approaching Complex Issues 0.11 0.11 0.20* -0.18

Advising Competencies Cluster 0.24* 0.04 0.25** -0.12

Addressing Quality Concerns 0.27** 0.10 0.18 -0.13

Aligning Resources 0.10 0.05 0.23* -0.18

Designing Team Structure 0.24* 0.02 0.19* 0.02

Integrating Diverse Perspectives 0.19* -0.02 0.22* -0.12

n=113

* p<.05; ** p<.01

D=Dopamine System; S=Serotonin System; T=Testosterone System; E/O=Estrogen/Oxytocin System

Page 28: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 28

Table 11 Correlation matrix: REACH PROFILES trait-based constructs and Fisher Temperament

Inventory

Constructs D S T E/O

Relational Drive 0.01 0.14 -0.12 0.31**

Relating Dimensions

Affiliation 0.05 -0.02 -0.04 -0.07

Consideration -0.07 0.19* -0.08 0.37**

Openness 0.00 0.04 -0.13 0.32**

Status Motivation 0.03 -0.07 0.08 -0.13

Self-protection 0.00 -0.11 -0.09 -0.17

Achievement Drive 0.53** 0.07 0.49** -0.22*

Achieving Dimensions

Intensity 0.34** 0.33** 0.40** -0.06

Assertiveness 0.44** 0.20* 0.45** -0.20*

Risk Tolerance 0.41** 0.02 0.39** -0.15

Adaptability 0.45** -0.28** 0.26** -0.14

Decision-making 0.07 -0.39** -0.21* 0.03

n=113

* p<.05; ** p<.01

D=Dopamine System; S=Serotonin System; T=Testosterone System; E/O=Estrogen/Oxytocin System

Page 29: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 29

REACH Profiles and DISC Model

Consultants, training professionals and colleagues (n=47) completed the REACH PROFILES and a survey of the

DISC model. Researchers gathered assessment scores for the purpose of evaluating the alignment of REACH

PROFILES constructs to the four DISC scales. The tables below reflect linear correlation statistics for REACH

PROFILES constructs and the DISC scales.

Table 12 - Correlation matrix: REACH PROFILES skill-based constructs and DISC model

Constructs D I S C

Reach Quotient (RQ) 0.18 -0.12 -0.34* -0.01

Counseling Competencies Cluster 0.09 -0.06 -0.27 -0.04

Assimilating New Members 0.05 -0.07 -0.21 -0.01

Cultivating Team Spirit 0.16 0.02 -0.31* -0.11

Identifying Personal Needs 0.00 -0.09 -0.14 -0.00

Recognizing Others’ Efforts 0.11 -0.08 -0.28 -0.02

Coaching Competencies Cluster 0.16 -0.03 -0.28 -0.09

Building Rapport 0.11 0.08 -0.25 -0.13

Easing Tensions Among Members 0.16 -0.00 -0.29 -0.07

Finding Opportunities For Synergy 0.18 -0.15 -0.24 0.00

Rallying Others Around A Cause 0.08 -0.05 -0.15 -0.08

Driving Competencies Cluster 0.22 -0.14 -0.38** 0.03

Setting Clear Expectations 0.24 -0.15 -0.32* -0.01

Evaluating Individual Performance 0.29* -0.10 -0.44** -0.00

Controlling Processes 0.13 -0.15 -0.33* 0.12

Approaching Complex Issues 0.11 -0.10 -0.24 0.02

Advising Competencies Cluster 0.20 -0.23 -0.30* 0.06

Addressing Quality Concerns 0.15 -0.13 -0.36* 0.10

Aligning Resources 0.06 -0.29* -0.14 0.16

Designing Team Structure 0.19 -0.05 -0.26 -0.10

Integrating Diverse Perspectives 0.23 -0.31* -0.22 0.08

n=47

* p<.05; ** p<.01

D=Dominance; I=Influence; S=Steadiness; C=Compliance

Page 30: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 30

Table 13Correlation Matrix: REACH PROFILES trait-based constructs and DISC model

Constructs D I S C

Relational Drive -0.19 0.07 0.15 -0.02

Relating Dimensions

Affiliation 0.06 0.20 0.04 -0.26

Consideration -0.21 -0.05 0.09 0.11

Openness -0.12 0.19 0.02 -0.08

Status Motivation -0.39** 0.05 0.40** -0.02

Self-protection 0.03 0.01 0.06 -0.12

Achievement Drive 0.50** -0.01 -0.31* -0.23

Achieving Dimensions

Intensity 0.47** -0.27 -0.29* 0.04

Assertiveness 0.57** 0.17 -0.47** -0.38**

Risk Tolerance 0.27 -0.06 -0.10 -0.13

Adaptability 0.01 -0.05 0.00 0.03

Decision-making 0.02 0.47** -0.08 -0.31*

n=47

* p<.05; ** p<.01

D=Dominance; I=Influence; S=Steadiness; C=Compliance

Page 31: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 31

Criterion-related Validity

A number of studies have explored practical relationships between RQ constructs and relevant organizational

measures. Such studies yield evidence of the validity and utility of the constructs when applied to workplace

settings. These studies were concurrent in their design, meaning, performance data were collected from

employers near the time participants completed the surveys (or immediately thereafter). As a result, the

findings presented in this section yield a potential understanding of the participants’ performance at the time

of the respective studies, rather than a clear prediction of future outcomes (Anastasi, and Urbina, 1997).

REACH PROFILES and RQ 360 in Mining Operations

A South African mining company directed its production supervisors and managers to complete the REACH

PROFILES as part of a criterion validation effort. The company provided performance rankings for each

production supervisor based on appraisals completed by their respective managers, leveraging the RQ 360

framework. The data were merged with participants’ REACH PROFILES responses. Researchers analyzed

potential associations between performance and REACH PROFILES results, including both skill-based

constructs (RQ and aligned competencies) and trait-based constructs (primary factors and aligned

dimensions).

The company provided ratings for each production supervisor, based on their respective managers’ direct

observation. In this way, the data reflected a multi-rater approach, wherein production supervisors provided

self-ratings that were then compared with ratings provided by their managers. Managers then grouped their

production supervisors into two categories for the purpose of the analysis. In addition, the company provided

engagement survey results that were analyzed for perceptions regarding safety and production. The tables

that follow reflect variance and linear correlation statistics for RQ constructs and supervisory performance.

Page 32: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 32

Table 14 Comparative analysis: RQ 360 skill-based constructs by performance rank

Constructs

Top 30%

Performers Bottom 30%

Performers

Difference

Reach Quotient (RQ) 3.13 1.95 1.18

Counseling Competencies Cluster 3.25 2.19 1.06

Assimilating New Members 3.50 2.75 0.75

Cultivating Team Spirit 3.25 2.00 1.25

Identifying Personal Needs 3.00 1.75 1.25

Recognizing Others’ Efforts 3.25 2.75 0.50

Coaching Competencies Cluster 3.25 1.81 1.44

Building Rapport 3.25 2.00 1.25

Easing Tensions Among Members 3.75 1.50 2.25

Finding Opportunities For Synergy 3.00 1.50 1.50

Rallying Others Around A Cause 3.00 2.25 1.75

Driving Competencies Cluster 3.25 1.94 1.31

Setting Clear Expectations 3.25 2.00 1.25

Evaluating Individual Performance 3.50 2.25 1.25

Controlling Processes 3.75 2.00 1.75

Approaching Complex Issues 2.50 1.50 1.00

Advising Competencies Cluster 2.75 1.25 1.50

Addressing Quality Concerns 2.75 2.00 0.75

Aligning Resources 2.75 1.50 1.25

Designing Team Structure 2.75 2.00 0.75

Integrating Diverse Perspectives 2.75 2.00 0.75

n=21

* p<.05; ** p<.01

The higher performing supervisors were rated as more competent on 16 out of the 16 RQ competencies.

Statistical significance was not evaluated by the company, although the differences reported above were

substantial across the RQ competencies.

Page 33: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 33

Table 15 Comparative analysis: REACH PROFILES trait-based constructs by performance rank

Constructs

Top 30%

Performers Bottom 30%

Performers

Difference

Relational Drive 48.18 29.00 19.18

Relating Dimensions

Affiliation 63.82 58.10 5.72

Consideration 52.55 30.60 21.95**

Openness 51.00 33.20 17.80

Status Motivation 61.45 43.60 17.85

Self-protection 64.09 46.00 18.09

Achievement Drive 48.91 44.90 4.01

Achieving Dimensions

Intensity 75.91 78.50 2.59

Assertiveness 70.82 70.70 0.12

Risk Tolerance 57.73 61.70 3.97

Adaptability 58.73 44.80 13.93

Decision-making 59.09 46.40 12.69

n=21

* p<.05; ** p<.01

Production supervisors differed significantly in the Consideration dimension of Relational Drive when

categorized by job performance. Although not statistically significant, other differences were notable –

namely, other dimensions of Relational Drive. This may suggest that higher performers were more apt than

their peers to display empathy and interpersonal warmth within their leadership style.

Page 34: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 34

Table 16Correlation matrix: REACH PROFILES skill-based constructs and safety emphasis

Constructs

Safety Emphasis

Reach Quotient (RQ) 0.57**

Counseling Competencies Cluster 0.49**

Assimilating New Members 0.45*

Cultivating Team Spirit 0.54**

Identifying Personal Needs 0.27

Recognizing Others’ Efforts 0.36*

Coaching Competencies Cluster 0.60**

Building Rapport 0.59**

Easing Tensions Among Members 0.64**

Finding Opportunities For Synergy 0.42*

Rallying Others Around A Cause 0.47*

Driving Competencies Cluster 0.52**

Setting Clear Expectations 0.38*

Evaluating Individual Performance 0.56**

Controlling Processes 0.30

Approaching Complex Issues 0.48**

Advising Competencies Cluster 0.48**

Addressing Quality Concerns 0.44*

Aligning Resources 0.48**

Designing Team Structure 0.53**

Integrating Diverse Perspectives 0.33

n=29

* p<.05; ** p<.01

Based on statistics shown above, production supervisors indicating a stronger emphasis on the safety of their

workers scored higher on 16 out of 16 RQ competencies.

Page 35: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 35

Table 17 Correlation matrix: REACH PROFILES trait-based constructs and safety emphasis

Constructs

Safety Emphasis

Relational Drive 0.12

Relating Dimensions

Affiliation 0.19

Consideration 0.13

Openness 0.09

Status Motivation 0.25

Self-protection 0.16

Achievement Drive 0.06

Achieving Dimensions

Intensity 0.07

Assertiveness -0.19

Risk Tolerance -0.15

Adaptability 0.24

Decision-making 0.10

n=29

* p<.05; ** p<.01

There was a nonsignificant linear relationship between safety emphasis and the trait based constructs, with a

minimal inclination potentially associated with more expressive Relational Drive.

Page 36: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 36

Table 18 – Comparative analysis: REACH PROFILES skill-based constructs by position

RQ

Constructs

Supervisor

Average Manager

Average

Difference

Reach Quotient (RQ) 3.93 3.64 0.29

Counseling Competencies Cluster 3.94 3.66 0.28

Assimilating New Members 3.76 3.88 0.12

Cultivating Team Spirit 3.65 3.97 0.32

Identifying Personal Needs 3.85 3.32 0.53

Recognizing Others’ Efforts 4.06 3.91 0.15

Coaching Competencies Cluster 3.90 3.53 0.37

Building Rapport 3.74 3.50 0.24

Easing Tensions Among Members 4.21 3.68 0.53

Finding Opportunities For Synergy 3.85 3.32 0.53

Rallying Others Around A Cause 3.82 3.59 0.23

Driving Competencies Cluster 4.05 3.77 0.28

Setting Clear Expectations 4.12 3.88 0.24

Evaluating Individual Performance 4.21 3.88 0.33

Controlling Processes 4.06 3.71 0.35

Approaching Complex Issues 3.82 3.62 0.20

Advising Competencies Cluster 3.77 3.54 0.23

Addressing Quality Concerns 3.88 3.74 0.14

Aligning Resources 3.79 3.56 0.23

Designing Team Structure 3.74 3.44 0.30

Integrating Diverse Perspectives 3.68 3.44 0.24

n=70

* p<.05; ** p<.01

In a comparison of skill-based constructs by position, production supervisors rated themselves higher on 14

out of 16 RQ competencies when compared to how managers rated themselves.

Page 37: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 37

Table 19 – Comparative Analysis: REACH PROFILES trait-based constructs by position

RQ

Constructs

Supervisor

Average Manager

Average

Difference

Relational Drive 39.41 44.15 4.74

Relating Dimensions

Affiliation 63.32 58.50 4.82

Consideration 41.12 41.97 0.85

Openness 44.91 52.15 7.24

Status Motivation 52.09 47.62 4.47

Self-protection 56.29 43.85 12.44

Achievement Drive 48.47 53.35 4.88

Achieving Dimensions

Intensity 78.50 77.32 1.18

Assertiveness 72.65 71.15 1.50

Risk Tolerance 56.74 44.74 12.00

Adaptability 51.82 49.26 2.56

Decision-making 55.56 66.47 10.91*

n=70

* p<.05; ** p<.01

In a comparison of trait-based constructs by position, production supervisors shared similar traits with their

managers, with the notable exception of the Decision-making dimension of Achievement Drive. The significant

differences reported on the Decisionmaking dimension may suggest that managers were more comfortable

leveraging experience, instinct and perception in making common decisions (with less reliance on detailed

analysis under typical circumstances).

Page 38: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 38

Table 20 Comparative Analysis: RQ 360 skill-based constructs by rater type

RQ SCORE

Constructs Self-

rating Managerial

Rating

Difference

Reach Quotient (RQ) 3.93 2.54 1.39

Counseling Competencies Cluster 3.94 2.72 1.22

Assimilating New Members 3.88 3.13 0.75

Cultivating Team Spirit 3.97 2.63 1.34

Identifying Personal Needs 3.85 2.38 1.47

Recognizing Others’ Efforts 4.06 3.00 1.06

Coaching Competencies Cluster 3.90 2.53 1.37

Building Rapport 3.74 2.63 1.11

Easing Tensions Among Members 4.21 2.63 1.58

Finding Opportunities For Synergy 3.85 2.25 1.60

Rallying Others Around A Cause 3.82 2.63 1.19

Driving Competencies Cluster 4.05 2.60 1.45

Setting Clear Expectations 4.12 2.63 1.49

Evaluating Individual Performance 4.21 2.88 1.33

Controlling Processes 4.06 2.88 1.18

Approaching Complex Issues 3.82 2.00 1.82

Advising Competencies Cluster 3.77 2.32 1.45

Addressing Quality Concerns 3.88 2.38 1.50

Aligning Resources 3.79 2.13 1.66

Designing Team Structure 3.74 2.38 1.36

Integrating Diverse Perspectives 3.68 2.38 1.30

n=21

* p<.05; ** p<.01

Page 39: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 39

As noted earlier, a multi-rater approach was applied to data collection for this study. Specifically, production

supervisors provided self-ratings for each of the RQ competencies. Managers were asked to rate production

supervisors’ performance on the RQ competencies as well. Interestingly, the production supervisors rated

themselves higher on 16 out of 16 competencies as compared to how their respective managers evaluated

these same competencies.

Page 40: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 40

Table 21 Comparative Analysis: : job performance by REACH PROFILES scores

RQ Score

Constructs

Above Below Mean

Mean

Difference

Contractor safety alignment 4.67 4.22 0.45

Accountability to deliver results 4.72 4.22 0.50*

Clarity in role expectations 4.72 4.33 0.39

Action supported by long-term plan 4.50 3.67 0.83*

Understand safety requirements 4.61 4.22 0.39

Pressured to violate safety rules 0.61 2.11 -1.50*

Safety emphasized over production 4.50 3.44 1.06*

Coaching minutes dedicated daily 49.69 40.00 9.69

n=15

* p<.05; ** p<.01

The company provided production supervisors’ responses to an engagement survey that had been

administered prior to and independent of the study. The production supervisors were categorized based on

their RQ score of above or below a mean of 3.62. A number of survey items indicated notable differences in

responses based on the production supervisors’ average RQ score.

The REACH PROFILES results provided substantive insight as to the performance and engagement of

production supervisors. Namely, higher performing production supervisors scored higher across all RQ clusters

and competencies, contributing to strong appreciation for the RQ framework in differentiating higher and

lower performing leaders. Further, production supervisors with higher RQ scores tended to exhibit a stronger

emphasis on recognizing safety and accountability as essential aspects of their role.

Production supervisors appeared to have a more positive view of their own RQ competencies as compared to

how they had been rated by their managers. In addition, production supervisors tended to self-rate their RQ

competencies more positively than the managers self-rated their own competencies. This finding underscores

the value of the multi-rater aspect of the RQ 360, which measures and compares such perceptions within the

RQ framework. Finally, production supervisors with higher RQ tended to devote more time to coaching their

employees, spending as much as 39 more hours on coaching annually.

Page 41: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 41

REACH PROFILES and RQ 360 in Retail Store Management

A Canadian retail store chain directed its store managers to complete the REACH PROFILES as part of a

criterion validation effort. The company also provided a performance ranking of each store manager according

to their observed competencies within the RQ 360 framework. The data were merged with managers’ REACH

PROFILES responses. Researchers analyzed potential associations between performance and REACH PROFILES

results, including both skill-based constructs (RQ and aligned competencies) and trait-based constructs

(primary factors and aligned dimensions). The tables that follow reflect linear correlation and variance

statistics for REACH PROFILES constructs and store manager performance.

Table 22 - Correlation matrix: REACH PROFILES skill-based constructs from self-ratings compared

to district manager-provided ratings via RQ 360

Constructs Correlation (r)

Reach Quotient (RQ) 0.29**

Counseling Competencies Cluster 0.25*

Assimilating New Members 0.02

Cultivating Team Spirit 0.01

Identifying Personal Needs 0.26*

Recognizing Others’ Efforts 0.28**

Coaching Competencies Cluster 0.31**

Building Rapport 0.23*

Easing Tensions Among Members 0.37**

Finding Opportunities For Synergy 0.02

Rallying Others Around A Cause 0.22*

Driving Competencies Cluster 0.20

Setting Clear Expectations 0.19

Evaluating Individual Performance -0.05

Controlling Processes 0.20

Approaching Complex Issues 0.19

Advising Competencies Cluster 0.28*

Addressing Quality Concerns 0.14

Aligning Resources 0.25*

Designing Team Structure 0.08

Integrating Diverse Perspectives 0.23*

n=82

* p<.05; ** p<.01

Page 42: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 42

Most RQ competencies exhibited linear associations between self-ratings and district manager-provided

ratings. Generally, there was significant agreement between store managers and their district managers in the

evaluation of RQ competencies.

Page 43: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 43

Table 23 - Comparative analysis: REACH PROFILES skill-based constructs compared by rater

source

RQ

Constructs

Self-

ratings

District ManagerProvided

Ratings

Difference

Reach Quotient (RQ) 3.74 3.19 0.55

Counseling Competencies Cluster 3.80 3.28 0.52

Assimilating New Members 3.81 3.29 0.52

Cultivating Team Spirit 3.88 3.28 0.60

Identifying Personal Needs 3.63 3.28 0.35

Recognizing Others’ Efforts 3.88 3.27 0.61

Coaching Competencies Cluster 3.76 3.19 0.57

Building Rapport 3.79 3.23 0.56

Easing Tensions Among Members 3.66 3.02 0.44

Finding Opportunities For Synergy 3.77 3.29 0.48

Rallying Others Around A Cause 3.83 3.21 0.62

Driving Competencies Cluster 3.73 3.17 0.56

Setting Clear Expectations 3.90 3.34 0.56

Evaluating Individual Performance 3.62 2.99 0.63

Controlling Processes 3.76 3.21 0.55

Approaching Complex Issues 3.67 3.15 0.52

Advising Competencies Cluster 3.63 3.14 0.49

Addressing Quality Concerns 3.67 3.13 0.54

Aligning Resources 3.70 3.27 0.43

Page 44: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 44

Designing Team Structure 3.65 3.07 0.58

Integrating Diverse Perspectives 3.53 3.07 0.46

n=82

* p<.05; ** p<.01

In a comparison of skill-based constructs by position, store managers rated themselves higher on 16 out of 16

RQ competencies when compared to ratings provided by their district managers.

Table 24 Correlation analysis: performance appraisals and engagement results by rater source

Correlation (r)

Constructs

Self-

ratings

District ManagerProvided

Ratings

Performance Appraisal 0.19 0.38

Leadership -0.01 0.54*

Training and development 0.27 -0.02

Teambuilding 0.19 0.41

Teamwork -0.14 0.13

Strategic thinking 0.23 0.11

Store presentation 0.20 0.27

Store safety 0.14 0.09

Recruiting 0.34 0.36

Loss prevention 0.17 0.03

Integrity -0.24 0.39

Financial management 0.00 0.11

Customer focus 0.16 0.34

Engagement Survey 0.05 0.30

Rated by superior 0.12 0.42

Rated by direct reports -0.19 0.26

Rated by self 0.31 0.06

n=17

* p<.05; ** p<.01

Page 45: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 45

The REACH PROFILES self-ratings did not reveal significant linear associations with the performance appraisals

or engagement survey results. However, the RQ 360 ratings provided by the District Managers exhibited more

consistent, positive associations with performance appraisals and engagement survey results.

Page 46: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 46

REACH PROFILES and RQ Culture Survey in Medical Research Financial

Management

A US-based medical school directed its staff to complete the REACH PROFILES as part of a leadership

development effort. In addition to the REACH PROFILES, the staff completed the RQ Culture Survey, a

questionnaire designed to measure engagement factors within the RQ model. The table below reflects linear

correlation statistics for REACH PROFILES skill-based constructs and the RQ Culture Survey scores.

Table 25 - Correlation matrix: REACH PROFILES skill-based constructs and engagement factors

Constructs

Enjoy

Work Respect

Leaders Retention

Likelihood

Reach Quotient (RQ) 0.02 0.09 -0.12

Counseling Competencies Cluster 0.02 0.07 -0.06

Assimilating New Members 0.04 0.03 -0.02

Cultivating Team Spirit 0.00 0.10 -0.07

Identifying Personal Needs -0.02 0.06 -0.09

Recognizing Others’ Efforts 0.05 0.03 -0.01

Coaching Competencies Cluster 0.01 0.09 -0.12

Building Rapport -0.03 0.07 -0.09

Easing Tensions Among Members 0.03 0.10 -0.05

Finding Opportunities For Synergy 0.03 0.07 -0.12

Rallying Others Around A Cause 0.01 0.07 -0.12

Driving Competencies Cluster 0.06 0.12 -0.12

Setting Clear Expectations 0.08 0.07 -0.10

Evaluating Individual Performance -0.01 0.10 -0.13

Controlling Processes 0.07 0.14 -0.09

Approaching Complex Issues 0.07 0.10 -0.09

Advising Competencies Cluster -0.02 0.06 -0.09

Addressing Quality Concerns 0.06 0.11 -0.06

Aligning Resources 0.02 0.04 -0.04

Designing Team Structure -0.02 -0.02 -0.13

Integrating Diverse Perspectives -0.12 -0.05 -0.12

n=81

* p<.05; ** p<.01

The analysis indicated nonsignificant associations between skill-based constructs and the engagement

factors: enjoyment at work, respect for leaders and retention likelihood.

Page 47: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 47

Table 26 Correlation matrix: REACH PROFILES trait-based constructs and engagement factors

Constructs

Enjoy

Work Respect

Leaders Retention

Likelihood

Relational Drive 0.06 0.13 0.21*

Relating Dimensions

Affiliation 0.26* 0.17 0.16

Consideration -0.08 0.09 0.21*

Openness 0.08 0.05 0.09

Status Motivation -0.02 -0.02 0.17

Self-protection 0.29** 0.21* 0.20

Achievement Drive 0.00 -0.02 -0.05

Achieving Dimensions

Intensity 0.06 0.04 -0.01

Assertiveness -0.10 -0.17 -0.11

Risk Tolerance -0.04 -0.04 -0.06

Adaptability 0.05 0.02 -0.06

Decision-making 0.10 0.10 -0.02

n=81

* p<.05; ** p<.01

The analysis yielded significant correlation between engagement factors and Relational Drive, as well as with

three of its dimensions.

Page 48: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 48

The participant sample was divided into two groups based on position: Team members and their respective

team leaders. REACH PROFILES results were compared by position.

Table 27 - Comparative analysis: REACH PROFILES skill-based constructs by position

Team

Constructs Members Leaders Difference

Reach Quotient (RQ) 3.29 3.40 0.11

Counseling Competencies Cluster 3.51 3.14 0.37

Assimilating New Members 3.66 3.29 0.37

Cultivating Team Spirit 3.49 3.29 0.20

Identifying Personal Needs 3.28 2.71 0.47

Recognizing Others’ Efforts 3.59 3.29 0.30

Coaching Competencies Cluster 3.26 3.50 0.24

Building Rapport 3.27 3.71 0.44

Easing Tensions Among Members 3.08 3.00 0.08

Finding Opportunities For Synergy 3.32 3.57 0.25

Rallying Others Around A Cause 3.36 3.71 0.35

Driving Competencies Cluster 3.10 3.29 0.19

Setting Clear Expectations 3.12 3.43 0.31

Evaluating Individual Performance 3.20 2.86 0.34

Controlling Processes 3.05 3.14 0.09

Approaching Complex Issues 3.01 3.71 0.70

Advising Competencies Cluster 3.30 3.64 0.34

Addressing Quality Concerns 3.47 3.71 0.27

Aligning Resources 3.16 3.29 0.13

Designing Team Structure 3.18 4.00 0.82

Integrating Diverse Perspectives 3.39 3.57 0.18

n=81

* p<.05; ** p<.01

Based on the comparative analysis, there were no significant differences in RQ by position. Generally, team

leaders scored slightly higher across the RQ competencies.

Page 49: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 49

Table 28 Comparative analysis: REACH PROFILES trait-based constructs by position

Team

Constructs Members Leaders Difference

Relational Drive 47.92 34.43 13.49

Relating Dimensions

Affiliation 39.84 48.57 8.73

Consideration 50.82 33.14 17.68

Openness 58.58 48.71 9.87

Status Motivation 54.62 52.14 2.48

Self-protection 63.81 68.00 4.19

Achievement Drive 36.14 50.14 14.00

Achieving Dimensions

Intensity 55.57 58.00 2.43

Assertiveness 47.82 54.43 6.61

Risk Tolerance 46.36 59.00 12.64

Adaptability 42.88 50.14 7.26

Decision-making 62.14 64.86 2.72

n=81

* p<.05; ** p<.01

Based on the comparative analysis, there were no significant differences in REACH PROFILES factors or

dimensions by position.

Page 50: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 50

Table 29 - Correlation matrix: team leaders’ REACH PROFILES skill-based constructs and team

members’ engagement ratings

Constructs

Enjoy

Work Respect

Leaders Retention

Likelihood

Reach Quotient (RQ) 0.18 0.29** 0.24*

Counseling Competencies Cluster 0.13 0.19 0.16

Assimilating New Members 0.13 0.20 0.17

Cultivating Team Spirit 0.10 0.15 0.17

Identifying Personal Needs 0.11 0.13 0.07

Recognizing Others’ Efforts 0.13 0.20 0.17

Coaching Competencies Cluster 0.23* 0.34** 0.23*

Building Rapport 0.26* 0.35** 0.21

Easing Tensions Among Members 0.20 0.29** 0.19

Finding Opportunities For Synergy 0.20 0.29** 0.21

Rallying Others Around A Cause 0.20 0.32** 0.25*

Driving Competencies Cluster 0.18* 0.33** 0.27*

Setting Clear Expectations 0.16 0.33** 0.29**

Evaluating Individual Performance 0.19 0.28* 0.23*

Controlling Processes 0.11 0.20 0.14

Approaching Complex Issues 0.21 0.39** 0.31**

Advising Competencies Cluster 0.14 0.27* 0.26*

Addressing Quality Concerns 0.06 0.16 0.22*

Aligning Resources 0.15 0.32** 0.29**

Designing Team Structure 0.16 0.29** 0.26*

Integrating Diverse Perspectives 0.16 0.24* 0.20

n=81

* p<.05; ** p<.01

The correlation analysis revealed significant association between the team leaders’ skill-based constructs and

their team members’ engagement. Specifically, team members with higher engagement tended to work

within teams whose leaders had higher RQ. Further, team leaders with higher RQ seemed to promote an

environment in which their team members expressed greater enjoyment at work, a greater respect for their

leaders and a lower likelihood to voluntarily terminate their employment over the near-term.

Page 51: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 51

RQ Culture Survey was administered to participants attending a leadership development session. This survey

is comprised of 16 ratings, designed to evaluate participants’ observation of RQ within their working

environment. In addition, the survey elicits ratings of workplace engagement, including the extent to which

the participants find enjoyment in their work, respect their team leaders and intend to remain employed at

the organization over the near-term.

Page 52: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 52

Table 30 - Correlation matrix: RQ Culture Survey scores and team members’ engagement ratings

Constructs

Enjoy

Work Respect

Leaders Retention

Likelihood

Reach Quotient (RQ) 0.54** 0.65** 0.41**

Counseling Competencies Cluster 0.46** 0.57** 0.30**

Assimilating New Members 0.40** 0.41** 0.23*

Cultivating Team Spirit 0.40** 0.46** 0.23*

Identifying Personal Needs 0.27** 0.50** 0.21*

Recognizing Others’ Efforts 0.39** 0.47** 0.30**

Coaching Competencies Cluster 0.55** 0.61** 0.51**

Building Rapport 0.49** 0.60** 0.41**

Easing Tensions Among Members 0.25* 0.29** 0.32**

Finding Opportunities For Synergy 0.46** 0.55** 0.46**

Rallying Others Around A Cause 0.49** 0.46** 0.37**

Driving Competencies Cluster 0.44** 0.47** 0.29**

Setting Clear Expectations 0.36** 0.37** 0.20

Evaluating Individual Performance 0.31** 0.26* 0.09

Controlling Processes 0.43** 0.49** 0.35**

Approaching Complex Issues 0.26* 0.37** 0.30**

Advising Competencies Cluster 0.38** 0.52** 0.28**

Addressing Quality Concerns 0.37** 0.27** 0.29**

Aligning Resources 0.24* 0.37** 0.06

Designing Team Structure 0.36** 0.52** 0.29**

Integrating Diverse Perspectives 0.18 0.42** 0.24*

n=91

* p<.05; ** p<.01

The correlation analysis revealed significant association between the participants’ RQ Culture Survey score, its

facets and the general engagement level of participants within the organization. Namely, participants

observing a higher degree of RQ within their working environment tended to report significantly stronger

inclination to find enjoyment in their work, respect their team leaders and intend to remain employed at the

organization over the near-term.

REACH PROFILES in Healthcare Management

An Australian aged care provider directed its residence managers to complete the REACH PROFILES as part of

a criterion validation effort. The company provided a category rating for each residence manager, based on

recent performance appraisals. The data were merged with managers’ REACH PROFILES responses.

Researchers analyzed potential associations between the performance rating and REACH PROFILES results,

Page 53: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 53

including both skill-based constructs (RQ and aligned competencies) and trait-based constructs (primary

factors and aligned dimensions). The tables that follow reflect linear correlation and variance statistics for

REACH PROFILES constructs and managerial performance.

Table 31 - Correlation matrix: REACH PROFILES skill-based constructs and performance rating

Constructs Rating

Reach Quotient (RQ) 0.13

Counseling Competencies Cluster 0.16

Assimilating New Members 0.14

Cultivating Team Spirit 0.21

Identifying Personal Needs 0.08

Recognizing Others’ Efforts 0.18

Coaching Competencies Cluster 0.19

Building Rapport 0.25

Easing Tensions Among Members 0.14

Finding Opportunities For Synergy 0.11

Rallying Others Around A Cause 0.19

Driving Competencies Cluster 0.04

Setting Clear Expectations 0.01

Evaluating Individual Performance -0.01

Controlling Processes 0.18

Approaching Complex Issues -0.05

Advising Competencies Cluster 0.13

Addressing Quality Concerns 0.19

Aligning Resources 0.13

Designing Team Structure 0.09

Integrating Diverse Perspectives 0.08

n=29

* p<.05; ** p<.01

Correlation statistics indicated nonsignificant linear associations between managerial performance and the RQ

competency. Generally, the Building Rapport and Cultivating Team Spirit competencies appeared to yield the

most consistent association with performance.

Table 32 Correlation matrix: REACH PROFILES trait-based constructs and performance rating

Constructs Rating

Relational Drive -0.11

Page 54: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 54

Relating Dimensions

Affiliation 0.11

Consideration -0.18

Openness 0.01

Status Motivation -0.24

Self-protection -0.05

Achievement Drive -0.09

Achieving Dimensions

Intensity -0.15

Assertiveness -0.11

Risk Tolerance -0.02

Adaptability -0.02

Decision-making 0.16

n=29

* p<.05; ** p<.01

Correlation statistics indicated that the association between trait-based constructs and performance were

mostly nonlinear in nature. Generally, the higher performers tended to reflect more of a more guarded (than

expressive) Relational Drive and a more methodical (rather than urgent) Achievement Drive.

Page 55: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 55

For purposes of comparing performance differences based on RQ score, the sample was divided into two

groups: higher and lower performance ratings. The table below reflects differences in RQ competencies based

on managerial performance.

Table 33 - Comparative analysis: REACH PROFILES skill-based constructs by performance rating

Constructs

Higher

Performers Lower

Performers

Difference

Reach Quotient (RQ) 3.96 3.47 0.49

Counseling Competencies Cluster 4.14 3.63 0.51

Assimilating New Members 4.17 3.67 0.50

Cultivating Team Spirit 4.13 3.50 0.63

Identifying Personal Needs 3.96 3.50 0.46

Recognizing Others’ Efforts 4.30 3.83 0.47

Coaching Competencies Cluster 3.96 3.33 0.63

Building Rapport 4.09 3.17 0.92*

Easing Tensions Among Members 3.87 3.33 0.54

Finding Opportunities For Synergy 3.87 3.33 0.54

Rallying Others Around A Cause 4.00 3.50 0.50

Driving Competencies Cluster 3.91 3.54 0.37

Setting Clear Expectations 4.00 3.67 0.33

Evaluating Individual Performance 3.87 3.50 0.37

Controlling Processes 4.00 3.33 0.67

Approaching Complex Issues 3.78 3.67 0.11

Advising Competencies Cluster 3.82 3.33 0.49

Addressing Quality Concerns 4.00 3.50 0.50

Aligning Resources 3.65 3.00 0.65

Designing Team Structure 3.91 3.50 0.41

Integrating Diverse Perspectives 3.70 3.33 0.37

n=29

* p<.05; ** p<.01

Managerial performance differed substantially based on RQ scores. Specifically, managers in the higher

performing group exhibited higher scores on 16 out of 16 RQ competencies.

Page 56: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 56

Table 34 Comparative analysis: REACH PROFILES trait-based constructs by performance rating

Constructs

Higher

Performers Lower

Performers

Difference

Relational Drive 59.22 72.67 13.45

Relating Dimensions

Affiliation 68.96 62.50 6.46

Consideration 52.04 75.83 23.79*

Openness 63.17 61.83 1.34

Status Motivation 60.57 74.83 14.26

Self-protection 71.22 77.33 6.11

Achievement Drive 55.70 54.50 1.20

Achieving Dimensions

Intensity 58.04 62.67 4.63

Assertiveness 62.61 66.17 3.56

Risk Tolerance 61.74 51.50 10.24

Adaptability 72.22 68.17 4.05

Decision-making 70.52 51.00 19.52

n=29

* p<.05; ** p<.01

Managerial performance differed substantially based on trait-based constructs. Specifically, managers in the

higher performing group exhibited significant difference in their scores on the Consideration dimension of

Relational Drive. Generally, the higher performing managers tended to reveal more objectivity (than

nurturing) in interpersonal interactions and leverage more perception and experience (than analysis) in

decisionmaking.

In summary, REACH PROFILES results indicated association with performance among the managers. The

findings reported herein suggest the employer may benefit from evaluating and cultivating growth in RQ

among the managerial ranks.

Page 57: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 57

REACH PROFILES in Industrial Tools and Equipment

A US-based industrial tool and equipment company directed its route sales associates to complete the REACH

PROFILES as part of a criterion validation effort. The company provided a performance rating for each

associate, based on managerial evaluations of sales activities. The data were merged with associates’ REACH

PROFILES responses. Researchers analyzed potential associations between performance and REACH PROFILES

results, including both skill-based constructs (RQ and aligned competencies) and trait-based constructs

(primary factors and aligned dimensions). The tables that follow reflect linear correlation and variance

statistics for REACH PROFILES constructs and sales performance.

Page 58: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 58

Table 35 Correlation matrix: REACH PROFILES skill-based constructs and performance ratings

Constructs

Performance

Rating

Reach Quotient (RQ) 0.49*

Counseling Competencies Cluster 0.55*

Assimilating New Members 0.24

Cultivating Team Spirit 0.46*

Identifying Personal Needs 0.36

Recognizing Others’ Efforts 0.31

Coaching Competencies Cluster 0.23

Building Rapport 0.16

Easing Tensions Among Members 0.16

Finding Opportunities For Synergy 0.13

Rallying Others Around A Cause 0.30

Driving Competencies Cluster 0.33

Setting Clear Expectations 0.37

Evaluating Individual Performance 0.30

Controlling Processes 0.26

Approaching Complex Issues 0.21

Advising Competencies Cluster 0.42

Addressing Quality Concerns 0.45*

Aligning Resources 0.25

Designing Team Structure 0.43

Integrating Diverse Perspectives 0.28

n=19

* p<.05; ** p<.01

Correlation statistics indicated a positive linear association between performance ratings and the skill-based

REACH PROFILES competencies. Overall, the Counseling Competencies Cluster appeared to offer the most

consistent association with performance ratings.

Page 59: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 59

Table 36 Correlation matrix: REACH PROFILES trait-based constructs and performance ratings

Constructs

Performance

Rating

Relational Drive -0.37

Relating Dimensions

Affiliation -0.33

Consideration 0.19

Openness -0.31

Status Motivation 0.12

Self-protection -0.32

Achievement Drive 0.50*

Achieving Dimensions

Intensity 0.14

Assertiveness 0.32

Risk Tolerance 0.55*

Adaptability 0.37

Decision-making 0.38

n=19

* p<.05; ** p<.01

Correlation statistics indicated a significant linear association between performance ratings and the trait-

based factor, Achievement Drive, as well as its Risk Tolerance dimension. Generally, those associates

exhibiting more urgency and boldness within their sales approach tended to experience stronger performance

outcomes.

Page 60: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 60

Table 37 Comparative analysis: REACH PROFILES skill-based constructs by performance rating

Constructs Top Others Difference

Reach Quotient (RQ) 4.50 3.78 0.72*

Counseling Competencies Cluster 4.68 3.95 0.73

Assimilating New Members 4.71 4.09 0.62

Cultivating Team Spirit 4.63 3.91 0.72*

Identifying Personal Needs 4.38 3.64 0.74

Recognizing Others’ Efforts 4.38 4.18 0.20

Coaching Competencies Cluster 4.25 3.91 0.34

Building Rapport 4.50 4.36 0.14

Easing Tensions Among Members 4.13 3.73 0.40

Finding Opportunities For Synergy 4.00 3.82 0.18

Rallying Others Around A Cause 4.38 3.73 0.65

Driving Competencies Cluster 4.31 3.73 0.58

Setting Clear Expectations 4.50 3.91 0.59

Evaluating Individual Performance 4.38 4.00 0.38

Controlling Processes 4.13 3.46 0.67

Approaching Complex Issues 4.25 3.55 0.70

Advising Competencies Cluster 4.19 3.55 0.64

Addressing Quality Concerns 4.25 3.36 0.89*

Aligning Resources 4.25 3.82 0.43

Designing Team Structure 4.13 3.55 0.58

Integrating Diverse Perspectives 4.13 3.46 0.67*

n=19

* p<.05; ** p<.01

The analysis of variance found significant performance differences based on the associates’ REACH PROFILES

skill-based competencies.

Page 61: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 61

Table 38 Comparative analysis: REACH PROFILES trait-based constructs by performance rating

Constructs Top Other Difference

Relational Drive 57.25 61.00 3.75

Relating Dimensions

Affiliation 57.75 67.00 9.25

Consideration 68.88 47.00 21.88

Openness 71.00 63.55 7.45

Status Motivation 66.00 50.00 16.00

Self-protection 48.50 45.36 3.14

Achievement Drive 69.00 38.91 30.09

Achieving Dimensions

Intensity 73.63 52.91 20.72

Assertiveness 74.13 52.91 21.22

Risk Tolerance 25.75 53.82 28.07*

Adaptability 47.25 36.46 10.79

Decision-making 63.13 44.91 18.22

n=29

* p<.05; ** p<.01

An analysis of variance found substantive differences in REACH PROFILES trait-based constructs between

higher and lower performers.

For purposes of comparing performance differences based on RQ score, the sample was divided into two

groups: above and below the average RQ score of 3.62. The table below reflects differences in sales

performance based on the associates’ RQ scores.

Page 62: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 62

Table 39 – Comparative analysis: sales outcomes by REACH PROFILES RQ scores

RQ Score

Constructs

Above Below Mean

Mean

Difference

90-day Average Sales $22,022 $19,986 $2,036

n=26

* p<.05; ** p<.01

Route sales performance differed substantially based on RQ scores. Specifically, associates with above average

RQ scores earned 10% higher sales amounts during the period observed.

In summary, REACH PROFILES results indicated association with performance within the industrial tool and

equipment market. Although the small sample size may limit broader applications emerging from the analysis,

it is clear that higher RQ scores were linked to increased sales, with substantial implications for the company’s

financial performance.

Page 63: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 63

REACH PROFILES in Audio/Video Equipment Sales

An audio/video equipment company directed its sales associates to complete the REACH PROFILES as part of a

criterion validation effort. The company provided performance outcomes for each associate, based on a 90-

day average sales performance. The data were merged with associates’ REACH PROFILES responses.

Researchers analyzed potential associations between performance and REACH PROFILES results, including

both skill-based constructs (RQ and aligned competencies) and trait-based constructs (primary factors and

aligned dimensions). The tables that follow reflect linear correlation and variance statistics for REACH

PROFILES constructs and sales performance.

Table 40 - Correlation matrix: REACH PROFILES skill-based constructs and sales performance

Constructs

Sales Amount

Profit Amount

Percent

Reach Quotient (RQ) 0.28 0.28 0.28

Counseling Competencies Cluster 0.10 0.08 0.02

Assimilating New Members 0.05 0.03 0.04

Cultivating Team Spirit 0.02 0.01 0.09

Identifying Personal Needs 0.17 0.14 -0.04

Recognizing Others’ Efforts 0.09 0.10 -0.04

Coaching Competencies Cluster 0.13 0.13 0.22

Building Rapport 0.09 0.09 0.18

Easing Tensions Among Members -0.12 -0.13 -0.01

Finding Opportunities For Synergy 0.32 0.31 0.30

Rallying Others Around A Cause 0.11 0.12 0.17

Driving Competencies Cluster 0.20 0.23 0.07

Setting Clear Expectations -0.29 -0.31 -0.28

Evaluating Individual Performance 0.33 0.35 0.02

Controlling Processes 0.07 0.11 0.11

Approaching Complex Issues 0.47 0.50 0.32

Advising Competencies Cluster 0.33 0.30 0.39

Addressing Quality Concerns 0.02 0.03 0.39

Aligning Resources 0.52* 0.49 0.08

Designing Team Structure -0.03 -0.06 0.03

Integrating Diverse Perspectives 0.41 0.37 0.45

n=15

* p<.05; ** p<.01

Page 64: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 64

Correlation statistics indicated a significant linear association between sales performance and the RQ

competency of Aligning Resources with Needs. Overall, the Advising Competencies Cluster appeared to offer

the most consistent association with sales performance.

Table 41 - Correlation matrix: REACH PROFILES skill-based constructs and sales performance

Constructs

Sales Amount

Profit Amount

Percent

Relational Drive -0.06 -0.09 -0.51*

Relating Dimensions

Affiliation -0.26 -0.28 -0.40

Consideration -0.10 -0.12 -0.46

Openness -0.10 -0.11 -0.37

Status Motivation -0.38 -0.36 -0.04

Self-protection 0.03 -0.01 -0.42

Achievement Drive 0.18 0.15 0.32

Achieving Dimensions

Intensity 0.43 0.38 0.14

Assertiveness 0.26 0.24 0.40

Risk Tolerance 0.18 0.14 0.34

Adaptability -0.44 -0.41 -0.47

Decision-making -0.11 -0.16 -0.09

n=150

* p<.05; ** p<.01

Correlation statistics indicated a linear association between sales performance and the primary factor,

Relational Drive. This may suggest the sales cycle places greater emphasis on independent, task-focused work

rather than cultivating interpersonal warmth and familiarity with prospective buyers. Although not statistically

significant, the results may suggest that associates exhibiting confidence, intensity and consistency in their

approach may tend to close higher sales compared to their peers. Finally, there may be a link between

competitiveness and sales performance as well, suggesting potential implications for the efficacy of incentives

and recognition programs.

For purposes of comparing performance differences based on RQ score, the sample was divided into two

groups: above and below the average RQ score of 3.62. The following table reflects practical differences in

sales performance based on the associates’ RQ scores.

Page 65: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 65

Table 42 – Comparative analysis: job performance by REACH PROFILES scores

RQ Score

Constructs

Above Below Mean

Mean

Difference

90-day Average Sales $58,748 $30,235 $28,513

90-day Average Profit $7,282 $3,939 $3,343

90-day Average Percent 11.65% 10.76% 0.89%

n=15

* p<.05; ** p<.01

Retail sales performance differed substantially based on RQ scores. Specifically, associates with above average

RQ scores earned 94% higher sales amounts during the period observed.

In summary, REACH PROFILES results indicated association with sales performance within the audio/visual

equipment market. Although the small sample size may limit broader applications emerging from the analysis,

it is clear that higher RQ scores were linked to increased sales, with substantive implications for the

company’s financial performance.

Page 66: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 66

REACH PROFILES in Retail Sales

A jewelry company directed its retail store associates to complete the REACH PROFILES (shortform) as part of

a criterion validation effort. The company provided performance outcomes for each associate, which were

merged with associates’ REACH PROFILES responses. Researchers analyzed potential associations between

performance and REACH PROFILES results, including both skill-based constructs (RQ and aligned

competencies) and trait-based constructs (the short-form REACH PROFILES only provides measures of two

primary drive factors). The tables that follow reflect linear correlation and variance statistics for REACH

PROFILES constructs and retail performance.

Table 43 - Correlation matrix: REACH PROFILES skill-based constructs and retail performance

Skill-based Constructs

Sales

Quantity Sales

Amount Appraisal

Ranking

Reach Quotient (RQ) 0.02 0.03 0.24

Counseling Competencies Cluster -0.07 -0.09 0.05

Assimilating New Members 0.06 0.08 0.20

Cultivating Team Spirit -0.00 -0.06 0.01

Identifying Personal Needs -0.17 -0.20 -0.12

Recognizing Others’ Efforts -0.09 -0.09 0.09

Coaching Competencies Cluster -0.02 -0.03 0.16

Building Rapport 0.05 0.08 0.21

Easing Tensions Among Members -0.04 -0.04 0.16

Finding Opportunities For Synergy -0.08 -0.12 0.12

Rallying Others Around A Cause 0.00 -0.04 0.07

Driving Competencies Cluster 0.06 0.09 0.31

Setting Clear Expectations 0.10 0.11 0.25

Evaluating Individual Performance 0.22 0.27 0.24

Controlling Processes -0.25 -0.22 0.19

Approaching Complex Issues 0.12 0.12 0.30

Advising Competencies Cluster 0.11 0.12 0.35

Addressing Quality Concerns 0.15 0.16 0.19

Aligning Resources 0.01 0.06 0.29

Designing Team Structure 0.14 0.12 0.34

Integrating Diverse Perspectives 0.05 0.05 0.30

n=28

* p<.05; ** p<.01

Page 67: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 67

Correlation statistics did not indicate statistically significant linear associations between retail jewelry sales

performance and the skill-based constructs measured by the REACH PROFILES.

Page 68: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 68

Table 44 - Correlation matrix: REACH PROFILES trait-based constructs and retail performance

Trait-based Constructs

Sales

Quantity Sales

Amount Appraisal

Ranking

Relational Drive -0.12 -0.20 -0.12

Achievement Drive 0.23 0.08 -0.46*

n=28

* p<.05; ** p<.01

Correlation statistics indicated a significant association between retail sales performance and the primary

factor of Achievement Drive. This finding may suggest that higher sales potential is linked to the associates’

use of a more methodical (rather than urgent) approach to selling jewelry in a retail environment.

For purposes of comparing performance differences based on RQ score, the sample was divided into two

groups: above and below the average RQ score of 3.62. The table below reflects practical differences in sales

performance based on the associates’ RQ scores.

Table 45 – Comparative analysis: REACH PROFILES scores and retail jewelry sales performance

RQ Score

Retail Sales Performance

Above Below Mean

Mean

Difference

Sales Quantity 2,268 1,686 582

Sales Amount $205,406 $153,562 $51,844

n=28

* p<.05; ** p<.01

Retail sales performance differed substantially based on RQ scores. Specifically, associates with above average

RQ scores closed 34% more sales during the period observed.

In addition to objective sales performance data, the company provided a subjective performance ranking for

the analysis. The subjective ranking was derived from senior managers’ appraisals of associates’ performance.

Such appraisals reflected the entire job domain, encompassing both sales and non-sales job functions. The

following table indicates that differences in REACH PROFILES results, including both skill-based and trait-based

constructs, may be linked to the performance ranking.

Page 69: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 69

Table 46 – Comparative analysis: REACH PROFILES skill-based constructs by performance rank

Constructs

Top Rank (n=8)

Average

Rank (n=20)

Difference

Reach Quotient (RQ) 3.58 3.19 0.39

Counseling Competencies Cluster 3.69 3.60 0.09

Assimilating New Members 4.00 3.60 0.40

Cultivating Team Spirit 3.63 3.60 0.03

Identifying Personal Needs 3.13 3.40 -0.27

Recognizing Others’ Efforts 4.00 3.80 0.20

Coaching Competencies Cluster 3.63 3.30 0.33

Building Rapport 3.88 3.35 0.53

Easing Tensions Among Members 3.63 3.25 0.38

Finding Opportunities For Synergy 3.38 3.35 0.03

Rallying Others Around A Cause 3.63 3.45 0.18

Driving Competencies Cluster 3.38 2.88 0.50

Setting Clear Expectations 3.50 3.00 0.50

Evaluating Individual Performance 3.50 2.90 0.60

Controlling Processes 3.38 2.95 0.43

Approaching Complex Issues 3.13 2.65 0.48*

Advising Competencies Cluster 3.56 2.93 0.63*

Addressing Quality Concerns 3.38 2.95 0.43

Aligning Resources 3.63 2.95 0.68

Designing Team Structure 3.63 2.95 0.68*

Integrating Diverse Perspectives 3.63 3.00 0.63

n=28

* p<.05; ** p<.01

When associates were grouped according to the subjective performance ranking, significant differences were

observed in REACH PROFILES skill-based constructs. The most pronounced differences were indicated in the

Driving and Advising Competencies

Clusters, although the top-ranked group indicated higher scores across all clusters.

Similarly, the top-ranked group demonstrated higher RQ in 15 of the 16 competencies.

Page 70: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 70

Table 47 - Comparative analysis: REACH PROFILES trait-based constructs by performance rank

Constructs

Top Rank (n=8)

Average

Rank (n=20)

Difference

Relational Drive 61.38 68.10 -6.72

Achievement Drive 26.75 52.65 -25.90

n=28

* p<.05; ** p<.01

Associates in the top-ranked group appeared to demonstrate a more methodical (rather than urgent)

approach to the sales process. This finding was supported by the correlation analysis, which indicated a

significant correlation between performance ranking and Achievement Drive.

In summary, REACH PROFILES results indicated association with certain aspects of job performance in the

retail jewelry store setting. Although the small sample size may limit broader applications emerging from the

analysis, it is clear that higher RQ scores were linked to performance as appraised by senior managers. In

addition, although not statistically significant, the differences in sales outcomes based on REACH PROFILES

results were practically significant and worthy of note.

Page 71: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 71

Summary of Criterion-related Validity Findings

Through numerous studies involving diverse workplace settings, the REACH PROFILES and its predecessor, the

LDP, demonstrated ample evidence of criterion-related validity (Malan-Rush, Cobanoglu, and Waldo, 2015;

Malan-Rush, Cobanoglu, and Waldo,

2016; Malan-Rush, Cobanoglu, Waldo, and Yang, 2014; Malan-Rush and Waldo, 2015;

Malan-Rush, Waldo, Wharton, Cobanoglu, and Yang, 2015; Waldo, Malan-Rush, Wharton, 2014; Waldo,

Malan-Rush, Wharton, and Cobanglu, 2014; Waldo, Wharton, and Parks, 2014; Wharton, Littlefield, and

Waldo, 2015). The accumulated evidence supports the utilization of the REACH PROFILES for a variety of

common workplace activities, including recruitment, selection, training, development, coaching and

succession planning. In such cases where a job analysis has determined that one or more of the REACH

PROFILES dimensions is job-related, a local validation effort may yield additional evidence to support

utilization in specific pre-employment selection processes. While the findings presented in this report were

localized in nature (meaning, they were focused on a specific employer and/or workplace setting), the

accumulation of evidence may be generalizable to related job types – with a particular focus on leadership,

management, sales and customer support roles.

Page 72: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 72

Transformation of Raw Scores

The REACH PROFILES is administered in five segments, each presenting 19 items. Participants are asked to

answer each item as mostly true or mostly false, based on their most common preferences and tendencies.

Upon completion, participants’ responses are collected as raw scores, representing the number of times their

answers reflected one of two styles on each construct. The accumulation of answers for each construct then

indicates more agreement with the left label or right label for that particular continuum.

For example, there are eight items utilized to derive the Decision-making dimension. On the left side of the

continuum underlying Decision-making is a more analytical style, while on the right side is a more intuitive

style. If a participant answers in the direction of being more analytical on seven out of the eight items, their

score on Decision making would reflect the more analytical style (a raw score would be one out of eight).

Figure 5 - Dimension Continuum Plot Example from the Leading Profile Report

Scores are depicted as a plot along a line that represents the continuum for each construct. To arrive at a

comparative plot, raw scores are converted to percentile scores based on the percentage of the normative

population (including more than 50,000 participants) earning at or near that same raw score. In the example

above, a raw score of one on the Decision-making dimension would equate to a percentile score of 26%. This

is because 26% of the normative population scored at or below the raw score of one. The participant’s

Decision-making score would be depicted as 26% to the right of the 0th percentile on the continuum, as

depicted above. This would suggest the participant tends to prefer a more analytical style in making common

decisions, when compared to the average adult working population.

The transformation of raw scores described above is repeated for each factor and dimension immediately

upon the participant answering the final LDP question. Only percentile scores are reported to the user, as

these offer greater comparative value in light of the normative population. Raw scores are retained as the

primary measures of each construct and serve as the basis for all the research reported herein.

Page 73: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 73

User Agreement

The information and data provided in connection with the REACH PROFILES, LDP, RQ 360 and RQ Culture

Survey are not in any way designed to replace the advice of a physician, psychiatrist, psychologist or other

medical professional. The assessments and their related results provided by the publisher and/or its affiliates

do not represent medical, diagnostic or psychiatric evaluations of any kind. The publisher makes no warranty

as to the accuracy, completeness, usefulness, value or fairness of the tests, nor does the publisher guarantee

against errors, failed links, omissions or lost data on its website.

Assessment results are intended to provide general information that may be helpful in professional, team and

organizational development applications. The user assumes full responsibility, and the publisher, its members,

advisory panel and affiliates shall not be liable for, (i) use and application of assessment, (ii) the adequacy,

accuracy, interpretation or usefulness of the assessment, and (iii) the results or information developed from

the use or application of the assessment. Users are encouraged to seek qualified legal counsel regarding the

use and application of the assessments for any employment-related purpose.

Users are advised that there are federal, state, and local regulations that may cover the use of assessments

and it is the user’s responsibility to consider and comply with these regulations. Users waive any claim or

rights of recourse on account of claims against the publisher, its members, or its advisory panel, either in their

own right or on account of claims against the publisher, its members, or its advisory panel by third parties.

Users shall indemnify and hold the publisher, its members, and its advisory panel harmless against any claims,

liabilities, demands or suits of third parties.

The foregoing waiver and indemnity shall apply to any claims, rights of recourse, liability, demand or suit for

personal injury, property damage, or any other damage, loss or liability, directly or indirectly arising out of,

resulting from or in any way connected with the assessments offered by the publisher, or the use, application,

adequacy, accuracy, interpretation, usefulness, or management of the assessments, or the results or

information developed from any use or application of the assessments, and whether based on contract

obligation, tort liability (including negligence) or otherwise. In no event will the publisher, its members, or its

advisory panel be liable for any lost profits or other consequential damages, or for any claim against users by a

third party.

Page 74: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 74

References

Anastasi, A., and Urbina, S. (1997). Psychological testing (7th ed.). Upper Saddle River, NJ: Prentice Hall.

Biddle, D. (2005). Adverse impact and test validation: A practitioner’s guide to valid and defensible

employment testing. Burlington, VT: Gower.

Cascio, W. F. (1998). Applied psychology in human resource management (5th ed.). Upper Saddle River,

NJ: Prentice Hall.

Cooper, D. R., and Emory, C. W. (1995). Business research methods (5th ed.). Chicago, IL: Richard Irwin.

Johnson-Vickberg, S. M. and Christfort, K. (2017, March 1). The new science of team chemistry: Pioneers,

drivers, integrators, and guardians. Harvard Business Review. Retrieved from: https://hbr.org/product/the-

new-science-of-team-chemistry/R1702BPDF-ENG

Malan-Rush, G., Cobanoglu, C., and Waldo, R. D. (2015). The use of personality styles in designing curriculum

in hospitality schools. Journal of Human Resources in Hospitality & Tourism,14, 2.

Malan-Rush, G., Cobanoglu, C., and Waldo, R. D. (2016). The impact of personality dimensions on

managers’ sense of self-efficacy within a managerial role. Presentation at the 21st Annual Graduate

Education and Graduate Student Research Conference in Hospitality and Tourism, Philadelphia, PA.

Malan-Rush, G., Cobanoglu, C., Waldo, R. D., and Yang, W. (2014). The use of personality styles of hotel

managers in designing curriculum in hospitality schools. Presentation at the 19th Annual Graduate

Education and Graduate Student Research Conference in Hospitality and Tourism, Houston, TX.

Malan-Rush, G., Cobanoglu, C., Waldo, R. D., and Yang, W. (2014). Managing with style: an analysis of work

styles of hotel managers. Journal of Human Resources in Hospitality & Tourism, 13, 2.

Malan-Rush, G., and Waldo, R. D. (2015). The impact of learning styles on managerial job performance:

the application of a personality-based model among hotel managers. Presentation at the November,

2015 International Interdisciplinary Business-Economics Advancement Conference, Ft. Lauderdale, FL.

Malan-Rush, G., Waldo, R. D., Wharton, R., Cobanoglu, C., and Yang, W. (2015). The psychometric validation of

a generalizable sales potential profile. International Journal of Sales, Retailing & Marketing, 4, 3.

Salovey, P. and Mayer, J. D. (1990). Emotional Intelligence. Imagination, Cognition and Personality, 9, 185-

211.

Schutte, N. S., Malouff, J. M., Hall, L. E., Haggerty, D.J., Cooper, J. T., and Golden, C. J. (1998). Development

and validation of a measure of emotional intelligence. Personality and Individual Differences, 25, 167-177.

Page 75: Technical Validation Report - reachecosystem.com · The most salient findings emerging from this study include: ... Reports, as described in the Principles for the Validation and

TECHNICAL VALIDATION PAPER

©2019 Leading Psychometric Solutions, LLC Page | 75

Society for Industrial and Organizational Psychology (2003). Principles for the validation and use of

personnel selection procedures. Bowling Green: Society for Industrial and Organizational Psychology, Inc.

Available from: http://www.siop.org/_principles/principles.pdf

United States Department of Labor Employment and Training Administration (1999).

Testing and assessment: An employer’s guide to good practices (data file).

Washington, DC: US Department of Labor. Available from:

https://wdr.doleta.gov/opr/fulltext/document.cfm?docn=6032

United States Equal Employment Opportunity Commission (1978). Uniform guidelines on employee

selection procedures. Washington, DC: EEOC. Available from:

http://www.uniformguidelines.com/uniformguidelines.html

Vecchione, M., Alessandri, G., Barbaranelli, C., and Caprara, G. (2011). Higher-order factors of the big five and

basic values: Empirical and theoretical relations. British Journal of Psychology, 102, 478-498.

Waldo, R. D., Malan-Rush, G., Wharton, R. B. (2014). Cultural perceptions of managerial style: is the perfect

“high-high” manager an American stereotype? Swiss Journal of Research in Business and Social

Sciences, 1, 3.

Waldo, R. D., Malan-Rush, G., Wharton, R., and Cobanoglu, C. (2014). Cultural perceptions of managerial

style: is the perfect “high-high” manager an American stereotype? Presentation at the May, 2014 Global

Interdisciplinary BusinessEconomics Advancement Conference, Tampa, FL.

Waldo, R. D., Wharton, R. B., and Parks, K. M. (2014). Achieving and relating:

validation of a two-factor model for managerial orientation. International Journal of Business and Social

Science, 5, 6.

Wharton, R., Littlefield, E., and Waldo, R. D. (2015). An analysis of work profiles across career paths:

managers and students. Presentation at the March, 2015 Southwestern Society of Economists Conference,

Houston, TX.