What Do Psychometric Results Really Mean?

Mirella De Civita Ph.D., PCC, MCEC President of Grand Heron International

Coaches Beware!

In the winter of 2012, Papillon MDC Inc. was asked by a corporate international client to review the competencies related to being an “Individual Contributor” and a “People Leader” identified by a third party and to determine how to integrate these competencies in our Selection and/or Development Assessment process. At the time, we had provided our client with a comprehensive report, delineating the pros and cons of the assessment practice in general, and argued for a holistic evaluation. We knew then, as is true even today, that our Leadership Assessment reports written by licensed psychologists would be used by hiring managers and most likely shared with an executive coach selected by the individual.

 In this month’s Guest Coach Blog, we take a look at “Executive Coaching and Assessments” from different perspectives. In the spirit of stimulating dialogue, allow me to share an important caveat when it comes to psychometric testing as well as underscore the importance of considering how the client makes sense of their own results. This is key to developing a trusting and collaborative relationship, which helps our clients move with confidence toward their desired state.

Relative or Absolute Comparisons

In the area of assessment, the decision to classify people into categories on a specific behavioral competence (e.g., Strategic Thinking) is often based on the assumption that there exists a psychometrically-sound tool that allows for accuracy in the assessment of the known competence. This also requires making an a priori decision to use either a relative or an absolute judgement for comparison purposes. For instance, is the person deemed to be in the “Management Exemplar” category of “Strategic Thinking” compared to a universal group of all persons similar to him/her (absolute judgement) or to a specific group of persons similar to him/her in the organization of interest (relative judgement)? Relatedly, the decision to accurately classify people into these categories also requires having a tool that is developed and tested (to meet all psychometric properties such as predictive validity, construct validity, concurrent validity, content validity, reliability, etc.) for the purpose of discriminating and accurately predicting dichotomous group membership based on a single variable or a linear combination of the interval variables. For example, if we want to determine whether person X fits in either the Management Exemplar; Management in progress; or Not yet Management ready category of the “Strategic Thinking” competence, we would need to find a way to determine his/her scores on the following variables that we decide may underlie this competence: (1) Conveys an understanding of the organization’s marketing position relative to competitors…; (2) Integrates and balances big-picture ideas…; (3) Leverages fresh perspectives…; and (5) Evaluates and pursues initiatives, investments. The scores we determine would then be compared to the discriminant function analysis output we generated with a representative group, depending on whether a relative or absolute group comparison was applied. Evidently, it is not as straightforward as it would seem to pass judgement on a client based on his/her score on any given competence.

More than the Sum of Their Parts 

People are far more complex than any one single competence. No single competence ought to be perceived as operating independently of the others. Strategic Thinking, for instance, is influenced by other elements of character (such as interpersonal, socio-relational, emotional, motivational needs, etc.) that define the uniqueness of the individual. Moreover, an individual’s success is often a combination of, and an interaction between, unique personal style and environmental context. When we classify individuals in categories on any specific behavioral competence, the role of the environment is absent. Yet, we know that people will react differently depending on environmental conditions. This means that even a client who scored high on the Strategic Thinking competence may not demonstrate all of what is expected when debating a given issue with people who stifle new ideas and discourage risk-taking. In fact, considering competencies independent of how they interact with one another and in the absence of environmental factors may lead clients to feel wholly responsible for not succeeding.

As coaches, we have a role to play in educating our clients as well as their corporate sponsors when it comes to accurately interpreting the results of psychometric tests that focus solely on a set of given competencies. We need to take a holistic perspective and aim to provide a thorough understanding of how the individual’s cognitive, emotional, and behavioral components interact to create a unique profile. And, we need to remember the importance of asking the client how he/she is making sense of the results. In the end, the relationship you hold with the client will not be based on trust when their perspective is not integrated in your interpretation of the results. Your aim is to grasp the essence of what is most important to your client while keeping in mind the relative vs. absolute caveat inherent in every test, and the uniqueness of your client’s inherent profile.

Coming Up Next…

Next up, guest coach Dr. Stacy Starkka shares with us her practice of working with assessments and the important role they play in the coaching conversation. As a Consulting Industrial-Organizational Psychologist, she uses assessment results as a guide into gaining a deeper appreciation of her clients, and what shows up for them as being important in their own journey forward. Stay tuned!

Learn more about Mirella De Civita

Want to be a featured Guest Coach Blogger? Send an email to Miranda Piccolino – miranda@ghi.email 

Scroll to Top