Transcript Cross-Informant Assessment: A Powerful Tool for

2010 SPA Annual Meeting
San Jose, California
Cross-Informant Assessment:
Concepts, Behavioral Science, and Applications
Marvin W. Acklin, Ph.D.
Honolulu, Hawaii
Joshua Dwire, PsyD
Grenada, The Caribbean
Cross-Informant Assessment: Concepts, Behavioral Science, and
Applications




Broad focus of this symposium is the use of
others in personality assessment
Using how people see each other as a
source of clinical information
Looking at the relationship between how
people are seen by themselves and how they
are see by others
Focus on applied clinical science
Cross-Informant Assessment: Concepts, Behavioral Science, and
Applications
Data Sources in Psychological Assessment

Current views of psychological assessment endorse the use of
multiple information sources.

Psychological assessment is concerned with a variety of test scores,
generally obtained from multiple test methods, considered in the
context of history, referral information, and observed behavior:
unstructured & structured interviews; performance-based cognitive &
personality tests, & observer rating scales (Meyer, et al, 2001)

Each information source has both strengths and constraints in the
assessment procedure.
Cross-Informant Assessment: Concepts, Behavioral Science, and
Applications
Reliance on Self-Report

Traditional diagnostic practices rely heavily on individual self report as
the main source of information in clinical assessment.

There are serious limitations in the validity of self-report, not the least
of which is the patient’s motivation to communicate frankly (selfpresentational bias).
Cross-Informant Assessment: Concepts, Behavioral Science, and
Applications
Use of Observer-Informant Information





The disciplined, empirically validated use of informants is both recent
and innovative. The use of informants is most effective using parallel
forms of items for behavior ratings.
The Achenbach System of Empirically Based Assessment (ASEBA) is
a life-span approach using self-report and Informant ratings is one
highly effective method of obtaining observer ratings.
I see you (observer rating)
I see you, you see me (Criss-cross)
I see you seeing me (metaperception; reflected appraisals)
Cross-Informant Assessment: Concepts, Behavioral Science, and
Applications
Limitations of Self-Report

Self report can be limited by defensive and self-presentational biases
(Westen & Weinberger, 2004).

“…people have minimal access to many of their cognitive processes,
and they often confabulate explanations for their behavior by applying
intuitive attributional theories…

Validity Scales are utilized to correct for self-presentational bias.

Critique of validity scales as a basis for correcting self reporting bias
(Piedmont et al, 2000).
Cross-Informant Assessment: Concepts, Behavioral Science, and
Applications

In order to overcome the drawback of solely using
self-reports in psychological assessment,
Achenbach, Dumenci, and Rescorla (2005)
suggested that cross-informed methods provide a
multifaceted approach to assessment in which
constructs for evaluation come from the same pool
of items.
Cross-Informant Assessment: Concepts, Behavioral Science, and
Applications
Concordance of Information Sources: Cross-Method and
Cross-Informant

Research demonstrates surprisingly low levels of concordance
between different information sources (Meyer, et al, 2001; Meyer,
2002).

“The substantial independence between sources of information is not a
phenomenon that is limited to rarified psychological research on
personality…” (Meyer, 2002, p. 75).

“…Campbell and Fiske (1959) noted how relative independence
among psychological methods can point to unappreciated complexity
in the phenomena under investigation.”
Cross-Informant Assessment: Concepts, Behavioral Science, and
Applications
Concordance of Information Sources: CrossInformants (Adults)

Research demonstrates surprisingly low levels of concordance
between self and other report and concordance in observer reports:
.681 substance abuse
.428 internalizing disorders
.438 externalizing disorders
.304 mean
.23 self vs. clinician
.26 self vs. aggregated peer rating
.29 self vs. spouse
Cross-Informant Assessment: Concepts, Behavioral Science, and
Applications
Adults








Self vs. spouse: Personality & mood
Self vs. spouse/partner: Big five
Self vs. peer: Personality & mood
Self versus peer: Big Five Factors
Self vs. significant other: Axes II PD diagnosis
Self vs. clinician: Axis II PD traits
Self vs. clinician: Big Five Factors
Self vs. cognitive test: Memory problems
.29
.44
.27
.31
.21
.33
.32
.13
Cross-Informant Assessment: Concepts, Behavioral Science, and
Applications
Children

Research also demonstrates surprisingly low levels
of concordance between self and other report and
concordance in observer reports ofnchildren:
.60 similar informants (e.g., parents)
.28 different informants
.22 self and other ratings
Cross-Informant Assessment: Concepts, Behavioral Science, and
Applications
Children





Self vs. parent: Behavioral & emotional problems
.29
Self vs. clinician: Behavioral & emotional problems .14
Self vs. teacher: Behavioral & emotional problems
.21
Parent vs. direct observer of child behavior:
.27
Low concordance suggests situational specificity in informant
ratings and the fundamental independence of ratings.
Cross-Informant Assessment: Concepts, Behavioral Science, and
Applications

One approach to capitalizing on the potential added
value of ratings by multiple informants is known as
“360° feedback” methodology. This methodology
entails comparing assessment data obtained from
ratings by multiple informants, as well as from selfratings. Many businesses have adopted 360o
feedback for assessing managers' job performance
by having subordinates, peers, and bosses rate the
managers' performance for comparison with the
managers' self-ratings (Brett & Atwater, 2001).
Cross-Informant Assessment: Concepts, Behavioral Science, and
Applications

Self-ratings had mean correlations of.19 with peer
ratings and.22 with supervisor ratings. And between
supervisors and peers, the mean correlation was.34.
Despite the relatively small correlations, a metaanalysis of the incremental validity of subordinate
and peer ratings led Conway, Lombardo, and
Sanders (2001) to conclude that the “results
provided justification for the added expense of
including subordinates and peers in a rating system”
Cross-Informant Assessment: Concepts, Behavioral Science, and
Applications
Cross-Informant Assessment of Personality
Disorders

Personality Disorders: “…diagnoses have only moderate
associations when they are derived from self-reports or the
reports of parents, significant others and clinicians

Self and/or Other attributional biases in personality disordered
reporters or informants may play a role in further reducing
concordance.
Cross-Informant Assessment: Concepts, Behavioral Science, and
Applications
Limitations of Observer-Reports

The constraints of observer ratings include parameters of the particular
type of relationships (e.g., spouse, coworker, therapist) and the setting
in which observations are made.

Appear to be independent sources

Others reports may be systematically biased due to a variety of
attributional biases—for example in divorcing spouses.
Cross-Informant Assessment: Concepts, Behavioral Science, and
Applications
Limitations of Observer-report

Theoretical implications of cross-method and cross-informant
methods.

Weak support for the concept of consensual reality. The low level of
concordance between sources, especially between self and observer
report threatens simplistic views of consensual reality.

On the other hand, “…using a composite judge can yield very large
validity gains when the redundancy of judges is low…” (Tsujimoto,
Hamilton, & Berger, 1990)
Cross-Informant Assessment: Concepts, Behavioral Science, and
Applications
Use of the Achenbach scales in assessment practice
The Achenbach System of Empirically Based Assessment
comprises a family of forms for assessing adaptive functioning
and problems is easy and cost-effective ways.
Child Behavior Checklist
Teacher-Caregiver Report Form
Youth Self-Report
Adult Self Report
Adult Behavior Checklist
Older Adult Self-Report
Older Adult Behavior Checklist
Cross-Informant Assessment: Concepts, Behavioral Science, and
Applications
Introduction to Symposium Papers




Cross-informant assess using the Inventory of
Interpersonal Problems (Pincus)
Cross-informed assessment of marital compatibility
(Dwire & Acklin)
Self and Informant ratings using the NEO-PIR (Kurtz)
Parent personality and infant temperament (Janson
& Narde)