Transcript 2014 Annual Conference - Construction Industry Institute

Quantitative Measurement of Project Manager
Competencies
Research Team 306
Implementation Session
2014 CII Annual Conference
July 21–23 • Indianapolis, Indiana
Research Team 306
Quantitative Assessment of Project Manager Competencies
Academic Team Members
Jose A Buitrago, Jr, CSA Group
Awad S. Hanna, University of Wisconsin- Madison
Richard D. Carrithers, Jr. , DTE Energy
Jeffrey S. Russell, University of WisconsinMadison
George Carter, Anheuser-Busch InBev
Carol C. Menassa, University of Michigan
Alexandra E. Beletic, University of WisconsinMadison
Industry Team Members
Chair: Richard D. Bradford, Bechtel Group, Inc
Vice Chair: Jeffery L. Moore, Ameren Missouri
Dale C. Allen, Shell Global Solutions
Robert Bryant, BP Refining & Logistics
Technology
Jeff Drinkard, Hargrove Engineers + Constructors
Ian D. Etzkin, URS Corporation
Roque Francisquez, SABIC - Saudi Basic Industries
Corporation
Bryan Hoerner, Audubon Engineering Company, LP
Richard A. Krout, Walbridge
Rita M. Norat, Air Products and Chemicals, Inc.
Nancy Skinkle, Architect of the Capitol
Frank Williamson, Wood Group Mustang
Agenda
• Research Summary
– Literature Review
– Data Collection and Analysis
– Tool Development and Validation
• Tool Walk Through
• Panel Discussion
• Questions/Answers
Research Objectives
The primary purpose of RT306’s research was to develop a
consistent methodology to define and quantitatively measure
the competencies of Project Managers (PMs) in order to:
– Identify PM competency gaps
– Focus PM training needs and development
– Develop and validate an implementable and useable Project
Manager Competency Assessment Tool (PM CAT)
PM CAT
(Project Manager Competency Assessment Tool)
A user-friendly tool designed to quantitatively assess and score PMs’ competencies.
1.
Provides a thorough list of competencies and definitions that a PM should
possess to successfully lead, manage and execute a project
2.
Assess the competencies of a PM and compares him/her to the industry
3.
Improves PM effectiveness by targeting specific development needs to
enhance their competencies in a personalized and more efficient manner
4.
Enhance the selection process of qualified or potential PMs
Background
• PMs must have the technical and leadership expertise, training, and
characteristics to bring about a successful realization of project goals
• Should continuously improve their skills through development and training
• Historically been evaluated by qualitative means
• CII commissioned RT-281 – Project Management Skills for the Future in 2010
– Anticipate the PM competencies to respond to industry changes and future challenges
– Served a s a platform for RT-306’s research
Research Methodology
Literature Review
• Industry-wide need for a simple and quantitative PM Competency Assessment
Tool
• Existing PM competency models
• Discover, include, define and properly weight all critical PM competencies
• Organized into four competency groups:
– Knowledge/Experience (altered from RT-281’s Technical/Virtual),
– Management
– Cognitive (& Attributes)
– Leadership
Existing Competency Model examples….
NASA Development Framework Leadership Model:
Management (NASA 2008)
Corporate Education Group Project Management
Competency Model (CEG 2008)
RT-306
List of PM
Competencies
Data Collection: Two Surveys
Preliminary Survey
Detailed Survey
• Objective of survey:
• Objective of survey:
– Eliminate unimportant
competencies
– Test competencies on industry PMs
– Assign competency weights
– Determine industry performance levels and
benchmarks
• Administered to RT-306 Members
• 5-point scale
– Ensure each competency is statistically significant
• Administered to PM Supervisors
– asked to answer about an average PM and a “stellar”
excellent PM
• Questions were developed with the team and
professionally edited to pinpoint the meaning of
each competency
Competency Weights
(from first survey)
Knowledge/Experience (KE)
Awareness of and knowledge to use of state of the art technology
Business/Financial Acumen
Certification/Training
Continuously Monitors and is Aware of Similar EPC projects
Disciplinary Understanding of all Phases of EPC Projects and their Interrelationship
Disciplinary Understanding of a PM Job
Management (M)
Ability to Look Ahead and Plan
Build Knowledge Network
Business Developments/Ability to Sell
Communications Management
Focus on Client's Needs
Human Resource Management
Integration Management
Internal and External relations
Issues Management/Conflict Resolution
Knowledge and Management of Legal Issues
Organizational Savvy
Organize Project Staff and Process Expertise
Project Controls
Leadership in Safety
Risk Management
Quality Management
22.1%
13.8%
18.6%
14.0%
13.5%
20.1%
20.1%
25.0%
6.4%
6.0%
6.3%
6.5%
6.9%
5.3%
5.7%
6.3%
6.5%
5.5%
6.3%
6.9%
7.0%
6.8%
5.9%
5.7%
Cognitive (CO)
Achievement and Action
Analytical Thinking
Courage
Impact and Influence
Initiative
Personal Effectiveness
Self-Awarness
Strategic Thinking
Vision
Attributes (CO-A)
Assertive/Aggressive/ Result Driven/ Decisive
Adaptable/Approachable/Attentively Listens to Others/Easily Relates to Others
Detail Oriented/ Organized
Energetic and Enthusiastic
Credible/ Honesty/ Ethics/ Integrity/ Trustworthy/ Loyalty/ Empathetic
Mature/ Possesses Self-Control Skills/Professional
Positive Attitude/ Selflessness
Accountable/ Responsible/ Reliable
Leadership (L)
Build Coalitions within Project Team
Build Consensus
Build Trust
Cultural and Ethnic Sensitivity/ Leverages Diverse Thinking
Develop and Mentor Others
Directness/Influential
Engages Others
Innovation
Possess Strategic Insight
Team Builder
23.6%
11.2%
10.8%
10.7%
11.2%
12.0%
10.7%
11.0%
11.5%
11.2%
11.4%
12.4%
12.5%
12.6%
13.4%
12.8%
12.4%
12.5%
29.3%
10.5%
9.2%
11.1%
9.8%
9.2%
9.7%
10.1%
9.3%
10.0%
11.1%
Data Collected
Respondent and Company Information
• 77 PM Supervisors from (at minimum) 29 companies
– 154 PMs were analyzed
Company Type
Company Projects
Architect
12%
Engineer
15%
Contractor
31%
Owner
42%
Private
47%
Public
53%
•
Company Size
Company Locations:
• United States (92%)
• International (8%)
[28%] < 250 Million
8%
28%
28%
[12%] 250 Million - 1
Billion
[24%] 1 Billion - 25 billion
12%
24%
[28%] 25 billion-50 billion
[8%] > 50 billion
Excellent versus Average Project Manager
Excellent PM
Total Number of Years working as PM
Number of Years Working as a PM for this Company
13.1
Average PM
11.3
6.8
6.6
Total number of Years of Relevant Previous Experience
15.2
16.8
Number of Projects completed as a PM
42.8
21.8
Age
46.6
46.8
Data Analysis
• Paired t-test
–
Tested if excellent PM scores were significantly
different then the average PM scores
• All competencies (except for one*) were
statistically different and therefore necessary to
quantifiably measure a successful PM
• PM scores were calculated with weights from
first survey
–
to make the industry distribution
–
to create benchmarks for the PM CAT
• Formula to calculate PM scores:
• Used in the PM CAT
–
where j denotes Project Manger
–
wi is the weight of the category
–
sij is the composite score of competency j for category i
(this is the individual question score times the competency
weight)
*The one exception, where there was no statistical difference between average and excellent PMs, was the attribute
“Credible/Honesty/Ethics/Integrity/Trustworthy/ Loyalty/ Empathetic”.
Formula Further Broken Down
1. Overall Formula
Scores were
normalized at each
step!
2. Summed Categories
3. Summed Competencies within the Category (only Knowledge and Experience shown)
Tool Description
• Reliably and consistently measure PM competencies
• Excel 2007 (v 12) or later versions
• Rates PM’s competence on an industry-based scale
• Identifies training and development opportunities (competency gaps)
• Helps in the selection or placement of PMs
• Should be completed by the PM's Supervisor
• Questions are job specific and relate to any phase of EPC projects
• PM CAT can be administered at any time during PM’s career
– can be administered multiple times on one person over time to see progression
• Should not be confused with or substituted for a performance rating system
Tool Validation: Beta Testing
• 14 PM supervisor participants
• 93% accuracy
– tool is suitable for use
– of the limited overseas sample (3
respondents): 100% accuracy
• Respondents said
– the tool overall was useful to somewhat
useful
• 2.7 on a scale of 5
– user friendly to very user friendly
• 3.35 on a scale of 5
• Asked if the research objectives of the tool were
met: identity competency gaps, focus training and
development, and develop and validate
implementable/useable tool.
– Yes: 10
– Maybe: 4
– No: 0
• Asked if they would use the tool in their company
– Yes: 9
–
Maybe: 2
–
No: 3
• Tool enhancements done based on feedback
received
The deployment of the PM CAT will help enhance overall
project performance by improving the process by which
Project Managers are developed and selected.
This, in turn, will contribute to CII’s mission to improve
overall industry performance
Tool Input
Questions focus on the competencies that a PM should possess to successfully lead, manage & execute a project
Tool Output
• PM Competence Percentile
– Rates PM’s competence on an industry-based scale
– Illustrated on a graph against industry benchmarks
• Detailed Competency Report
– Highlights PM’s strong and weak competencies
– Opportunities for development and training (competency gaps)
Percentile Score and Graph
Detailed Competency Results
Tool Walk Through…
Panel Discussion
Research Objectives
The primary purpose of RT306’s research was to develop a
consistent methodology to define and quantitatively measure the
competencies of PMs in order to:
– Identify PM competency gaps
– Focus PM selection, training needs and development
– Develop and validate an implementable and useable PM CAT
Question and Answers