Transcript CMM vs. ISO David S. Craft CIRM, PMP 11 April 2007

CMM vs. ISO
David S. Craft CIRM, PMP
11 April 2007
Agenda
Who Am I
Software Systems Development
ISO
CMM
Sarbanes Oxley
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Who Am I
Managing Consultant, Engineering and Manufacturing Services
Inventory Control Manager
Shift Supervisor
Internal ISO Auditor
Team Leader
Industrial Engineer
Consultant
Materials Manager
VISTA Volunteer
Manager Production Planning & Control
Chief Industrial Engineer
CMM vs. ISO, Sarbanes Oxley
Project Manager
11 April 2007
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Process
To Develop Software and Systems You Need A Process
• Anything goes
• Defined
• Structured
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Process, people and technology are the major determinants of project cost,
quality and schedule.
CMM vs. ISO, Sarbanes Oxley
11 April 2007
11 April 2007
CMM vs. ISO, Sarbanes Oxley
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Common Misconceptions
I don’t need defined processes I have:
–Really good people
–Advanced Technology
–An experienced manager
Defined Processes:
–Interfere with creativity
–Equals bureaucracy + regimentation
–Isn’t needed when building prototypes
–Is only useful on large projects
–Hinders agility in fast moving projects
–Costs too much
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Why We Need Standard Processes
Estimating (History)
•
Scope
•
Cost
•
Time
•
Tools
Deliver the Product to Estimate (Visibility)
•
Time
•
Cost
•
Quality
Handling/Controlling Changes
CMM vs. ISO, Sarbanes Oxley
•
Planned
•
Unplanned
•
Scope Creep
11 April 2007
How to Achieve Quality Processes
ISO
CMM
11 April 2007
CMM vs. ISO, Sarbanes Oxley
ISO – CMM Differences
ISO9001:2000
CMMI-DEV
International standard, applies to all
types of organizations, supports both
product and service oriented
organizations
Written specifically for software
development companies
A brief document – about 25 pages
long, identifying the minimal
requirements for a quality system
A detailed document – over 500 pages
long
Emphasizes on a management of
continuous improvement process,
based on the PDCA (Plan-Do-CheckAct) model
Emphasizes on achieving “maturity”
and improving its process continuously
One level of standard. The standard is
based on recommendation
Defines 5 maturity levels of the
organization, covering 25 process
areas (PAs)
11 April 2007
Netta Dotan, Quality Assurance & project management, Ronkal Office Technologies
CMM vs. ISO, Sarbanes Oxley
ISO – CMM Differences – My View
ISO 9000
SW-CMMI
Outwardly focused
Inwardly focused
Minimum requirements with
implied continuous
improvements
Explicit continuous quality
improvement
Registration Document
No documentation
Certification audit for a 50
employee organization will be
executed by 1 -12 auditors
during one day
Certification audit for a 50
employee organization will be
executed by 4 auditors during
4-5 days
11 April 2007
Netta Dotan, Quality Assurance & project management, Ronkal Office Technologies
CMM vs. ISO, Sarbanes Oxley
ISO – CMM Similarities
Both require the organization be explicit about what their processes and
quality systems are
Say what you do; do what you say
The organization records and tracks data for objective analysis
Require strong management support to succeed
Provide a structured and measured approach to quality improvement
Require an outside audit for “certification”
Both are refined/improved over time
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Meet ISO
The International Organization for Standardization (ISO) is a worldwide
federation of national standards bodies from some 162 countries,
representing approximately 95% of worldwide production. ISO is a nongovernmental organization established in 1947 to promote the
development of standardization and related activities in the world with a
view to facilitating international exchange of goods and services and
development of cooperation in the spheres of intellectual, scientific,
technological and economic activity
ISO (International Organization for Standardization) is the world's largest
developer and publisher of International Standards.
ISO is a non-governmental organization that forms a bridge between the
public and private sectors. On the one hand, many of its member institutes
are part of the governmental structure of their countries, or are mandated
by their government. On the other hand, other members have their roots
uniquely in the private sector, having been set up by national partnerships
of industry associations. Therefore, ISO enables a consensus to be
reached on solutions that meet both the requirements of business and the
11 April 2007
broader needs of society.
CMM vs. ISO, Sarbanes Oxley
ISO’s Impact
In the global economy
ISO 9001:2000 and ISO 14001:2004 have become thoroughly integrated with
the world economy.
ISO 9001:2000 is now firmly established as the globally accepted standard for
providing assurance about the quality of goods and services in suppliercustomer relations.
The positive roles played in globalization by ISO’s standards for quality and
environmental management systems include the following:
•
a unifying base for global businesses and supply chains – such as the
automotive and oil and gas sectors
•
a technical support for regulation – as, for example, in the medical
devices sector
•
a tool for major new economic players to increase their participation in
global supply chains, in export trade and in business process
outsourcing;
•
a tool for regional integration – as shown by their adoption by new or
potential members of the European Union
In the rise of services in the global economy – nearly 33 % of ISO 9001:2000
certificates in 2005 went to organizations in the service sectors.
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Where are the Standards (12/31/09)
Sector
Standards
Generalities, Infrastructure and Sciences
Pages
1,601
64,568
734
29,491
Engineering Technologies
4,937
223,394
Electronics, Information Technology and
Telecommunications
2,902
506,057
Transport and Distribution of Goods
1,957
55,646
Agriculture and Food Technology
1,054
26,286
Materials Technology
4,373
114,269
Construction
380
14,632
Special Technologies
145
3,602
18,083
737,345
Health, Safety and Environment
Total
11 April 2007
CMM vs. ISO, Sarbanes Oxley
What are standards?
Standards are documented agreements containing technical specifications
or other precise criteria to be used consistently as rules, guidelines, or
definitions of characteristics, to ensure that materials, products, processes
and services are fit for their purpose.
For example, the format of the credit cards, phone cards, and "smart" cards
that have become commonplace is derived from an ISO International
Standard. Adhering to the standard, which defines such features as an
optimal thickness (0,76 mm), means that the cards can be used worldwide.
International Standards thus contribute to making life simpler, and to
increasing the reliability and effectiveness of the goods and services we
use.
Last modified 2002-07-17
11 April 2007
CMM vs. ISO, Sarbanes Oxley
What standards do
ISO standards:
•
Make the development, manufacturing and supply of products and services more
efficient, safer and cleaner
•
Facilitate trade between countries and make it fairer
•
Provide governments with a technical base for health, safety and environmental
legislation, and conformity assessment
•
Share technological advances and good management practice
•
Disseminate innovation
•
Safeguard consumers, and users in general, of products and services
•
Make life simpler by providing solutions to common problems
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Which ISO Standards
ISO 9000 represents consensus on what requirements a quality system
must meet but does no dictate how they should be met.
The ISO 9000 series addresses quality management and quality
assurance standards. It is designed to assist organizations in
implementing and operating an effective quality management system
(QMS). ISO 9001 defines what quality standards should be followed.
It does not tell how.
The ISO 9000:2000 series is based on 8 key principles: Customer
Focus, Leadership, Involvement of People, Process Approach, System
Approach to Management, Continual improvement, Factual Approach
to Decision Making and Mutually Beneficial Supplier Relationships
11 April 2007
CMM vs. ISO, Sarbanes Oxley
ISO 9000 family
The ISO 9000 family addresses "Quality management".
This means what the organization does to fulfill:
•
The customer's quality requirements
•
Applicable regulatory requirements,
•
Enhance customer satisfaction,
•
Achieve continual improvement of its performance in pursuit
of these objective
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Quality System Documentation
Level 1
Quality
Manual
Defines
Approach and
Responsibility
Level 2
Procedures
Defines
Who, What, When
Level 3
Work/Job
Instructions
Answers
How
Level 4
Records/Documentation
CMM vs. ISO, Sarbanes Oxley
11 April 2007
Results: shows that
the system is
operating
ISO 9001:2000 Structure
4.
5.
6.
Quality Management System
4.1 General requirements
4.2 Document requirements
Management
Responsibility
5.1 Management
commitment
5.2 Customer focus
5.3 Quality policy
5.4 Planning
5.5 Responsibility, authority,
communication
5.6 Management review
Resource Management
6.1 Provision of resources
6.2 Human resources
6.3 Infrastructure
6.4 Work environment
CMM vs. ISO, Sarbanes Oxley
7.
Product realization
7.1 Planning of product realization
7.2 Customer-related processes
7.3 Design and development
7.4 Purchasing
7.5 Production and service provision
7.6 Control of monitoring and
measuring devices
8.
Measurement, Analysis &
Improvement
8.1 General
8.2 Monitoring and measurement
8.3 Control of nonconforming product
8.4 Analysis of data
8.5
Improvement
11 April
2007
Evaluation
ISO is a certification model. Typically, an internal quality system
assessment (audit) is performed, repairs made and the organization
may then submit to a formal system audit lasting for several days
performed by one of the ISO certification Bodies. The certificate
usually is valid for three years and also requires that a system of
Quality Management be in place, including performance of regular
internal audits and intermediate external audits.
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Meet CMMI
CMMI® (Capability Maturity Model® Integration) models are
collections of best practices that help organizations to improve their
processes. These models are developed by product teams with
members from industry, government, and the Software Engineering
Institute (SEI). These models provides a comprehensive integrated
set of guidelines for developing products and services.
The CMMI-DEV model provides guidance for applying CMMI best
practices in a development organization. Best practices in the model
focus on activities for developing quality products and services to
meet the needs of customers and end users.
Other CMMI models:
•
•
•
Acquisition
Services
People
CMM vs. ISO, Sarbanes Oxley
11 April 2007
Scope of CMMI
CMMI is designed to help identify and prioritize process improvement opportunities
and facilitate organizational change management. The model is used for internal
process improvement, sourcing selection and benchmarking, rather than certification
CMMI is organized as a process framework that cluster related practices into
process areas that, when performed collectively, satisfy a set of goals. It requires
that you define specific practices to meet specific goals but does not define how
they are to be implemented.
The CMMI provides two representations – staged and continuous. The staged view
provides five maturity levels: Initial, Managed, Defined, Quantitatively Managed,
and Optimizing and 22 process areas PAs). The PAs at each maturity level build on
the previous level. Alternatively, continuous representation is used to focus on a
process capability in a desired functional area (project management, process
management, engineering and support) rather that maturity levels.
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Evaluation
This is not a certification model, but ratings may be announced and
published. The SEI publishes ratings provided the company gives it
permission. Formal appraisals are typically 5 – 10 days and led by SEIauthorized internal or external lead appraisers, using trained teams and a
formal methods. The method is named SCAMPI (Standard CMMI
Appraisal Method for Process Improvement).
11 April 2007
CMM vs. ISO, Sarbanes Oxley
11 April 2007
CMM vs. ISO, Sarbanes Oxley
SCAMPI – Standard CMMI Appraisal Method for
Process
Improvement
11 April
2007
CMM vs. ISO, Sarbanes Oxley
Process Areas
Requirements Management
Organizational Process Definition
Project Planning
Organizational Training
Project Monitoring & Control
Integrated Project Management
Supplier Agreement Management
Risk Management
Measurement & Analysis
Integrated Teaming
Process & Product Quality
Assurance
Integrated Supplier Management
Configuration Management
Decision Analysis & Resolution
Requirements Development
Organizational Environment for
Integration
Technical Solution
Organizational Process Performance
Product Integration
Quantitative Project Management
Verification
Organizational Innovation & Deployment
Validation
Causal Analysis
11 April 2007& Resolution
Organizational Process Focus
CMM vs. ISO, Sarbanes Oxley
11 April 2007
CMM vs. ISO, Sarbanes Oxley
11 April 2007
EIA – Electronic Industries Alliance Interim Standard
CMM vs. ISO, Sarbanes Oxley
11 April 2007
CMM vs. ISO, Sarbanes Oxley
11 April 2007
CMM vs. ISO, Sarbanes Oxley
11 April 2007
CMM vs. ISO, Sarbanes Oxley
11 April 2007
CMM vs. ISO, Sarbanes Oxley
11 April 2007
CMM vs. ISO, Sarbanes Oxley
11 April 2007
CMM vs. ISO, Sarbanes Oxley
11 April 2007
CMM vs. ISO, Sarbanes Oxley
11 April 2007
CMM vs. ISO, Sarbanes Oxley
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Staged
CMM Process Areas
CMM vs. ISO, Sarbanes Oxley
Process Area
Continuous
L2
Requirements Management
Engineering
L2
Project Planning
Project Mgmt
L2
Project Monitoring and Control
Project Mgmt
L2
Supplier Agreement Management
Project Mgmt
L2
Measurement and Analysis
Support
L2
Process and Product Quality Assurance
Support
L2
Configuration Management
Support
L3
Requirements Development
Engineering
L3
Technical Solution
Engineering
L3
Product Integration
Engineering
L3
Verification
Engineering
L3
Validation
Engineering
L3
Organizational Process Focus
Process Mgmt.
L3
Organizational Process Definition
Process Mgmt.
L3
Organizational Training
Process Mgmt.
L3
Integrated Project Management
Project Mgmt
L3
Risk Management
Project Mgmt
L3
Integrated Teaming
Project Mgmt
L3
Integrated Supplier Management
Project Mgmt
L3
Decision Analysis and Resolution
Support
L3
Organizational Environment for Integration
Support
L4
Organizational Process Performance
Process Mgmt.
L4
11 April 2007
Quantitative Project
Management
Project Mgmt
L5
Organizational Innovation and Deployment
Process Mgmt.
L5
Causal Analysis and Resolution
Support
Examples of CMMI Impact: ROI
5:1 ROI for quality activities (Accenture)
13:1 ROI calculated as defects avoided per hour spent in training and
defect prevention (Northrop Grumman Defense Enterprise Systems)
Avoided $3.72 M in costs due to better cost performance (Raytheon
North Texas Software Engineering) as the organization improved
from SW-CMM level 4 to CMMI level 5
2:1 ROI over 3 years (Siemens Information Systems Ltd, India)
2.5:1 ROI over 12st year, with benefits amortized over less than 6
months (reported under non disclosure)
(reported by the American Society for Quality)
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Sarbanes-Oxley Implications
With its more than 300 discrete points of enforceable law, this is the most
significant piece of account legislation passed since the formation of the SEC in
1933
SOX was passed with the specific intent of increasing accountability and
attempting to install ethical behavior in financial reporting and business
operations.
With this increase spotlight on reporting, companies must invest resources and
focus into their internal control process
The Act created the Public Company Accounting Oversight Board (PCAOB) to
oversee the activities of the auditing profession and mandated reforms to enhance
corporate and criminal fraud accountability.
A goal of SOX legislation is to continually improve the transparency of financial
and business events that can impact the accuracy and future validity of financial
statements. Projects to improve processes and regular review of controls will
become common-place activities as compliance evolves. Tools that simplify
project completion and track status will better enable organization to costeffectively undertake these projects.
11 April 2007
CMM vs. ISO, Sarbanes Oxley
SOX Major Section
302 – Corporate Responsibility for Financial Reports
•
Requires Executives to certify the accuracy of corporate financial
reports
404 – Management Assessment of Internal Controls
•
Requires executives and auditors to confirm the effectiveness of
internal controls for financial reporting
409 – Real Time Issuers Disclose
•
Requires any material changes in financial state of issuer be
communicated quickly and with supporting data to the public
11 April 2007
CMM vs. ISO, Sarbanes Oxley
Implications for IT
Configuration management is now a must
Change controls must be handled more carefully
Security, security, security
All system changes must be verifiable by a clear audit trail
Reduce reliance on batch processing, update data warehouse more
frequently
Interfaces from any financial system must be documented and
controlled
IT activities must be aligned with the company’s governance and risk
policies
11 April 2007
CMM vs. ISO, Sarbanes Oxley