Laatuominaisuuksien standardin ISO25010 ja testausarviointistandardin ISO33063 esittely Risto Nevalainen, Senior Advisor, FiSMA ry

Sisältö      Hieman FISMAsta SC7 standardoinnista WG6 ja SQUARE, ISO/IEC 25000    Standardit ja mallit, lähinnä 25010 sarja Esimerkkejä Mitä seuraavaksi WG10 ja Testing SPICE, ISO/IEC 33063    Standardit ja mallit, 330xx sarja 33063, esimerkkejä Mitä seuraavaksi Keskustelua  HUOM: esittelykalvot ovat englanniksi, helpompi esittää standardit niin ja on valmista materiaalia. Käännöstöitä ei nyt harkinnassa. FiSMA 2012 2

FiSMA organisation 2012 (changes possible) Annual meeting Board 6 members CP: Tuukka Haarni Managing Director Erkki Savioja Senior Advisor Risto Nevalainen Senior Advisor Pekka Forselius FiSMA SPIN CP: Jukka Märijärvi Testing Standards and Models CP: Heikki Uusitalo, Endero Standards and Models CP: Kimmo Vaikkola, Fujitsu IT service mgmt CP: Jyrki Lahnalahti, Inspecta Scope Manager Forum CP: Hannu Lappalainen FiSMA 2012 3

SWG 1

Business Planning Group

SWG 5

Standards Management Group

WG2

Systems & Software Documentation

WG4

Tools and Environment

WG6

Software Product Measurement and Evaluation

WG7

Life Cycle Management

JTC 1 SC7 Structure

LCPHAG SC7

Life Cycle Process Harmonization Advisory Group Secrétariat

SWG22

Vocabulary Maintenance

WG10

Process Assessment

WG19

Techniques for Specifying IT Systems

WG20

Software Engineering Body of Knowledge

WG21

Software Asset Management

WG23

Systems Quality Management

WG24

SLC Profiles and Guidelines for VSE

WG25

IT Service Management

WG26

Software Testing

WG27

IT Enabled Services (BPO)

WG28

CIF Usability

WG40

IT Governance

WG42

Architecture

ISO/IEC JTC1/SC7/WG6 Responsibility and Structure Title:  Software Product and Systems Quality Scope:  Development of Standards and Technical Reports for Software Product and System Quality Requirements, Measurement and Evaluation   Organizational Structure:  SQuaRE Series (ISO/IEC 25000 ~ 25099) => WG6 Functional Size Measures => WG6/FSM SG CIF (ISO/IEC 25060 ~ 25069) => JWG => WG28 FiSMA 2012 5

Organization of SQuaRE series of International Standards FiSMA 2012 6

Quality model division, 2501x      This division states the general requirements for a quality model, recommended model, and guides to customize and use the model.

This division consists of two parts: ISO/IEC 25010: System and software quality models   A quality in use model composed of five characteristics (11 subc.).

A product quality model composed of eight characteristics (31 subc.).

ISO/IEC 25012: Data quality model  The data quality model defined in this International Standard categorizes quality attributes into fifteen characteristics (22 views) considered by two points of view: inherent and system dependent.

New: ISO/IEC 25011 Service Quality  Professor Markku Tukiainen is nominated as co-editor by Finland FiSMA 2012 7

SQuaRE Architecture and projects

ISO/IEC 2503n: Quality Requirement Division

25030: Quality Requirements (IS)

ISO/IEC 2501n: Quality Model Division

25010: System and software quality models (IS) 25012: Data Quality Model (IS)

ISO/IEC 2500n: Product Quality General Division

25000: Guide to SQuaRE (IS) 25000 Rev.: Guide to SQuaRE (WD) 25001: Planning and Management (IS)

ISO/IEC 2504n: Quality Evaluation Division

25040: Quality Evaluation Process (IS) 25041: Evaluation Guide for Developers, Acquirers and Independent Evaluators (DIS)

ISO/IEC 2502n: Quality Measurement Division

25020: Measurement Reference Model (IS) 25021: Quality Measure Elements (TR) 25021: Quality Measure Elements (DIS) 25022: Measurement of Quality in Use (WD) 25023: Measurement of Sys & SWP Quality (WD) 25024: Measurement of Data Quality (WD) 2504n: Evaluation Module (NYA) 25045: Evaluation Module for Recoverability (IS)

ISO/IEC 25050 ~ 25099: SQuaRE Extension Division

25051: Requirements for quality of COTS software product and instructions for testing (IS) 25051 Rev.: Requirements for quality of COTS

ISO/IEC 25060 ~ 25069: Common Industry Format for Usability JWG JTC1/SC7 and ISO/TC159/SC4

software product and instructions for testing (WD) ISO/IEC JTC1/SC7/WG6 Paris Meeting Report - 2011 -05 8 FiSMA 2012 8 8

Overview of using SQuaRE Customer User Development Organization Acquirer Evaluator Developer provide guidance Product Quality Requirements determines guidance Product Quality Evaluation perform supports Product Quality Model and Measurement supports FiSMA 2012 9

Structure used for the quality models Note: also 330xx will use the same structure in near future FiSMA 2012 10

Quality in the lifecycle Process influences Software product influences influences Effect of software product Process quality Internal properties External properties Quality in use Process measures depends on Internal measures depends on External measures depends on Quality in use measures Contexts of use FiSMA 2012 11

Quality in use model in 25010 FiSMA 2012 12

Product quality model (system, software) in 25010 FiSMA 2012 13

Next steps in SQUARE    25051 ballot for renewal, CD is closing 15.10.2012

25020 series is now started, will add details in quality models by defining set of metrics per model    25020, 25021, 25022, 25023, 25024 So far 9126 is still valid Professor Markku Tukiainen is nominated as co-editor by Finland in 25022 25040 series is also under work, mainly in CD phase FiSMA 2012 14

Ohjelmistotuotteen laadun mittarit, 2 kpl FiSMA suositus  

Suositeltu metriikka: A.1 Käyttäjän työn tehostuminen

   Tyyppi: Johdettu mittari Oleellinen sisältö: Käyttäjätehtävien aste, jota ohjelmisto tukee verrattuna kaikkiin käyttäjätehtäviin. Suositeltu metodi on tapaustutkimus. Mitä mittari kertoo: Kuinka hyvin ja kattavasti ohjelmisto on toteutettu täyttämään käyttäjän tarpeita.

Suositeltu metriikka: A.2 Käyttäjän tyytyväisyys

  Tyyppi: Suora mittari Oleellinen sisältö: Käyttäjän kokema tyytyväisyys (user experience). Voidaan jakaa osa-alueisiin. Suositeltu metodi on Net Promoter.  Mitä mittari kertoo: Kuinka käyttäjä kokee ohjelmiston toimivan hänen kannaltaan, esim. käytettävyys ja esteettömyys. FiSMA 2012 15

ISO/IEC JTC1/SC7/WG10 Responsibility and Structure Title:  Process Assessment Scope:  Development of Standards and Technical Reports for Software Product and System Quality Requirements, Measurement and Evaluation   Organizational Structure:  SPICE Series, Published (ISO/IEC 15504 Parts 1 -10) SPICE Series, Under development (ISO/IEC 3300xx) Benchmarking, ISO/IEC 29155 (several parts) FiSMA 2012 16

ISO/IEC 330xx Plan FiSMA 2012 17

ISO/IEC 33063 exemplar process assessment model for software testing  One of the assessment models.  Other identified models are software, systems and services assessment model.  Started almost two years ago   WD in 2011 CD1 during winter 2012  Lot of comments, also from SYTYKE/Testaus OSY  Finland voted against: too complex, too much overlap  Current draft is CD2   Simplified structure: 8 processes Guidance on how the planning and scoping of an assessment is done with this process assessment model for software testing (Annex B)  NOTE: Also other test assessment models are developing! FiSMA 2012 18

Overall architecture FiSMA 2012 19

Processes in ISO/IEC 33063 FiSMA 2012 20

Assessment indicators FiSMA 2012 21

Example: Test Completion process (Normative part) FiSMA 2012 22

Example: Test Completion Process (practice indicators) FiSMA 2012 23

Example: Test Completion Process (work product indicators) FiSMA 2012 24

Capability indicators, capability levels in 33063    Same structure as in draft CD3 ISO/IEC 33020 ”Process measurement framework for assessment of process capability and organizational process maturity”     9 process attributes 6 capability levels, ordinal value NPLF rating for process attribute rating Additional indicators as in ISO/IEC 15504-5:2012 model (software) No maturity scale No definition or adaptation of assessment by different classes   Note: Possibility that testing (and V&V) are in future also in some other standards in 330xx family Note: Possibility that someone proposes other kind of process quality characteristics than process capability FiSMA 2012 25

Next steps in 33063  CD ballot is open, closes 21.11.2012

   FiSMA meeting is 1.11.2012 to handle comments Also comments from Testaus OSY are welcome!

Expected future: DIS ballot winter 2013(if ISO/IEC 33020 moves also to DIS), publication late 2013   Note: many other parts in 29119 are also now under ballot  29119-4 DIS ballot closes 21.11.2012

Note: NWIP proposal in topic ”Keyword driwen testing” is open, closes 22.12.2012. Any volunteer from Finland to participate?

FiSMA 2012 26

Guidance in using 33063 at more detailed level   Each process can be assessed at more detailed level  Example: Test Management process can be rated separately for Unit testing, System testing and for Reliability testing  Need for aggregation: higher level rating is combination of several lower level ratings. What is the aggregation rule? Also processes from other models can be included in the assessment scope  Example: DEV.5 Software Integration from ISO/IEC 15504:5 (2012) FiSMA 2012 27

Training for FiSMA members in testing standards during 2013  Two courses are planned for 2013, no details yet:    2-day 29119 course 2-day Testing SPICE (33063) course No policy how Testaus OSY members can participate, any ideas?

  Note: One SPICE training week is also planned, focuses on highly critical applications (Nuclear SPICE etc) Note: Safety Manager course will be offered for FiSMA members in February 2013, will include heavy practices for V&V. Automotive and industrial automation are main domains. Participation is free also for Testaus OSY. FiSMA 2012 28