COMSATS Islamabad

Lecture 17

Enterprise Systems Development ( CSC447 )

Muhammad Usman, Assistant Professor

Software Architecture Evaluation

Why Analyzing SA

Marry your architecture in haste and you can repent in leisure.

-Barry Boehm

• Being the foundation for any software system, Architecture can allow or preclude almost all system’s quality attributes.

• Software quality cannot be appended late in a project • Proposed design may or may not be suitable w.r.t. some qualities of importance – It should be assessed before long term investment • Acquiring a large software system

Why Analyzing SA

• Cost Vs Benefits • Costs – Staff time – Consumption of experienced designers • Benefits • At AT&T over last eight years average cost savings due to full architecture review is 10% • A large company avoided a multimillion-dollar purchase when architecture of global information system they were procuring was found to be incapable of providing the desired system attributes necessary to support a product line

Why Analyzing SA

• Benefits • An architecture review of a retail merchandise revealed early that there would be peak order performance problems that no amount of hardware could fix; a major business failure was avoided.

• In the architecture review of revision control system, a number of severe limitations in achieving system portability and modifiability were uncovered

Why Analyzing SA

• Benefits – Early detection of problems – Validation of requirements • Architectural reviews help uncover conflicts and tradeoffs – Improvement in architecture

Quality Attributes

• Evaluation and Quality attributes • Categories – System Quality attributes discernable at runtime – System Quality attributes Not discernable at runtime

Quality Attributes

• System Quality attributes discernable at runtime – Performance – Security – Reliability – Usability

Quality Attributes

• System Quality attributes Not discernable at runtime – Modifiability – Portability – Reusability

Analysis/Review techniques

• Questioning techniques – Scenario based techniques – Questionnaire based techniques – Checklist based techniques • Relationship between Scenarios, questionnaires and checklists • Measuring techniques – Metrics,Simulation and Prototypes – These techniques provide quantitative results and provide answers to the questions a review team already have about particular qualities.

• An existing prototype or simulation may be an answer to an issue raised by a questioning technique

Analysis/Review techniques

• Classification of Techniques – Generality • General purpose (Questionnaire,Metrics) or domain specific (Scenarios,checklists etc) – Level of detail • Coarse (Questionnaire), medium(Scenarios) and fine(Metrics) – Phases • Early, middle and post deployment – What is Reviewed • Artifact (All) or Process(Questionnaire and checklist)

Scenarios

• Scenarios are brief narratives of expected or anticipated use of a system from both development and end-user viewpoint • Quality attributes such as modifiability etc are abstract and vague – No universal measures for these attributes. Context dependent measures are possible – Scenarios are used to represent operational context for a system

Scenarios

• The process of choosing scenarios for analysis forces designers to consider the future uses of and changes to the system – How will this architecture accommodate the following change?

Scenarios

• Benefits of using Scenarios in analysis – Better understanding of requirements – Stakeholder buy-in – Better documentation – Requirements tractability at architectural level • Specifically quality attributes

Scenario Based Analysis Techniques

• They are many – Software Architecture analysis method(SAAM) – SAAM for complex scenarios (SAAMCS) – SAAM for evolution and reusability(SAAMER) – Aspectual SAAM (ASAAM) – Architecture level Modifiability Analysis (ALMA) – ATAM,CBAM,ARID,PASA,CBAR….

Software Architecture Analysis Method (SAAM)

SAAM

Scenarios and their Evaluation

• Develop task scenarios that illustrate the kinds of activities the system must support and the kinds of changes which it is anticipated will be made to the system over time.

• Scenario evaluations : Direct/Indirect scenarios – For each indirect scenario, list the changes to the architecture that are necessary for it to support the scenario and estimate the cost of performing the change.

– A modification to the architecture means that either a new component or connection is introduced or an existing component or connection requires a change in its specification.

– For each indirect scenario the impact, or set of changes, that scenario has on the architecture should be described

Scenarios and their Evaluation

• Scenario Interaction – Different indirect scenarios may necessitate changes to the same components or connections • Scenarios

interact

in that component on connector – Scenario interaction measures the extent to which the architecture supports an appropriate separation of concerns – SAAM favors the architecture with the fewest scenario conflicts

Overall Evaluation

• Finally, weight each scenario and the scenario interactions in terms of their relative importance.

• Determine an overall ranking. This is a subjective process involving all of the stake-holders in the system. • The weighting chosen will reflect the relative importance of the quality factors that the scenarios manifest

Validation of SAAM

•

Global information system

• A company was contemplating the purchase of a system as the infrastructure to support applications development for multimedia communication with unlimited conferencing.

• The purchasing company wanted some assurance that the architecture of the system they purchased was going to provide for the generic satellite-based multi-user applications they anticipated developing in the near and long term. • As a result of the analysis, the company decided to not purchase the system, avoiding an investment of tens of millions of dollars.

Validation of SAAM

•

Air traffic control

• This was an investigation of a complex, real-time system against a set of proposed changes to that system. • The purpose of the evaluation was to determine whether future development on this system was justified.

• The change scenarios were intended to represent appropriate manifestations of the abstract qualities of performance and availability. • The result of this evaluation was a decision to proceed with the proposed changes

Validation of SAAM

•

WRCS

• This case study was an analysis of a commercial version control/configuration management tool • This analysis covers all activities of SAAM and shows all artifacts of a SAAM evaluation that can be produced.

• The result of the analysis was that significant problems were discovered with the product’s architecture, with respect to the scenarios considered

WRCS

• WRCS provides the functionality to allow developers of projects the ability to create archives, compare files, check files in and out, create releases, back up to old versions of files, and so on.

• “Project'' in this context means any group of related files that, when linked together appropriately, form a finished product. • WRCS keeps track of changes made to these files as they evolve over time • It provides capabilities for multiple users to work on the same project within their own private work areas,

Applying SAAM Steps Develop Scenarios/Describe Candidate Architecture

Applying SAAM Steps Develop Scenarios/Describe Candidate Architecture

•

User:

• 1. • 2.

Compare binary file representations

stored in the files.

Configure the product's toolbar

associated with a button in the toolbar.

. Compare binary files generated by other products. For example, FrameMaker files are stored in a binary representation. But when we are comparing two versions of a FrameMaker file we want to see our editing changes in a human-readable form, and not the changes to the binary codes . Change the icons and actions •

Maintainer:

• 3.

Port to another operating system

.

• 4.

Make minor modifications to the user interface

. Add a menu item, change the look and feel of a dialog box.

•

Administrator:

• 5.

Change access permissions for a project

.

• 6.

Integrate with a new development environment

. Attach for example to Symantec C++.

Perform Scenario Evaluation

Reveal Scenario Interaction

Fish eye diagram of Scenario Interaction

Overall Evaluation

• Prioritize the scenarios which have been identified as potentially problematic • The WRCS analysis identified a number of severe limitations in achieving the extra functional qualities of portability and modifiability. • A major redesign of the system was recommended.

• Having gone through an analysis procedure such as SAAM before implementation would have substantially contributed to avoiding the problems which the WRCS developers now face

Results and Lessons

• SAAM is for people • SAAM and traditional architectural metrics • Determining the proper level of architectural description • Determining the proper set of scenarios.