Transcript Building a Layered Framework for the Table Abstraction

Designing a Flexible Framework
for a Table Abstraction
H. Conrad Cunningham 1 Yi Liu 2 Jingyi Wang 3
1
University of Mississippi
2 South Dakota State University
3Acxiom Corporation
Hot Spots, Patterns, and Contracts:
An Exploration of Software
Framework Design
H. Conrad Cunningham 1 Yi Liu 2
1
University of Mississippi
2 South Dakota State University
Hot Spots References

Coplien, J., Hoffman, D., Weiss, D.
Commonality and variability in software
engineering, IEEE Software, vol. 15, no. 6,
pp. 37-45, 1998.

Schmid, H. A., Framework design by
systematic generalization, In: Fayad, M. E.,
Schmidt, D. C., Johnson, R. E., editors,
Building Application Frameworks: ObjectOriented Foundations of Framework
Design, Wiley, pp. 353-378, 1999.
26-Jan-2010
3
Patterns References

Gamma, R., Helm, R., Johnson, R.,
Vlissides, J., Design Patterns: Elements of
Reusable Object-Oriented Software,
Addison Wesley, 1995.

Buschmann, F., Meunier, R., Rohnert, H.,
Sommerlad, P., Stal, M., Pattern-Oriented
Software Architecture: A System of Patterns,
Wiley, 1996.
26-Jan-2010
4
Contracts References

Meyer, B., Object-Oriented Software
Construction, 2nd Edition, Prentice Hall
PTR, 1997.
26-Jan-2010
5
Source Papers

Cunningham, H. C., Liu, Y., Wang. J., Designing a
flexible framework for a table abstraction, Chapter
13 in: Chan, Y., Talburt, J., Talley, T., editors, Data
Engineering: Mining, Information, and
Intelligence, Springer, pp. 279-314, 2010.

Cunningham, H. C., Wang, J., Building a Layered
framework for the table abstraction, In
Proceedings of the 16th ACM Symposium on
Applied Computing, pp. 668-674, 2001.
26-Jan-2010
6
Software Interfaces Project
(1996-2000)
Context: development of an instructional data
and file structures library (for Java in CS 3)


artifacts for study of good design techniques
system for use, extension, and modification
Motivation: study techniques for


26-Jan-2010
presenting important methods to students
(frameworks, software design patterns, design by
contract, etc.)
unifying related file and data structures in
framework
7
Table Abstract Data Type (ADT)

Collection of records
o
o


each a finite sequence of data fields
key field value uniquely identifies record within
collection
Operations on collection and on record using key
Several different implementations possible
26-Jan-2010
key1
data1
key2
key3
key4
data2
data3
data4
8
Table ADT Operations








Insert new record
Delete existing record given key
Update existing record
Retrieve existing record given key
Get number of records
Query whether contains given key
Query whether empty
Query whether full
26-Jan-2010
9
Software Framework



Generic application allowing creation of
members of family of related programs
Reusable design expressed as set of
abstract classes and way they collaborate
Common and variable aspects known as
frozen spots and hot spots
Frozen spots
Hot spots
Framework
Framework
library
User-supplied code
(application specific)
26-Jan-2010
10
Software Design Patterns

Describe recurring design problems arising in
specific contexts

Present well-proven generic solution schemes

Describe solution’s components and their
responsibilities and relationships

To use:
o
select pattern that fits problem
o
structure solution to follow pattern
26-Jan-2010
11
Software Design Contracts

Preconditions for correct use of operation

Postconditions for correct result of operation

Invariant conditions for correct implementation of class
Insert record operation
pre: record is valid and not already in table
post: record now in table
Invariant for table
all records are valid, no duplicate keys
26-Jan-2010
12
Initial Table Framework Requirements
1.
2.
3.
4.
5.
Provide Table ADT functionality for clientdefined records and keys
Support many implementations of access and
storage mechanisms
Separate key-based access mechanism from
storage mechanism
Present coherent abstractions with well-defined
interfaces
Use software design patterns and design contracts
26-Jan-2010
13
Hot Spot Analysis of Requirements
1.
2.
3.
4.
5.
Provide Table ADT functionality for clientdefined records and keys
Support many implementations of access and
storage mechanisms
Separate key-based access mechanism from
storage mechanism
Present coherent abstractions with well-defined
interfaces
Use software design patterns and design contracts
26-Jan-2010
14
Table Hot Spot Analysis
Frozen spot
1.
Table ADT functionality
Hot spots
1.
Variability in keys (defined by clients)
2.
Variability in records (defined by clients)
3.
Variability in external representation of record state
4.
Variability in access (i.e, indexing) mechanism
5.
Variability in storage mechanism
Hot spots #1 and #2 not completely independent
Hot spot #3 quite subtle
Desire to separate concerns (i.e., the hot spots)
26-Jan-2010
15
Layered Architecture Pattern




Distinct groups of services
Hierarchical arrangement of groups into layers
Layer implemented with services of layer below
Enables independent implementation of hot spots
in layers
Client Layer
Access Layer
Storage Layer
26-Jan-2010
16
Applying Layered Architecture Pattern
Client Layer – Hot spots #1 and #2
o includes client application programs
o implements record and key abstractions
o uses layer below to store and retrieve records
Access Layer – Frozen spot and Hot spot #4
o includes table implementations
o provides key-based access to records for layer above
o calls back to record and key implementations in level above
o uses physical storage in layer below
Storage Layer – Hot spot #5
o storage managers
o provides physical storage for records
o calls back to record implementations in layers above
26-Jan-2010
17
Client Layer of Architecture
Encapsulate hot spots #1 and #2
1. Variability in keys (defined by clients)
2. Variability in records (defined by
clients)
26-Jan-2010
18
Client Layer Design
Challenges
o enabling Access Layer to access clientdefined records and keys
o avoiding unnecessary programming to use
common data types
Pattern
o Interface – applied independently for hot
spots #1 (keys) and #2 (records)
26-Jan-2010
19
Client Layer Interfaces
(1 of 2)
Hot spot #1 – Variability in keys


Must be able to (at a minimum) compare with
total ordering
Choose to use Comparable interface from Java
API for convenience
Comparable interface for keys
o
int compareTo(Object key)
compares object with argument
26-Jan-2010
20
Client Layer Interfaces
(2 of 2)
Hot spot #2 – Variability in records


Must be able to extract keys for comparison
Choose to define an appropriate new interface
Keyed interface for records
o
Comparable getKey() extracts key from
record
26-Jan-2010
21
Client Layer Model
Abstract predicates – for use in specification
of this and other layers (not necessarily
implemented)
o
boolean isValidKey(Object key)
yields true for valid key values
o
boolean isValidRec(Object rec)
yields true for valid record values
26-Jan-2010
22
Client Layer Design Contracts
Hot spot #1 Comparable
int compareTo(Object key)
Pre: isValidKey(this) && isValidKey(key)
Post: result == (if this < key then -1
else if this == key then 0
else 1)
Hot spot #2 Keyed
Comparable getKey()
Pre: isValidRec(this)
Post: (result == this.key)
&& isValidKey(result)
26-Jan-2010
23
Client Layer Challenges Met?
Enabling Access Layer to access clientdefined records and keys

records and keys implementations encapsulated
within objects that implement Keyed and
Comparable
o
Avoiding unnecessary programming to use
common data types

o
26-Jan-2010
reuse Comparable from Java API
24
Access Layer of Architecture
Provide frozen spot
Table ADT functionality
Encapsulate hot spot #4:
Variability in access (i.e, indexing)
mechanism
26-Jan-2010
25
Access Layer Design
Challenges
o enabling use of client-defined keys and records
o enabling diverse implementations of indexing
structure
o enabling use of diverse storage structures
Pattern
o Interface – applied for frozen spot (Table ADT)
and hot spot #4 (access)
26-Jan-2010
26
Access Layer Interface Table
void insert(Keyed r) inserts r into table
void delete(Comparable key) removes record with key
void update(Keyed r) changes record with same key
Keyed retrieve(Comparable key) returns record with
key
int getSize() returns size of table
boolean containsKey(Comparable key) searches for
key
boolean isEmpty() checks whether table is empty
boolean isFull() checks whether table is full
o
for unbounded, always returns false
26-Jan-2010
27
Access Layer Model
Partial function table :: Comparable  Keyed
o represents abstract table state
o consider this: number of entries finite or infinite? If finite,
bounded or unbounded? If infinite, countable or not?
o #table in postcondition denotes table before operation
o use as either function or set of key-record pairs
Abstract predicates (to capture environmental assumptions)
o isValidKey(Object) and
isValidRec(Object) defined in Client Layer to
identify valid keys and records
o isStorable(Object) defined in Storage Layer to
identify records that can be stored
26-Jan-2010
28
Table Design Contract (1 of 5)
Invariant
( k, r : r == table(k) :
isValidRec(r) && isStorable(r)
&& k == r.getKey() )
All stored keys and records in the table are valid
and capable of being stored on the chosen external
device, and the records can be assessed by their
keys.
26-Jan-2010
29
Table Design Contract (2 of 5)
void insert(Keyed r) inserts r into table
Pre: isValidRec(r) && isStorable(r) &&
!containsKey(r.getKey())&& !isFull()
Post: table == #table  {(r.getKey(),r)}
void delete(Comparable key) removes record with key
from table
Pre:
isValidKey(key) && containsKey(key)
Post: table == #table - {(key,#table(key))}
Question: Does this allow table be initialized nonempty?
26-Jan-2010
30
Table Design Contract (3 of 5)
void update(Keyed r)changes record with same key
Pre: isValidRec(r) && isStorable(r) &&
containsKey(r.getKey())
Post: table == (#table {(r.getKey(),#table(r.getKey()))} ) 
{(r.getKey(),r)}
Keyed retrieve(Comparable key) returns record
with key
Pre: isValidKey(key) && containsKey(key)
Post: result == #table(r.getKey())
26-Jan-2010
31
Table Design Contract (4 of 5)
int getSize() returns size of table
Pre: true
Post: result == cardinality(#table)
Note: If size not finite, need way to represent infinity
boolean containsKey(Comparable key) searches
table for key
Pre: isValidKey(key)
Post: result == defined(#table(key))
26-Jan-2010
32
Table Design Contract (5 of 5)
boolean isEmpty()checks whether table is empty
Pre: true
Post: result == (#table = )
boolean isFull()checks whether table is full
– for unbounded, always returns false
Pre : true
Post: result == (#table implementation has
no free space to store record)
26-Jan-2010
33
Client/Access Layer Interactions

Client calls Access Layer class implementing
Table interface

Access calls back to Client implementations
of Keyed and Comparable interfaces
26-Jan-2010
34
Access Layer Challenges Met?

Enabling use of client-defined keys and records
o

Enabling diverse implementations of the table
o

callbacks to Comparable and Keyed abstractions
that hide the implementation details
careful design of table interface semantics using
design by contract
Enabling use of diverse storage mechanisms
o
26-Jan-2010
calls to Storage Layer interfaces
35
Storage Layer of Architecture
Encapsulate hot spot #5:
Variability in storage mechanism
26-Jan-2010
36
Storage Layer Design
Challenges
o
o
o
o
o
supporting client-defined records
supporting diverse table implementations in Access
Layer (simple indexes, hashing, balanced trees, etc.)
allowing diverse physical media (in-memory, ondisk, etc.)
enabling persistence of table, including access layer
decoupling implementations as much as possible
Patterns
o
o
26-Jan-2010
Bridge
Proxy
37
Bridge Pattern


Decouple “interface” from “implementation”
o table from storage in this case
Allow them to vary independently
o plug any storage mechanism into table
uses
Table
Simple
Indexed
File
26-Jan-2010
Hashed
File
RecordStore
Slotted
File Store
Vector
File
38
Proxy Pattern


Transparently manage services of target object
o isolate Table implementation from
nature/location of record slots in
RecordStore implementation
Introduce proxy object as surrogate for target
RecordStore
Table
RecordSlot
handle
26-Jan-2010
39
Storage Layer Interfaces
RecordStore
o operations to allocate and deallocate storage slots
RecordSlot
o operations to get and set records in slots
o operations to get handle and containing
RecordStore
26-Jan-2010
40
Storage Layer Model
(1 of 2)
Partial function store :: int  Object
o represents abstract RecordStore state
Set Handles  int, NULLHANDLE  Handles
Set alloc  Handles
o represents set of allocated slot handles
Set unalloc == Handles - alloc
o represents set of unallocated slot handles
Abstract predicate isStorable(Object)
o depends on storage mechanism
26-Jan-2010
41
Storage Layer Model
(2 of 2)
Invariant:
( h,r : r == store(h) : isStorable(r))
&&
( h :: h  alloc == defined(store(h)))
All records in the store are capable of being
stored on the selected medium and the stored
records can be access by their handles.
26-Jan-2010
42
RecordStore Interface
RecordSlot newSlot()
allocates a new record slot
RecordSlot getSlot(int handle)
rebuilds record slot using given handle
void releaseSlot(RecordSlot slot)
deallocates record slot
26-Jan-2010
43
RecordStore Design Contract (1 of 2)
RecordSlot newSlot() allocates a new record slot, but it might
be done in a lazy fashion
Pre: true
Post: result.getContainer() == this &&
result.getRecord() == NULLRECORD &&
result.getHandle()  #alloc
&&
result.getHandle()  alloc  {NULLHANDLE}
Note: NULLRECORD built according to Null Object pattern
RecordSlot getSlot(int handle) rebuilds record slot using
given handle
Pre: handle  alloc
Post: result.getContainer() == this
&&
result.getRecord() == #store(handle) &&
result.getHandle() == handle
26-Jan-2010
44
RecordStore Design Contract (2 of 2)
void releaseSlot(RecordSlot slot) deallocates record
slot
slot.getHandle()  alloc  {NULLHANDLE}
&& slot.getContainer() == this
Post: alloc == #alloc - {slot.getHandle()} &&
store == #store –
{(slot.getHandle(),slot.getRecord())}
Pre:
26-Jan-2010
45
RecordSlot Interface
void setRecord(Object rec) stores rec in this slot

allocation of handle done here or already done by getSlot
Object getRecord() returns record stored in this slot
int getHandle() returns handle of this slot
RecordStore getContainer() returns reference to
RecordStore holding this slot
boolean isEmpty() determines whether this slot empty
26-Jan-2010
46
RecordSlot Model
Each slot has two attributes:

reference to RecordStore to which this RecordSlot
belongs, which is immutable

handle for the associated physical storage slot in the
RecordStore (as constrained in RecordStore)
Invariant:
getHandle()  alloc  {NULLHANDLE}
The handle of a RecordSlot object denotes a slot of the
store that has been allocated unless it has the value
NULLHANDLE.
26-Jan-2010
47
RecordSlot Design Contract (1 of 3)
void setRecord(Object rec) stores rec in this slot
o allocation of handle done here or already done by getSlot()
Pre: isStorable(rec)
Post:
Let h == getHandle():
(h  #alloc  store == (#store {(h,#store(h))})  {(h,rec)} ) &&
(h = NULLHANDLE 
( g : g  #unalloc :
alloc == #alloc  {g} &&
store = #store  {(g,rec)} ))
26-Jan-2010
48
RecordSlot Design Contract (2 of 3)
Object getRecord() returns record stored in this slot
Pre: true
Post: Let h == getHandle():
(h  #alloc  result == #store(h)) &&
(h == NULLHANDLE  result == NULLRECORD)
int getHandle() returns handle of this slot
Pre: true
Post: result == this.handle
26-Jan-2010
49
RecordSlot Design Contract (3 of 3)
RecordStore getContainer() returns reference to
RecordStore holding this slot
Pre: true
Post: result == this.container
boolean isEmpty() determines whether this slot empty
Pre: true
Post: result ==
(getHandle() == NULLHANDLE ||
getRecord() == NULLRECORD)
26-Jan-2010
50
Storage Layer Challenges Met?
(1 of 2)

Supporting client-defined records
o

Supporting diverse table implementations in
Access Layer
o

deferred to Externalization Module
careful design of RecordStore and RecordSlot
abstractions to have sufficient functionality
Allowing diverse physical media
o
o
26-Jan-2010
careful design of RecordStore abstraction to hide
media, implementable in many ways,
partly deferred to Externalization Module
51
Storage Layer Challenges Met?
(2 of 2)

Decoupling implementations as much as possible
o use of RecordSlot, handle, and
Externalization Module

Enabling persistence of Table, including Access
Layer
o store RecordStore identifier and handles
26-Jan-2010
52
Access/Storage Layer Interactions

Access Layer Table calls RecordStore in
Storage Layer to get RecordSlot object

Access Layer calls RecordSlot to store and
retrieve its records

If needed, RecordSlot calls back to Record
implementation in Access Layer
o
interface Record defined in Externalization Module
26-Jan-2010
53
Externalization Module of Architecture
Encapsulate Hot Spot #3:
Variability in external representation of
record state
26-Jan-2010
54
Externalization Module Design
Challenges (e.g., to support Storage Layer)
o
supporting client-defined records
o
allowing diverse physical media
o
decoupling implementations
Pattern: Interface
26-Jan-2010
55
Externalization Module Interface
Record
o
void writeRecord(DataOutput out)
writes the client’s record to stream out
o
void readRecord(DataInput in)
reads the client’s record from stream in
o
int getLength() returns number of
bytes in external representation of client’s
record (e.g., written by writeRecord)
26-Jan-2010
56
Externalization Module Contract
(1 of 2)
void writeRecord(DataOutput out) writes this
record to stream out
Pre: true
Post: suffix of stream out ==
this record’s state encoded as byte sequence
void readRecord(DataInput in) reads this record
from stream in
Pre: true
Post: this record’s state ==
prefix of stream in decoded from byte sequence
26-Jan-2010
57
Externalization Module Contract
(2 of 2)
int getLength() returns number of bytes in external
representation of this record (e.g., written by
writeRecord)
Pre: true
Post: result == number of bytes in external
representation of this record
State Restoration Property:
If, for some record, writeRecord() is followed by
readRecord() of the same byte sequence, the observable
state of the record will be unchanged.
26-Jan-2010
58
Externalization Challenges Met?

Supporting client-defined records
o records needing to be externalized implement
Record interface

Allowing diverse physical media
o Record interface implementation provides low-level
physical representation of record storable on most
media

Decoupling implementations
o Record decouples use from implementation
26-Jan-2010
59
Externalization Module Interactions

Record used by Storage Layer may be defined
by either layer above
o
o

hold one Client Layer Keyed record
hold multiple, or fractional, Client Layer records
managed by Access Layer (e.g, multiway tree nodes)
Storage Layer calls back to Record
implementation in layer above
o
26-Jan-2010
implementation in Access Layer might call back to
implementations in Client Layer
60
Abstraction Usage Relationships
Client Layer
Externalization
Module
Keyed
Comparable
Record
Access Layer
Table
Storage
Layer
RecordStore
26-Jan-2010
RecordSlot
61
Other Design Patterns Used





Null Object (for NULLRECORD in Storage Layer)
Iterator
o extended Table operations
o query mechanism
o utility classes
Template Method
Decorator
Strategy
26-Jan-2010
62
Evolving Frameworks Patterns



Generalizing from three examples
Whitebox framework
Component library
o




Wang prototype: two Tables and three RecordStores
Hot spots
Pluggable objects
Fine-grained objects
Blackbox framework
26-Jan-2010
63
Conclusions





Hot spot analysis reveals generalization
opportunities
Design patterns offer systematic way to discover
reliable designs
Design contracts help make specifications precise
Novel design separates access and storage
mechanisms
Case study potentially useful for educational
purposes
26-Jan-2010
64
Future Work









Study more formal use of Schmid’s generalization
methodology
Study use of function generalization methodology
Modify prototypes to match revised design
Redevelop prototypes to use Scala with generics instead
of Java without generics
Rethink Externalization, perhaps using Strategy pattern
Study hot spots and build finer-grained component
library
Adapt earlier work of students on AVL and B-Tree
class libraries
Integrate into SoftwareInterfaces library
Develop instructional materials
26-Jan-2010
65
Acknowledgements







Jingyi Wang for her work on prototype framework
Yi Liu and Jingyi Wang for their work on the papers
Wei Feng, Jian Hu, and Deep Sharma for their work on
earlier table-related libraries
Sudharshan Vazhkudai, Jennifer Jie Xu, Vandana
Thomas, Cuihua Zhang, Xiaobin Pang, and Ming Wei
for work on other frameworks
Pallavi Tadepalli for collaboration on function
generalization
Various reviewers including Bob Cook, Jennifer Jie Xu,
Chuck Jenkins, and Pallavi Tadepalli
Acxiom Corporation for its support
26-Jan-2010
66