Transcript Document 7385145

An Integrated Synchronization and
Consistency Protocol for the
Implementation of a High-Level Parallel
Programming Language
Martin C. Rinard
University of California, Santa Barbara
Santa Barbara, California 93106
Context
• Parallel Programming System
• Single Address Space Programming Model
• Message Passing Machine
• Key Functionality
• Synchronization
• Communication
• Consistency
• Straightforward Implementation Separates Protocols
• Synchronization Messages
• Communication and Consistency Messages
• Redundant Messages
Jade
High-Level, Implicitly Parallel Language
• Single Address Space
• Serial Semantics
• Portable
• Shared Memory Machines - Stanford DASH
• Message Passing Machines - iPSC/860, CM-5
• Heterogeneous Networks of Workstations
• Message Passing Implementation
• Combines Synchronization and Consistency Protocols
• Result: Single Efficient Protocol
Structure of Talk
• Jade Programming Model
• Synchronization Mechanism
• Integrated Synchronization and Consistency Protocol
• Experimental Results
Programming Model
• Programmer
• Starts With A Serial Program
• Identifies Data Granularity using Shared Objects
• Identifies Computation Granularity using Tasks
• Provides Access Specification For Each Task
• Jade Implementation Analyzes Access Specifications
• Automatically Extract Parallelism
• Synchronize Computation
• Generate Communication
• Ensure Consistency of Shared Objects
• Preserves Serial Semantics
Programming Model
Starts With Serial Program
Programmer
Data Granularity - Shared Objects
Task Granularity
Data Usage Information
Implementation
Extracts Concurrency
Synchronizes Computation
Generates Communication
Ensures Consistency of Shared Objects
Concurrency Principle
• Tasks Execute Serially if
• One declares a write to a piece of data
• Other declares either read or write to the same piece of data
• Same execution order as serial program
rd
wr
rd
wr
wr
• Otherwise, Tasks Execute Concurrently
wr
wr
rd
wr
rd
wr
Key Aspects
• Implementation Manages Concurrency
• Synchronization
• Communication
• Consistency
• Promotes Modularity
• Serial Semantics
• Deterministic Execution
• Only Serial Programs
• Portable
Object Queues
wr
wr
rd
wr
rd
rd
wr
rd
wr
wr
Object Queues
wr
wr
wr
Time
wr
rd
wr
rd
rd
wr
rd
rd
wr
rd
rd
wr
rd
rd
wr
rd
rd
wr
rd
Consistency Problem
Processor 0
Processor 1
wr
Time
rd
wr
• Invalidate Protocol
• Update Protocol
rd
wr
Expensive Global Operations
Consistency Mechanism
• Object Replica Has Version Number
• Object Queue Maintains
• Latest Version Number
• Owner of Latest Version
• Queue Tells Each Task
• What Version to Access
• Owner of that Version
• Enabled Task Checks Version Numbers
• If Obsolete, Fetch From Owner
• If Up to Date, No Fetch
Consistency Mechanism
wr
2
wr
rd
wr
rd
wr
rd
rd
rd
wr
3
rd
Processor
0
2
wr
Processor
1
2
rd
wr
2
rd
3
rd
wr
2
rd
wr
rd
3
rd
3
rd
rd
3
3
rd
rd
3
wr
wr
Result
• A More Efficient Consistency Mechanism
• No Required Invalidate Messages
• No Required Update Messages
• Enabling Features
• High-Level Language Model
• Data Usage Information
• Integration With Synchronization Mechanism
Experimental Results
• Compare Three Consistency Mechanisms
• Update (Broadcast if all Processors should receive Object)
• Invalidate
• Version
• Applications
• Panel Cholesky
• Ocean
• Water
• Collected Data on iPSC/860
• Communication Volume
• Number of Messages
Panel Cholesky
Number of Messages
Communication Volume
Ocean
Number of Messages
Communication Volume
Water
Number of Messages
Communication Volume
Adaptive Broadcast
• Goal
• Advantages of Broadcast for Widely Accessed Data
• Without Disadvantages of Update Protocol
• Policy
• If Every Processor Reads Same Version of Object
• Jade Implementation Uses Broadcast Protocol for that Object
• Result
• Performs as well as Update Protocol for Water
• Performs as well as Version for Panel Cholesky and Ocean
Related Work
• Consistency Based on Version Numbers
• NAMOS (Reed)
• Sprite (Nelson, Welch, Ousterhout)
• SAM (Scales, Lam)
• VDOM (Feeley, Levy)
• TreadMarks (Keleher, Dwarkadas, Cox, Zwaenepoel)
• Midway (Zekauskas, Sawdon, Bershad)
Conclusion
• Jade Presents High-Level Abstractions
• Serial Semantics
• Single Address Space
• Jade Implementation Provides an Integrated Protocol
• Synchronization
• Communication
• Consistency
• Experimental Results
• Adaptive Broadcast