Transcript High-Performance Schedulers Francine Berman

High-Performance Schedulers
Francine Berman
CSCI 599 Grid Computing
Class Presentation
Caimu Tang
7/21/2015
High-Performance Schedukers by
Francine Berman
Why scheduling
• Optimize Performance: execution time,
throughput, fairness and etc. (QoS)
• Load balancing.
• Help to design an effective program model.
• Ubiquity. process scheduling in operating
system, task scheduling in parallel
computing and scheduling in real life too.
7/21/2015
High-Performance Schedukers by
Francine Berman
Scheduling in GRID
• Application level.
• resource e.g. data, communication
bandwidth.
• Models, scheduling policy, program model,
performance model, performance
measurement.
• Current performance measure, minimize
execution time.
7/21/2015
High-Performance Schedukers by
Francine Berman
Requirements on GRID
scheduling model
• Adaptive to the dynamic environment.
• Adaptive to the varying performance
metrics upon the course of application
execution.
• Performance predictions over time.
• Coarse and fine-tuning the component
parameters.
7/21/2015
High-Performance Schedukers by
Francine Berman
Techniques commonly employed
• Parameterize the components in an
application.
• Make use of dynamic information, e.g. CPU
slots available percentage, network
bandwidth available percentage.
• Compositional scheduling model, structural
character of application and dynamic
interaction with grid environment.
7/21/2015
High-Performance Schedukers by
Francine Berman
Program Model
• Data-flow-style program graphs. Program
dependency graph, dependency graph based
on task phase,or coarse-grained task
dependency graph.
• Application is represented by its
characteristics. Resource requirements,
some problem in some specific domain.
7/21/2015
High-Performance Schedukers by
Francine Berman
Example: Task Dependency Graph
7/21/2015
High-Performance Schedukers by
Francine Berman
Performance Model
• Scheduler-driven performance model. Using
dynamic information, skeleton performance
model, last program iteration as benchmark, using
offline mapping indexed by run-time information.
• User-driven performance model. Using static
and dynamic information, depending on
programmer to use the system characteristics.
7/21/2015
High-Performance Schedukers by
Francine Berman
Scheduling Policy
• Choose a set of resources to achieve the
performance goal.
• Fist Come, First Serve.
• Preemptive.
• Fair Queuing.
• And etc.
7/21/2015
High-Performance Schedukers by
Francine Berman
AppLes: Application-Level
Scheduler
• Everything evaluated in terms of the impact on the
application, so the resources are evaluated in terms
of the predicted capacities and their potential for
requirements.
• No resource manager is assumed.
• On User-level, no specific privilege required.
• Heterogeneous and cross organization.
• Depends on use Network Weather Service for the
dynamic resource load and availability.
7/21/2015
High-Performance Schedukers by
Francine Berman
AppLes(Cont’d)
• Information gathered by the network weather service is
used to parameterize performance models and to predict
the state of grid resources at the time the application will
be scheduled.
• Time balancing, all processors are assigned some possibly
nonuniform amount of the goal that they will all finish at
roughly the same time.
• Compositional component models is deployed.
• Adaptive scheduling scheme.
7/21/2015
High-Performance Schedukers by
Francine Berman
DSSA:Digital Sky Survey Analysis
• DAS, Data Assimilation System, Characteristics.
– I/O Bound
– Huge bandwidth for data transfer
– Data represenation (Data set, metadata)
– Data federation.
• DSS
– Huge amount of image data (250GB of metadata per single plate).
– Data set can be extracted from the various database for reanalysis
and possibly create new high integrity data set.
– Data curation/validation will be performed on the dataset upon
needs.
– Multiple data repositories. Database federation.
7/21/2015
High-Performance Schedukers by
Francine Berman
Scheduling DSSA using AppLeS
• DSSA, digitization, curation and validation of
photographic plates for archiving, querying and etc.
(metadata, dataset replication, dataset database federation)
• Communication resource scheduling using dynamic
information provided by Network Weather Service.
• Compositional performance model. (Metainformation,
parameters for dynamic resource information).
• Select the candidate to minimize the execution time.
• Actuate the selected schedule.
• AppLes may help DSSA to modify to adapt AppLes and
achieve overall high performance.
7/21/2015
High-Performance Schedukers by
Francine Berman
Trends
• Using dynamic information. (adaptation)
• Using metainformation. (QoIn)
• Using realistic programs, more application
code specific.
• Restricting domain, more domain specific.
• Develop language interface, automating
scheduling process.
7/21/2015
High-Performance Schedukers by
Francine Berman
Challenges
• Portability vs. Performance, minimize the
performance impact of architecture
independence.
• Grid-Aware programming, using HighPerformance scheduler and leverage the
performance potential of grid environment.
• Scalability.
7/21/2015
High-Performance Schedukers by
Francine Berman
Challenges (Cont’d)
• Efficiency, scheduler overhead should not
affect the normal application execution or
should be kept at minimal level possible.
• Repeatability, i.e. Consistency,
predictability.
• Multischeduling, cooperating with resource
schedulers, stability and no thrashing.
7/21/2015
High-Performance Schedukers by
Francine Berman
System Support
• Dynamic monitoring mechanisms, information persistency,
provide faithful information to scheduler. (extensible and
flexible).
• High-level language support. Provide uniform semantics
cross computational grids, flexible so that low level service
may change so long as the high-level semantics is
consistent.
• Integration with other software tools.
• Assistance for multischeduling, information interfaces,
data synchronization,
7/21/2015
High-Performance Schedukers by
Francine Berman
Conclusion
• Scheduling is the key for performance in
grid environment.
• Coordinating resources in grid environment
• Most advanced grid application are targeted
to specific resources.
• High-Performance Scheduling Evolution.
7/21/2015
High-Performance Schedukers by
Francine Berman