Transcript BQS Update    New Users Needs More users & machines, Scalability issues Needs for more sophisticated monitoring and control over the system    [email protected] Architecture Scheduling Job Types HEPIX Edinburgh 26/5/04

BQS Update



New Users Needs
More users & machines, Scalability issues
Needs for more sophisticated monitoring and
control over the system



[email protected]
Architecture
Scheduling
Job Types
HEPIX Edinburgh 26/5/04
Architecture Update
MySQL DB
DB Agent
query
Client
BQS
Scheduler
Worker
[email protected]
DB Agent
Worker
HEPIX Edinburgh 26/5/04
Scheduling Update



[email protected]
Scheduler
Resources
Quasi Interactive Jobs
HEPIX Edinburgh 26/5/04
Scheduler

More Control for Operation and
Administration:




Weight of Past Resource Usage
And Group Objectives
Max Job Duration
Small Jobs Bias
[email protected]
HEPIX Edinburgh 26/5/04
Logical Resources

Beyond Traditional Resources


Logical Resources




E.G. Disk, Time, Memory
Name
Max Available
Restricted Flag
Admin Defined Resources

[email protected]
E.G. HPSS
HEPIX Edinburgh 26/5/04
Logical U_Resource

Created & managed by Users:

Decide of the Name: u_XXX

Receive Privilege bqs.u_XXXadmin

Set Max Available and Restricted Flag

Grant/deny bqs.u_XXXusage privilege
[email protected]
HEPIX Edinburgh 26/5/04
Privilege Management

A General Service, APIs, Commands To:
Grant, Deny, Check & List Privileges

Given to Users, Groups and Machines

EG in BQS applid:
bqs.admin, bqs.oper, bqs.spawn_forbidden
[email protected]
HEPIX Edinburgh 26/5/04
New Job Types

Parallel Jobs

Arborescent Jobs

GRID Jobs
[email protected]
HEPIX Edinburgh 26/5/04
Parallel Jobs

2 new submit options: proc, ptype

proc: Number of WorkPoints

ptype: PVM, MPICH, LAM-MPI
[email protected]
HEPIX Edinburgh 26/5/04
Parallel Jobs
Client
query
BQS DB
Agent
BQS
Master
Worker
[email protected]
MySQL
DB
DB DB
Agent
Worker
HEPIX Edinburgh 26/5/04
Arborescent Job


SNOVAE:
many related small tasks
need short global response time
Schedule and spawn as one Job to
reduce BQS latency

Runs on a number of WorkPoints

User must describe tasks dependencies
[email protected]
HEPIX Edinburgh 26/5/04
GRID Jobs




Real and Generic Accounts
AFS Tokens and Certificates
Specific RH 7.3 + LCG Soft
Full Production Farm
(Currently a Specific “lcg”
Logical Test Farm for
Validation)
[email protected]
HEPIX Edinburgh 26/5/04
Other Projects


BIO: Quasi Interactives Jobs
Installation and documentation
for LCG
[email protected]
HEPIX Edinburgh 26/5/04