Globus Toolkit Developer Tutorial: Basics
Download
Report
Transcript Globus Toolkit Developer Tutorial: Basics
Basics
Globus Toolkit™ Developer Tutorial
The Globus Project™
Argonne National Laboratory
USC Information Sciences Institute
http://www.globus.org/
Copyright (c) 2002 University of Chicago and The University of Southern California. All Rights Reserved.
This presentation is licensed for use under the terms of the Globus Toolkit Public License.
See http://www.globus.org/toolkit/download/license.html for the full text of this license.
Setting the Foundation
Strategies for the Globus Toolkit
– Layered architecture, protocols & APIs
Naming conventions
– Functions, variables, structures, etc.
Installation tree structure
– Where things get installed
Source tree structure
– Where things come from
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
2
Globus Approach
Software toolkit addressing key technical areas
– Offer a modular “bag of technologies”
– Enable incremental development of grid-enabled
tools and applications
– Define and standardize grid protocols and APIs
(Our software development supports this goal.)
Focus is on inter-domain issues, not clustering
– Supports collaborative resource use spanning
multiple organizations
– Integrates cleanly with intra-domain services
– Creates a “collective” service layer
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
3
Globus Approach
Focus on architecture issues
Applications
– Provide implementations of
Diverse global services
grid protocols and APIs as
basic infrastructure
– Use to construct high-level,
domain-specific solutions Core Globus
Design principles
services
– Keep participation cost low
– Enable local control
– Support for adaptation
Local OS
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
4
Layered Architecture
Applications
High-level Services and Tools
DRM
Cactus
GASS
GridFTP
MPI
PUNCH
Nimrod/G
Condor-G
Core Services
Metacomputing
Directory
Service
Condor
MPI
LSF
PBS
July 21, 2015
globusrun
Grid Status
NQE
Globus
Security
Interface
Local
Services
GRAM
Replica
Catalog
Linux
Globus Toolkit™ Developer Tutorial: Basics
I/O
TCP
UDP
AIX
Solaris
5
Why C
(Why Not C++)
Portability
– Most C++ compilers do not support complete ANSI C++
standard
– This is getting better, but currently C++ is still a
headache for highly portable code
Linkage
– Linking with Fortran can be difficult
Religion
– C++ gives too much rope to hang yourself
– Good C coding practices are usually adequate
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
6
Naming Conventions
Globus Toolkit is comprised of a set of
modules, each with a unique name
A set of related modules may be part of a
larger package
In order to avoid name collisions, all
names begin with globus or grid
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
7
Function Naming Conventions
All function names and other exported
symbols follow the form:
globus_<package>_<module>_<action>
The <package> is omitted if the module is
not part of a larger package of related APIs
Examples:
– globus_gram_client_job_submit(…)
– globus_nexus_send_rsr(…)
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
8
Function Return Values
Most functions return either an int or
globus_result_t value
Older APIs return value: int
– Like errno
– GLOBUS_SUCCESS (0) indicates that the
function was successful
– Any other return value is an error code
– Error codes defined in module headers
– Many modules still use this approach
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
9
Function Return Values
Newer APIs return value: globus_result_t
– New approach - like exception objects
– GLOBUS_SUCCESS still indicates success
– Otherwise can be expanded to a globus_error_t
“object”
> Error type hierarchy with single inheritance
> Can easily match against types (similar to “catch”)
> Can be used to return extended information about the error
– New 1.1 modules use this approach
– Will gradually be pushed through other
modules
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
10
Library Naming Conventions
Library names follow the same naming
convention as the functions in that library:
libglobus_<package>_<module>.a
Currently shared libraries are not
supported
Examples:
– libglobus_gram_client.a
– libglobus_nexus.a
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
11
Program Naming Conventions
All program names start with globus or grid
Dash (-) is used as a word delimiter
Many Globus programs are not directly
related to a single module
– May use multiple modules to implement
higher level functionality
> e.g. globusrun, globus-job-submit
– May not be directly related to any module
> e.g. globus-sh-exec
– But module name used where appropriate
> e.g. globus-gass-server
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
12
Install Tree Structure
GLOBUS_LOCATION environment variable
points to the install tree, containing:
– bin: Programs for your path
– sbin: System binaries
– include, lib: Development headers & libraries
– libexec: Programs/scripts used by libraries
– etc: Configuration files
– man: Man pages
– var: Log files, etc.
– share: Other shared data files
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
13
Uses
Globus Toolkit Modules
gass
gass_cache
rsl
gass_transfer
duroc_control
duroc_runtime
gram_client
gram_myjob
hbm_datacollector
nexus
hbm_client
io
ldap
gss_assist
utp
gss
dc
mp
common (used by everything)
module activation, threads, libc wrappers, callbacks, errors, etc.
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
14
New Globus Toolkit
Data Grid Modules
Custom
Servers
Custom
Clients
globus-url-copy
globus_gass
Replica
Programs
globus_replica_manager
globus_gass_copy
globus_gass_transfer
globus_ftp_client
globus_ftp_control
globus_replica_catalog
Legend
Program
globus_io
OpenLDAP client
Library
Already
exist
July
21, 2015
globus_common
GSI (security)
Globus Toolkit™ Developer Tutorial: Basics
15
Security Modules
gssapi
– Grid Security Infrastructure
– GSS-API implementation
– SSL authentication (public key)
gss_assist
– Simplified interface to GSS-API
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
16
Resource Management Modules
RSL parser
GRAM
– client API: simple client
– myjob API: parallel job bootstrapping
– jobmanager API: server infrastructure
– gatekeeper: root security & dispatch svc
DUROC
– control API: client
– runtime API: job startup barriers, etc
– bootstrap API: parallel job bootstrapping
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
17
GridFTP Modules
GridFTP control
– Low level handling of control and data
channels
GridFTP client
– High level client
> Get, put, 3rd party transfer, cd, mkdir, rmdir, etc…
– Rich plug-in support for reliability and
monitoring
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
18
GASS (File Access) Modules
GASS url copy
– url-to-url file copy, for file:, http:, ftp: urls
– Wrapped by globus-url-copy program
GASS file
– open(), close(): works on files and URLs
GASS transfer
– Client & server handling of get/put requests
GASS server_ez
– Simple, embeddable server
GASS cache
– Cache for remote files (prefetch etc.)
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
19
Information Services Modules
MDS tools
– client programs
– server tools
– built on OpenLDAP
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
20
Miscellaneous Modules
Common
– Fundamental system and portability API
IO
– Fundamental I/O API
Data conversion
UTP timing package
Miscellaneous programs and scripts
Programs that use multiple APIs
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
21
Communication Modules
MP
– Simple wrapper around message passing
libraries
Duct
– Used by GRAM and DUROC for startup and
bootstrapping
Nexus (deprecated)
– Multi-method communication library
– Being deprecated
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
22
Summary
Hopefully you should be starting to get
your bearing within the Globus Toolkit
Next each module will be explored, via:
– Summary of functionality
– Walk-through with sample tools
– Walk-through with APIs
– Discussion of interesting issues
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
23