Transcript Globus Toolkit Developer Tutorial: Basics

Basics
Globus Toolkit™ Developer Tutorial
The Globus Project™
Argonne National Laboratory
USC Information Sciences Institute
http://www.globus.org/
Copyright (c) 2002 University of Chicago and The University of Southern California. All Rights Reserved.
This presentation is licensed for use under the terms of the Globus Toolkit Public License.
See http://www.globus.org/toolkit/download/license.html for the full text of this license.
Setting the Foundation

Strategies for the Globus Toolkit
– Layered architecture, protocols & APIs

Naming conventions
– Functions, variables, structures, etc.

Installation tree structure
– Where things get installed

Source tree structure
– Where things come from
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
2
Globus Approach

Software toolkit addressing key technical areas
– Offer a modular “bag of technologies”
– Enable incremental development of grid-enabled
tools and applications
– Define and standardize grid protocols and APIs
(Our software development supports this goal.)

Focus is on inter-domain issues, not clustering
– Supports collaborative resource use spanning
multiple organizations
– Integrates cleanly with intra-domain services
– Creates a “collective” service layer
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
3
Globus Approach

Focus on architecture issues
Applications
– Provide implementations of
Diverse global services
grid protocols and APIs as
basic infrastructure
– Use to construct high-level,
domain-specific solutions Core Globus

Design principles
services
– Keep participation cost low
– Enable local control
– Support for adaptation
Local OS
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
4
Layered Architecture
Applications
High-level Services and Tools
DRM
Cactus
GASS
GridFTP
MPI
PUNCH
Nimrod/G
Condor-G
Core Services
Metacomputing
Directory
Service
Condor
MPI
LSF
PBS
July 21, 2015
globusrun
Grid Status
NQE
Globus
Security
Interface
Local
Services
GRAM
Replica
Catalog
Linux
Globus Toolkit™ Developer Tutorial: Basics
I/O
TCP
UDP
AIX
Solaris
5
Why C
(Why Not C++)

Portability
– Most C++ compilers do not support complete ANSI C++
standard
– This is getting better, but currently C++ is still a
headache for highly portable code

Linkage
– Linking with Fortran can be difficult

Religion
– C++ gives too much rope to hang yourself
– Good C coding practices are usually adequate
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
6
Naming Conventions



Globus Toolkit is comprised of a set of
modules, each with a unique name
A set of related modules may be part of a
larger package
In order to avoid name collisions, all
names begin with globus or grid
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
7
Function Naming Conventions

All function names and other exported
symbols follow the form:
globus_<package>_<module>_<action>


The <package> is omitted if the module is
not part of a larger package of related APIs
Examples:
– globus_gram_client_job_submit(…)
– globus_nexus_send_rsr(…)
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
8
Function Return Values


Most functions return either an int or
globus_result_t value
Older APIs return value: int
– Like errno
– GLOBUS_SUCCESS (0) indicates that the
function was successful
– Any other return value is an error code
– Error codes defined in module headers
– Many modules still use this approach
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
9
Function Return Values

Newer APIs return value: globus_result_t
– New approach - like exception objects
– GLOBUS_SUCCESS still indicates success
– Otherwise can be expanded to a globus_error_t
“object”
> Error type hierarchy with single inheritance
> Can easily match against types (similar to “catch”)
> Can be used to return extended information about the error
– New 1.1 modules use this approach
– Will gradually be pushed through other
modules
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
10
Library Naming Conventions

Library names follow the same naming
convention as the functions in that library:
libglobus_<package>_<module>.a


Currently shared libraries are not
supported
Examples:
– libglobus_gram_client.a
– libglobus_nexus.a
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
11
Program Naming Conventions

All program names start with globus or grid

Dash (-) is used as a word delimiter

Many Globus programs are not directly
related to a single module
– May use multiple modules to implement
higher level functionality
> e.g. globusrun, globus-job-submit
– May not be directly related to any module
> e.g. globus-sh-exec
– But module name used where appropriate
> e.g. globus-gass-server
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
12
Install Tree Structure

GLOBUS_LOCATION environment variable
points to the install tree, containing:
– bin: Programs for your path
– sbin: System binaries
– include, lib: Development headers & libraries
– libexec: Programs/scripts used by libraries
– etc: Configuration files
– man: Man pages
– var: Log files, etc.
– share: Other shared data files
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
13
Uses
Globus Toolkit Modules
gass
gass_cache
rsl
gass_transfer
duroc_control
duroc_runtime
gram_client
gram_myjob
hbm_datacollector
nexus
hbm_client
io
ldap
gss_assist
utp
gss
dc
mp
common (used by everything)
module activation, threads, libc wrappers, callbacks, errors, etc.
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
14
New Globus Toolkit
Data Grid Modules
Custom
Servers
Custom
Clients
globus-url-copy
globus_gass
Replica
Programs
globus_replica_manager
globus_gass_copy
globus_gass_transfer
globus_ftp_client
globus_ftp_control
globus_replica_catalog
Legend
Program
globus_io
OpenLDAP client
Library
Already
exist
July
21, 2015
globus_common
GSI (security)
Globus Toolkit™ Developer Tutorial: Basics
15
Security Modules

gssapi
– Grid Security Infrastructure
– GSS-API implementation
– SSL authentication (public key)

gss_assist
– Simplified interface to GSS-API
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
16
Resource Management Modules

RSL parser

GRAM
– client API: simple client
– myjob API: parallel job bootstrapping
– jobmanager API: server infrastructure
– gatekeeper: root security & dispatch svc

DUROC
– control API: client
– runtime API: job startup barriers, etc
– bootstrap API: parallel job bootstrapping
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
17
GridFTP Modules

GridFTP control
– Low level handling of control and data
channels

GridFTP client
– High level client
> Get, put, 3rd party transfer, cd, mkdir, rmdir, etc…
– Rich plug-in support for reliability and
monitoring
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
18
GASS (File Access) Modules

GASS url copy
– url-to-url file copy, for file:, http:, ftp: urls
– Wrapped by globus-url-copy program

GASS file
– open(), close(): works on files and URLs

GASS transfer
– Client & server handling of get/put requests

GASS server_ez
– Simple, embeddable server

GASS cache
– Cache for remote files (prefetch etc.)
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
19
Information Services Modules

MDS tools
– client programs
– server tools
– built on OpenLDAP
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
20
Miscellaneous Modules

Common
– Fundamental system and portability API

IO
– Fundamental I/O API

Data conversion

UTP timing package

Miscellaneous programs and scripts

Programs that use multiple APIs
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
21
Communication Modules

MP
– Simple wrapper around message passing
libraries

Duct
– Used by GRAM and DUROC for startup and
bootstrapping

Nexus (deprecated)
– Multi-method communication library
– Being deprecated
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
22
Summary


Hopefully you should be starting to get
your bearing within the Globus Toolkit
Next each module will be explored, via:
– Summary of functionality
– Walk-through with sample tools
– Walk-through with APIs
– Discussion of interesting issues
July 21, 2015
Globus Toolkit™ Developer Tutorial: Basics
23