Transcript Document

Web Services for Computing
Portals
Marlon Pierce
Community Grids Lab
Indiana University
What Are Web Services?
• Web services framework is an XML-based distributed
object/service/component system.
– SOAP, WSDL, WSIL, UDDI
• Basic ideas is to build a distributed invocation system out
of existing Web standards.
– Most standards defined by W3C, Oasis (IP considerations)
• Very loosely defined, when compared to CORBA, etc.
– Take what you need
– Ignore what you don’t like
– Develop what’s missing.
XML Overview
• XML is a language for building languages.
• Basic rules: be well formed and be valid
• Particular XML “dialects” are defined by an XML
Schema.
– XML itself is defined by its own schema.
• XML is extensible via namespaces
• Many non-Web services dialects
– RDF, SVG, MathML, XForms, XHTML
• Many basic tools available: parsers, XPath, and
XQuery for searching, etc.
XML and Web services
• XML provides a natural substrate for distributed computing:
– Its just a data description.
– Platform, programming language independent.
• So let’s describe the pieces.
• Web Services Description Language (WSDL)
– Describes how to invoke a service (compare with CORBA IDL).
– Can bind to SOAP, other protocols for actual invocation.
• Simple Object Access Protocol (SOAP)
– Wire protocol extension for conveying RPC calls.
– Can be carried over HTTP, SMTP.
• Web Services Inspection Language (WSIL)
– A very simple “discovery” mechanism for locating WSDL service
definitions.
Web Services in Action
• The following examples illustrate how Web
services interact through a browser
interface.
• The browser interface could easily be
changed to any other client.
Client
Browser
(1) Client
Requests
(3)
Bind and
Use Service
Service
Provider
User Interface
Server
(HTTP)
(2)
Find Service
(SOAP/HTTP)
Information
Repository
(0)
Publish Services
Service
Provider
Service
Provider
Service
Provider
Browser Interface
HTTP(S)
UI Server has stubs
for all services (data
base access, job
submission, file
transfer, etc.)
A particular server
has several
service implementations.
Backend is a database,
application code plus
operating system.
User Interface
Server + Client
Stubs
SOAP/HTTP(S)
Server plus
Service
Implementations
Local invocation, JDBC
connection or Grid Protocol
Backend
Resources
User Interface
(1) (6)
(10) (15)
Client Stubs
(2)
(11)
(5)
(4)
DB Service 1
JDBC
(3)
DB
Host 1
(7)
(9)
(14)
Job Sub/Mon
And File
Services
(8) (12)
(13)
Operating and
Queuing
Systems
Host 2
(19)
(16)
(18)
DB Service 2
JDBC
(17)
DB
Host 3
Web Services Description
Language
Defines what your service does and
how it is invoked.
WSDL Overview
• WSDL is an XML-based Interface Definition Language.
– You can define the APIs for all of your services in WSDL.
• WSDL docs are broken into four major parts:
– abstract interface definitions (including messages and data
definitions).
– protocol bindings (to SOAP, for example)
– service point locations (URLs)
• Some interesting features
– A single WSDL document can describe several versions of an
interface.
– A single WSDL doc can describe several related services.
WSDL Example: Job Submission
• The example is a simple service that can executes
local (to the server) Unix commands.
• Service implementation (in Java) has a single
method
– ExecLocal takes a single string argument (the command
to exec) and returns a 2D string array (standard out and
error).
• The WSDL maps to a Java interface in this case.
<?xml version="1.0" encoding="UTF-8"?>
<wsdl:definitions>
<wsdl:message name="execLocalCommandResponse">
<wsdl:message name="execLocalCommandRequest">
<wsdl:portType name="SJwsImp">
<wsdl:operation name="execLocalCommand" parameterOrder="in0">
<wsdl:input message="impl:execLocalCommandRequest"
name="execLocalCommandRequest"/>
<wsdl:output message="impl:execLocalCommandResponse"
name="execLocalCommandResponse"/>
</wsdl:operation>
</wsdl:portType>
<wsdl:binding name="SubmitjobSoapBinding" type="impl:SJwsImp">
<wsdlsoap:binding style="rpc" transport="http://schemas.xmlsoap.org/soap/http"/>
<wsdl:operation name="execLocalCommand">
<wsdlsoap:operation soapAction=""/>
<wsdl:input name="execLocalCommandRequest">
<wsdl:output name="execLocalCommandResponse">
</wsdl:operation>
</wsdl:binding>
<wsdl:service name="SJwsImpService">
<wsdl:port binding="impl:SubmitjobSoapBinding" name="Submitjob">
</wsdl:service>
</wsdl:definitions>
WSDL Elements I
• Types: describes custom XML data types (optional) used in
messages.
– For OO languages, types are a limited object serialization.
• Message: abstractly defines the messages that need to be
exchanged.
– Conventionally messages are used to group requests and responses.
– Each method/function in the interface contains 0-1 request and 0-1
response messages.
– Consists of part elements. Usually you need one part for each
variable sent or received. Parts can either be XML primitive types
or custom complex types.
Types for Job Submission
• Recall that the job submission service sends
a string (the command) and returns a 2D
array.
• Strings are XML Schema primitive types, so
we don’t need a special definition.
• Arrays are not primitive types. They are
defined in the SOAP schema, so we will
import that definition.
Example: WSDL types for Custom
Data Definition
<wsdl:types>
<schema
targetNamespace="http://.../GCWS/services/Submitjob"
xmlns="http://www.w3.org/2001/XMLSchema">
<import namespace="http://schemas.xmlsoap.org/soap/encoding/" />
<complexType name="ArrayOf_xsd_string">
<complexContent>
<restriction base="soapenc:Array">
<attribute ref="soapenc:arrayType" wsdl:arrayType="xsd:string[]" />
</restriction>
</complexContent>
</complexType>
<element name="ArrayOf_xsd_string" nillable="true"
type="impl:ArrayOf_xsd_string" />
</schema>
</wsdl:types>
Message Elements for Job
Submission Service
• Our service implementation has one method of the form (in
Java)
public String[] execLocalCommand(String cmd)
• This will require one “request” message and one
“response” message.
• Each message has one part:
– Request message must send the String cmd.
– Response must get back the String[] array (defined previously as a
custom type).
• If we had to pass two input variables, our “request”
message would need two part elements.
• Note the name attributes of messages are important!
Message Examples for Job
Submission Service
<wsdl:message name="execLocalCommandResponse">
<wsdl:part name="execLocalCommandReturn"
type="impl:ArrayOf_xsd_string" />
</wsdl:message>
<wsdl:message name="execLocalCommandRequest">
<wsdl:part name="in0" type="xsd:string" />
</wsdl:message>
portTypes
• portType elements map messages to operations.
• You can think of portType==class, operation==class
methods.
• Operations can contain input, output, and/or fault bindings
for messages.
• An operation may support of the following message styles:
–
–
–
–
One-way: request only
Two-way: request/response
Solicit-response: server “push” and client response
Notification: one-way server push
portType for JobSubmit
• We previously defined the messages and
types needed. Now we bind them into the
portType structure.
• PortType names are important
– Will be referenced by binding element.
• Note names of previously defined messages
are used as references in the operations.
Example WSDL Nugget
<wsdl:portType name="SJwsImp">
<wsdl:operation name="execLocalCommand"
parameterOrder="in0">
<wsdl:input message="impl:execLocalCommandRequest"
name="execLocalCommandRequest" />
<wsdl:output message="impl:execLocalCommandResponse"
name="execLocalCommandResponse" />
</wsdl:operation>
</wsdl:portType>
PortType Bindings
• portTypes are abstract interface definitions.
– Don’t say anything about how to invoke a remote
method.
• Remote invocations are defined in binding
elements.
• Binding elements are really just place holders that
are extended for specific protocols
– WSDL spec provides SOAP, HTTP GET/POST, and
MIME extension schema examples.
SOAP Bindings for JobSubmit
Service
• Note that the binding element contains a mixture of tags
from different namespaces (wsdl and wsdlsoap).
• WSDL child elements for binding element are operation,
input, and output.
• WSDLSOAP elements are from a different XML schema
(a new one, neither WSDL or SOAP).
– This is how you extend WSDL bindings: define a new schema that
gives mapping instructions from WSDL to the protocol of choice.
• The binding element name is important, will be used as a
reference by the final port binding.
<wsdl:binding name="SubmitjobSoapBinding" type="impl:SJwsImp">
<wsdlsoap:binding style="rpc"
transport="http://schemas.xmlsoap.org/soap/http" />
<wsdl:operation name="execLocalCommand">
<wsdlsoap:operation soapAction="" />
<wsdl:input name="execLocalCommandRequest">
<wsdlsoap:body encodingStyle=http://schemas.xmlsoap.org/soap/encoding/
namespace="http://.../GCWS/services/Submitjob"
use="encoded" />
</wsdl:input>
<wsdl:output name="execLocalCommandResponse">
<wsdlsoap:body encodingStyle=http://schemas.xmlsoap.org/soap/encoding/
namespace=http://.../GCWS/services/Submitjob
use="encoded" />
</wsdl:output>
</wsdl:operation>
</wsdl:binding>
SOAP Binding in Detail
<wsdlsoap:body
encodingStyle=http://schemas.xmlsoap.org/soap/encodi
ng/
namespace=http://.../GCWS/services/Submitjob
use="encoded" />
• All this really means is “encode the message by the rules in
encodingStyle and put it in the SOAP body.”
Service and Port Definitions
• So far, we have defined the class method
interfaces (portTypes) and the rules for
binding to a particular protocol.
• Port elements define how the bindings (and
thus the portTypes) are associated with a
particular server.
• The service element collects ports.
Service and Port Elements for the
Job Submission Service
<wsdl:service name="SJwsImpService">
<wsdl:port binding="impl:SubmitjobSoapBinding"
name="Submitjob">
<wsdlsoap:address
location="http://.../GCWS/services/Submitjob" />
</wsdl:port>
</wsdl:service>
Explanation
• Note the port element’s binding attribute points to
the appropriate binding element by name.
• The only purpose of the port element is to point to
a service location (a URL). This is done by
extension (SOAP in this case.)
• Ports are child elements of the service element. A
service can contain one or more ports.
– Note the value of multiple ports: a single portType may
correspond to several ports, each with a different
protocol binding and service point.
WSDL Trivia
• The schema rules allow all of the elements we have
discussed to appear zero or more times.
• A single WSDL file may contain many portTypes
(although this is not usual).
– You may want to do this to support multiple interface definitions of
a service for backward compatibility.
• Multiple ports may also be used to provide different views
of a service
– One portType defines the interface.
– Another provides access to metadata about the service.
– Yet another may define how the service interacts with other
services via notification/event systems.
• This is being exploited by OGSA.
Open Grid Services Architecture
in One Slide
• Conventions for metadata about services
– serviceData: extends portType elements to provide
additional metadata needed to describe the particular
service.
• When is the service available?
– serviceDataDescription: defines serviceData element
rules.
• Is an element a string? An int?
• How may times can it occur?
• Various supplementary portTypes
– GridService: allows you to query serviceData
– Notification ports: communicate state changes.
Simple Object Access Protocol
A message format for exchanging
structured, typed information
SOAP Basics
• SOAP is often thought of as a protocol extension
for doing RPC over HTTP.
• This is not completely accurate: SOAP is an XML
message format for exchanging structured, typed
data.
• It may be used for RPC in client-server
applications but is also suitable for messaging
systems (like JMS) that follow one-to-many (or
publish-subscribe) models.
SOAP Structure
• A SOAP message is contained in an
envelop.
• The envelop element in turn contain (in
order)
– An optional header with one or more child
entries.
– A body element that can contain one or more
child entries. These child entries may contain
arbitrary XML data.
SOAP Headers
• Headers are really just extension points where you
can include elements from other namespaces.
– i.e., headers can contain arbitrary XML.
• Header entries may optionally have a
“mustUnderstand” attribute.
– mustUnderstand=1 means the message recipient must
process the header element.
– If mustUnderstand=0 or is missing, the header element
is optional.
SOAP Body
• The SOAP body contains any number of
entries that must be immediate children.
• Body entries are really just placeholders for
arbitrary XML from some other namespace.
Example Messages
• Recall the WSDL interface for “SubmitJob”
– Sends one string command
– Returns array of strings for standard out and error.
• The envelop is decorated with a few useful namespaces:
– soapenv defines the version
– xsd is the Schema definition itself
– xsi defines some useful constants.
• The body is just an arbitrary XML fragment.
– Assumes the recipient knows what this means.
– Recipient must looks up the ExecLocalCommand operation in the
JobSubmit service and passes it one string argument.
– The ns1 namespace tells the recipient the WSDL namespace that defines
the service.
– xsi:type lets the recipient know that the arbitrary XML element in0 is in
fact a string, as defined by the XML Schema.
SOAP Request
<soapenv:Envelope
xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:xsd=http://www.w3.org/2001/XMLSchema
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soapenv:Body>
<ns1:execLocalCommand
soapenv:encodingStyle
="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:ns1
="http://.../GCWS/services/Submitjob/GCWS/services/Submitjob">
<in0 xsi:type="xsd:string">/usr/bin/csh /tmp/job.script</in0>
</ns1:execLocalCommand>
</soapenv:Body>
</soapenv:Envelope>
Example Response
• The structure is the same as the request.
• The interesting thing here is that the request
returns a 2-element array of two strings.
– Arrays not defined by XML schema
– SOAP encoding does define arrays, so use xsi:type to
point to this definition.
– <item></item> surrounds each array element.
• Note that arbitrary XML returns can likewise be
encoded this way.
– Use xsi:type to point to a schema.
SOAP Response
<soapenv:Envelope
xmlns:soapenv=http://schemas.xmlsoap.org/soap/envelope/
xmlns:xsd=http://www.w3.org/2001/XMLSchema
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soapenv:Body>
<ns1:execLocalCommandResponse
soapenv:encodingStyle=
http://schemas.xmlsoap.org/soap/encoding/
xmlns:ns1=
"http://.../GCWS/services/Submitjob/GCWS/services/Submitjob">
<execLocalCommandReturn xsi:type="soapenc:Array“
soapenc:arrayType="xsd:string[2]"
xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/">
<item></item> <item></item>
</execLocalCommandReturn>
</ns1:execLocalCommandResponse>
</soapenv:Body>
</soapenv:Envelope>
Developing Web Services
Using Apache Axis to develop Java
implementations of Web services.
Web Service Development Tools
• Web service toolkits exist for various
programming languages:
– C++,Python, Perl, various Microsoft .NET kits.
• We’ll concentrate on building Java Web
services with Apache Axis.
• Language and implementation
interoperability is addressed through WS-I.
– http://www.ws-i.org/
Apache Axis Overview
• Apache Axis is a toolkit for converting Java
applications into Web services.
• Axis service deployment tools allow you to
publish your service in a particular application
server (Tomcat).
• Axis client tools allow you to convert WSDL into
client stubs.
• Axis runtime tools accept incoming SOAP
requests and redirect them to the appropriate
service.
Developing and Deploying a Service
• Download and install Tomcat and Axis.
• Write a Java implementation
– Our SubmitJob is a simple example but services can get quite
complicated.
– Compile it into Tomcat’s classpath.
• Write a deployment descriptor (WSDD) for your service.
– Will be used by Axis runtime to direct SOAP calls.
• Use Axis’s AdminClient tool to install your WSDD file.
• That’s it.
– Axis will automatically generate the WSDL for your service.
Sample WSDD
<deployment name="Submitjob"
xmlns="http://xml.apache.org/axis/wsdd/"
xmlns:java="http://xml.apache.org/axis/wsdd/providers/java">
<service name="Submitjob" provider="java:RPC">
<parameter name="scope" value="request"/>
<parameter name="className"
value="WebFlowSoap.SJwsImp"/>
<parameter name="allowedMethods"
value="execLocalCommand"/>
</service>
</deployment>
Explanation
• Use Axis’s command-line AdminClient tool
to deploy this to the server.
• Axis will create a service called
– http://your.server/services/SubmitJob
• WSDL for service is available from
– http://your.server/services/SubmitJob?wsdl
• A list of all services is available from
– http://your.server/services
Check your Tomcat Server
for a list of deployed
services.
WSDL generated by
inspecting the Java
implementation. Can be
download from the
server.
(XML was shown in
earlier slides)
Building a Client with Axis
• Obtain the WSDL file.
• Generate client stubs
– Stubs look like local objects but really convert
method invocations into SOAP calls.
• Write a client application with the stubs
– Can be a Java GUI, a JSP page, etc.
• Compile everything and run.
Sample Java Client Code
/**Create SubmitJob client object and point to the
service you want to use */
SubmiJob sjws = new
SubmitJobServiceLocator().getSubmitjob(new
URL(http://your.server/services/SubmitJob));
/** Invoke the method as if local. */
String[] messages =
sjws.execLocalCommand(command);
Two Notes On Client Stubs
• Axis stubs convert method calls into SOAP
requests but WSDL does not require the use
of SOAP.
– Web Service Invocation Framework (WSIF) from IBM
allows flexibility of protocols. (Alek Slominski, IU)
• Client stubs introduce versioning problems.
– We are developing dynamic (stubless) clients that
construct SOAP messages by inspecting WSDL at
runtime.
Web Service URLs
• Java
– http://xml.apache.org/axis/
• XSOAP: C++ and Java toolkits for WS
– http://www.extreme.indiana.edu/xgws/xsoap/
• gSOAP: C++ SOAP toolkit
– http://www.cs.fsu.edu/~engelen/soap.html
• Python Web Services:
– http://pywebsvcs.sourceforge.net/
• Perl:
– http://www.soaplite.com/
GEM Portal
Services and user interfaces for our
earthquake modeling portal.
Solid Earth Research Virtual Observatory Grid
• A number of simulation methods for studying earthquakes are
being developed by GEM consortium including:
– Simplex, Disloc (JPL)
– Virtual California (UC-Davis)
– PARK codes (Brown)
• As codes become more robust and accepted, problems emerge:
– Need to manage information about distributed data sources:
multiple databases, sensors, simulated data.
– Need to organize, manage information about multiple code
installation sites.
– Need to simplify access to data, use of codes, and use of
visualization/analysis tools for broad range of users
– Need to link together
• NASA funded activity to develop SERVOGrid
Interoperability framework
SERVOGrid Architecture
Repositories
Federated Databases
Database
Loosely Coupled
Filters
Sensor Nets
Streaming Data
Database
Closely Coupled Compute Nodes
Analysis and
Visualization
Grid Web Service and Portlet based Portal
Architectures
• Web services are part of a broad industry and academic
initiative to build distributed computing infrastructure
around existing standards (HTTP, XML, etc).
• Web services can be used to provide lightweight service
descriptions and send messages between components in
XML.
• OGSA (Open Grid Service Architecture) builds on basic
Web Services
– Component model for Grid replacing/building on CORBA,
COM, Enterprise Javabeans etc.
• Portlets provide component model for user interfaces
Computing Portal Grid Web Services
• We have built a suite of general purpose Grid Web services
for managing distributed applications.
• Core Computing services define general purpose functions:
– Ex: job submission, file transfer, job monitoring, management of
jobs and results
– Described as a GridShell as plays same role to Grid that Shell does
for UNIX on a single machine
• Application Grid Web services include metadata about
applications.
– Built on top of core services.
– Original application NOT changed
• We have developed a toolkit that allows one to convert
general software packages into Grid Web Services and
manage application collections
User
Services
System
Services
Grid
Computing
Environments
Portal
Services
System
Services
Application
Application Metadata
Service
Middleware
System
Services
Actual Application
System
Services
System
Services
Raw (HPC)
Resources
“Core”
Grid
Database
Application Grid Web Services
• AGWS are designed to make scientific applications (i.e.
earthquake modeling codes) into Grid Resources
• AGWS services are described by two XML Schemas:
– Abstract descriptors describe application options. Used
by the application developer to deploy his/her service
into the portal.
– Instance descriptors describe particular user choices
and archive them for later browsing and resubmission.
Portals Aggregate Access to
Distributed Services
Client
Browser
(1) Client
Requests
(3)
Bind and
Use Service
Service
Provider
User Interface
Server
(HTTP)
(2)
Find Service
(SOAP/HTTP)
Information
Repository
(0)
Publish Services
Service
Provider
Service
Provider
Service
Provider
Browser Interface
JSP + Client Stubs
DB Service 1
Job Sub/Mon
And File
Services
JDBC
DB
Host 1
DB Service 2
JDBC
Operating and
Queuing
Systems
Host 2
DB
Host 3
GEM Portlets and Portal Stacks
Aggregation Portals
User facing Web
Service Ports
Application Grid Web
Services
Core Grid Services
Message Security, Information Services
• User interfaces to GEM
services (Code
Submission, Job
Monitoring, File
Management for Host X)
are all managed as
portlets.
• Users, administrators can
customize their portal
interfaces to just precisely
the services they want.
File Selector Screenshot
• Users can view files on remote hosts
– Shown is user file list for host noahsark.
– Host solar is also shown.
• Files can be uploaded or downloaded between PC and
remote host.
• Files may be crossloaded between two remote hosts.
• Same interface definition may be used to access
databases.
File management
Tabs indicate available
portlet interfaces.
Lists user files on
selected host, noahsark.
File operations include
Upload, download,
Copy, rename, crossload
Application Selection Screen Shots
• Users can select from available codes
(Simplex, Disloc, various VirtualCalifornia)
• Users also select from available hosts for
particular code (grids, noahsark, solar
shown).
• Selecting solar (Sun 64 Node E10000)
prompts user for information needed to use
PBS script generating service.
User Application Selection and Submission
Generate
script for job
submission
Select desired
application and
host
Application Administrator Interfaces
• Application Administrators deploy and
manage applications
• Update screen shows various application
and host parameters that are set by the user.
• These are used to generate user interfaces
for the selected codes.
Administer Grid Portal
Provide information
about application
and
host parameters
Select application
to edit
Futures I: General Capabilities
• System builds on Gateway Portal developed for NCSA Alliance and
DoD HPCMO. This project intends to:
• Integrate portlets from other groups in collaborations (common
interoperability framework for all tiers)
• Continue developing message-based security system (currently
available as demonstration) that signs SOAP messages.
• Integrate AGWS and Results Manager with arbitrary Schema
Wizard/Storage/Browser system used in Indiana XML newsgroups.
• Add negotiation capabilities to Web Services for version control
• Track evolving OGSA service standards and implementations.
• Add dynamic information service and monitoring capabilities for
transient Web Services.
• Distributed, ant-based tools to simplify service upgrades of
distributed servers (and simplify our lives).
Futures II: SERVOGrid
• Assist with design of common data format and
associated tools for GEM codes.
– i.e. SERVOGrid Object Model defined in XML
• Add support for service linkage (workflow) for
specific GEM applications
– e.g. Disloc<->Simplex interoperation).
• Add support for Web-based visualization with
portlet interface.
• SERVOGrid portal will be publicly available
Web Services for Computational
Web Portals
Marlon Pierce
Community Grids Lab
Indiana University
Grid Computing Environments
• GCEs are a general name for both Grid clients and
middleware.
• GCEs aim to bridge the gap between users and Grid
infrastructure developers.
• GCE-RG: Official GCE forum in the GGF.
– Geoffrey Fox, Dennis Gannon, Mary Thomas are cochairs.
• The work presented here is an outgrowth of several GGF
and workshop meetings.
• See Grid Computing: Making the Global Infrastructure a
Reality " edited by Fran Berman, Geoffrey Fox and Tony
Hey
– http://www.grid2002.org
XML and Web Services
• Web services use Internet standard protocols and
technologies for distributed computing.
• Basic components
– Wire exchange of XML messages (SOAP)
• Extensible message wrapper capable of carrying general XML, binary
data, remote procedure call instructions.
– XML descriptions of service interfaces (WSDL)
• Can express detailed description of how to invoke service
– But the architecture is loosely connected
• Web services are very loosely coupled
– Distributed development
– Distributed deployment
What is Missing?
• Transient services
– Web services are not permanent or static.
– May need to be created dynamically and registered.
– One of the issues being addressed by the Open Grid Service
Architecture/Infrastructure
• Still must address numerous other issues
–
–
–
–
–
–
–
Metadata information services.
Aggregation of web services for scientific applications.
Aggregation of clients into component environments.
Message security.
(Workflow).
(Quality of service).
(Federation of services).
Problems with Traditional Portal Architecture
•
•
Web browser
Portals accesses heterogeneous back
ends and grids through a particular
middle tier.
Most portal projects are not
interoperable
–
–
•
Middle tier software incompatible
Wide range of protocols.
–
–
Portal developers don’t have to reinvent
every single important service.
Users will have access to more services
than any one project can provide.
Users will be able to pick up the best
available implementation of a service.
Web browser
?
services
services
…
Why do we need the portal
interoperability aspects of services?
–
…
Back end resources
…
Back end resources
Portals + Web Services
Client
Browser
(1) Client
(HTTP)
Requests
(3)
Bind and
Use Service
Service
Provider
User Interface
Server
(2)
Find Service
(SOAP/HTTP)
Information
Repository
(0)
Publish Services
Service
Provider
Service
Provider
Service
Provider
Core Web Services
• Let’s get started writing
services!
• Given WSDL and SOAP,
what can you build?
• Site Dependent Services
–
–
–
–
Job Submission
File Transfer
Job Monitoring
Host Monitoring
• Site Independent Services
– Context Management
– Batch/RSL Generation
– Archiving Services
• These core services are
simple, stateless.
• Many others (from
Gannon group, for
example)
Example Core Service:
Context Management
• Common problem of portals is to store all of the metadata
associated with user sessions.
• Context Management service provides simplest possible
data model
– Context manager provides an easy interface to data trees.
– Context data nodes are defined by recursive schema that hold
optional, unbounded name/value pairs and child nodes.
• We use CM to store locations of job scripts, miscellaneous
file URIs, etc.
• CM metadata stored on file systems, XML-native
databases, ….
– Actual data may be anywhere.
• Searched with XPath queries.
Application Codes as Web Services
• Scientific applications consist of several core Web
services.
– Get files to right place, script submission instructions,
submit the job, get notified at various states.
• We need a meaningful metadata model for
applications
– Describe application-specific requirements
– Describe bindings of applications to host environments
and to Web services
Application Web Services
• (From discussions with Tomasz Haupt)
• We divide application lifecycles into several
stages:
– Abstract state: describes optional choices and
configurations that are available.
– Ready state: Specific choices are made
– Submitted: Application is running (and OGSI takes
over)
– Completed: Application is finished, but we need to
archive information about it.
AWS Schema Structure
• Two sets of XML schema:
– Application Descriptors: describe abstract state.
– Application Instance Descriptors: describe particular
instance states (ready, running, archived).
• Schema sets are arranged hierarchically
– Applications contain hosts
– Schema are designed to be pluggable
• Don’t like my queue description schema? Plug in your own.
General Metadata Web Services
(MDWS)
• Application web services are
based around schema.
• They are part of a larger class
of XML schema-based
metadata services.
• We need to automate the
deployment of these services.
• Current process is time
consuming and fragile
– An affront to any virtuous
programmer.
• Deployment Steps
– Develop XML schema
– Bind schema to data classes
(Castor)
– Write Façade classes to
simplify use of fine-grained
interface.
– Compile and deploy service
– Throw WSDL to client
– Create client stubs
– Write Web interface or GUI
client with stubs.
– Scream when schema changes.
Automating Metadata Web Services
(MDWS)
• Metadata services are extremely important, should
be very wide spread.
– Should just create your schema and go.
• Need to drastically simplify process of
redeploying services when schema are updated.
• Need to keep metadata schema creation
democratic
– Keep standards at the right level: standardize around
schemas and a few XML languages
Schema Wizard: First Steps for
Automating MDWS
• The Schema Wizard reads in XML schema and
maps these to JSP pages.
– Nice mapping between HTML visual widgets and
schema elements
• It also ties form actions to appropriate data
objects.
• Use the Schema Wizard to create a user interface
for your schema
• Use the generated User Interface to create XML
instances
SchemaWizard Architecture
Annotated XML Schema
Castor Schema Unmarshaller
Castor
Source
Generator
Web
Application
Template
JavaBeans
Castor SOM
Schema
Parser
Velocity
Templates
Java Compiler
Libraries
Classes
JSPs
XML Form Wizard created as a Web Application
XML Schema location is given to
SchemaWizard.
XML Form Wizard is generated.
XML instance is marshaled.
Next Steps for Schema Wizard
• Schema Wizard currently used to automate local (to the
server) XML message creation for publication.
– More about this in a minute
• We also want to use the SW create WSDL wizards for
metadata services
– Use Castor to represent schemas as Java classes on a remote server.
– Note that get/sets define the interface, so any schema expressible
as WSDL.
– Use WSDL (actually, convenient intermediate form) in the schema
wizard.
– Schema Wizard forms tied to classes that can discover and
dynamically invoke methods to get/set remote XML instance.
XML Metadata Nugget Management
• All web service entities are described with metadata and
given URIs
– Ex: Application Web Services define metadata about applications.
– Application instance records archive all portal sessions.
• What do we need?
– Schema wizards to simplify deployment of new schema services and
creation of valid instances.
– Federating messaging systems to deliver created nuggets.
– Distributed persistent storage
– Name based browsing and searching of XML nugget catalogs.
• Such systems should be able to handle any metadata
– Create, post, and browse application instance data, citations, news
postings, etc.
Wizard
Publish with JMS
or Narada
URI directories cataloged
with RSS, documents
retrieved on demand.
Blah blah blah
Federated data storage
(Files, XML, RDB)
Nugget Browser
Blah blah blah
Single Publish/Subscribe Layer
Computational Web Portal Stack
Aggregate Portals
User Interfaces
Application Web Services
and Workflow
Core Web Services
Message Security, Information
• Web service dream is that
core services, service
aggregation, and user
interface development
decoupled.
• Q: How do I manage all
those user interfaces?
• A: Use portlets.
Portlets in One Slide
• Portlets are pieces of (Java) code that run in the UI
server.
• Each portlet corresponds to one content provider
(local or remote).
• Portals aggregate portlets into a single, usercustomizable display.
• Jetspeed is a free, open source portlet
implementation.
• Jetspeed portlets need to be extended to handle our
requirements.
Message Level Security
Aggregate Portals
User Interfaces
Application Web Services
and Workflow
Core Web Services
Message Security, Information
• Multiple layers of
security.
• Transport level
security provides
point-to-point
protection.
• Also need message
level security for endto-end protection.
Message Signing with SAML and Kerberos
• SAML expresses security
assertions in XML.
• Demonstration Steps
– Establish both servlet session and
GSS context between the UI and
AS.
– UI signs SAML assertion and
SOAP Body message with GSS
Context’s wrap method.
– Service extracts SAML assertion
and SOAP Body message with
GSS Context’s unwrap method
from AS and verifies it.
Web Browser
HTTPS
User Interface
Server
Authentication
Service
Kerberos Client
Kerberos Server
HTTP(S)
+SOAP
+signed SAML
SOAP Service
An assertion-based authentication
service for Gateway Web Services
The authentication
service
Process the SOAP
message
Client login process
for the user
authentication
Initialize the secure
context to get the
shared key.
Client
Generate the
assertion such as
SAML, WS-security.
Unwrap the assertion.
Test the user validity.
Sign the assertion.
Unwrap the SOAP Body
message.
Add it to SOAP
Header.
Sign the SOAP Body
message.
Check
the
assertion
type such as SAML,
WS-security and the
security
mechanism
such as Kerberos, PKI.
Internet
(HTTP)
cloud
Rebuild the SOAP
message.
Add it to SOAP Body.
Process the SOAP
message.
Send the SOAP
request.
Send the SOAP response.
Acknowledgements
• Mary Thomas, Steven Mock and the TACC/SDSC
teams.
• Alliance Portal Project (ask for demo)
• Geoffrey Fox
• Ozgur Balsoy (Schema Wizard, Metadata
Management)
• Choonhan Youn (Core and Application Web
Services, SAML)
• Galip Aydin, Ahmet Topcu, Ali Kaplan, Beytullah
Yildiz (Metadata management)
Information and Downloads
• Web services: www.gatewayportal.org
• Application Metadata Schemas:
www.servogrid.org/GCWS/Schema
• Schema wizard and metadata browser:
www.xmlnuggets.org
• Email me: [email protected]
• See me at “Research in Indiana” booth.
Web Services Overview
• Web services use Internet standard protocols and
technologies (XML) to build distributed services and
applications.
• Leverages XML’s strengths
– General purpose language definition rules.
– Extensibility through namespaces as well as specific schema
elements.
– Limited inheritance
– Reusable software (parsers,Castor)
• Limitations:
– Not good for high performance messaging.
– Most pieces still being developed
Information Archival Web Service
(TACC/SDSC)
• Collects machine status info
• Uses various mechanisms for collecting info
• Queue info, job info, node info
– MDS, Ganglia, Clumon, PBS qstat,….
• All info expressed in schema-based XML
• Secure (SSL, password protected)
• Info is archived
– Some averaged
– Others are snapshots
SchemaWizard Architecture
•
•
•
SW consists of an empty Web application, the SchemaParser, and Velocity Macro
template files.
First, SW unpacks and deploys a predefined and empty Web application into a Web
server’s application repository.
A schema whose location is provided by a user is read in to create an in-memory
representation (SOM) and also to create Java files.
– SOM=Castor’s Schema Object Model
•
•
•
Java files are compiled, and binaries are placed into the new project’s classes
directory.
Each schema element is mapped to a self-contained JSP nugget.
JSP nuggets are generated from templates.
– One template for each element type (simple, complex, enumerated, unbounded,….).
– Velocity is used for convenient scripting of JSP.
•
•
The final JSP page is an aggregate of the JSP nuggets files (using <%@:include>).
Complex schema elements are mapped to JavaBeans generated from the schema
with Castor.
– Scripting templates set up the imports
XML Form Wizard
Project Page
• create a session
• initialize the
environment
• retrieve an XML
instance list
Complex Types
New
command
list
XML
instance
XML
• create a complex
type bean
• include sub
elements
• if submitted,
validate the content
Index Page
• create a root
element bean as new
or from existing XML
• include sub element
pages
• if submitted,
validate the bean
content
• if editing is
complete, generate
XML
includes
Simple Types
• if parent bean has
data, display content
for this type (or
property) using a
form element
• if form is submitted,
update the parent
bean for this type
includes
SchemaParser
Data Flow and Actions
Traverse schema for types
Castor SOM
Schema object
Collect type information, create a context
Individual types
Decide template:
JavaBeans info
Velocity context
with type info
Templates
Context,
template
Project page
Index page
Simple type
Enumerated simple type
Unbounded simple type
Complex type
Unbounded complex type
Velocity Template Engine
JSP
SchemaWizard and XML
A Mapping of XML to SchemaWizard Parts
SchemaWizard
Annotated
XML Schema
input of
SchemaParser
generates
follows
outputs
XML Instances
edited
XML Form
Wizard
SAML example
<?xml version="1.0" encoding="UTF-8"?>
<ns1:AssertionSpecifier xmlns:ns1="http://www.oasis-open.org/committees/security/docs/draft-sstc-schema-assertion-27.xsd">
<ns1:Assertion MajorVersion="1" MinorVersion="0"
AssertionID="156.56.104.10.1037385546507"
Issuer="Gateway Web Portal" IssueInstant="2002-11-15T13:39:06.507-05:00">
<ns1:Conditions NotBefore="2002-11-15T13:34:06.518-05:00" NotOnOrAfter="2002-11-15T13:49:06.518-05:00">
<ns1:AudienceRestrictionCondition>
<ns1:Audience>http://www.gatewayportal.org/agreement.xml</ns1:Audience>
</ns1:AudienceRestrictionCondition>
</ns1:Conditions>
<ns1:AuthenticationStatement
AuthenticationMethod="urn:ietf:rfc:1510" AuthenticationInstant="2002-11-15T13:39:06.558-05:00">
<ns1:Subject>
<ns1:NameIdentifier
SecurityDomain="www.gatewayportal.org" Name="cyoun"/>
<ns1:SubjectConfirmation>
<ns1:ConfirmationMethod>urn:ietf:rfc:1510</ns1:ConfirmationMethod>
<ns1:SubjectConfirmationData>A Kerberos Ticket</ns1:SubjectConfirmationData>
</ns1:SubjectConfirmation>
</ns1:Subject>
<ns1:AuthenticationLocality IPAddress="156.56.104.10" DNSAddress="grids.ucs.indiana.edu"/>
</ns1:AuthenticationStatement>
<ns1:AuthorizationDecisionStatement Resource="AccessLevel" Decision="Permit">
<ns1:Subject>
<ns1:NameIdentifier
SecurityDomain="www.gatewayportal.org" Name="cyoun"/>
<ns1:SubjectConfirmation>
<ns1:ConfirmationMethod>urn:ietf:rfc:1510</ns1:ConfirmationMethod>
<ns1:SubjectConfirmationData>A Kerberos Ticket</ns1:SubjectConfirmationData>
</ns1:SubjectConfirmation>
</ns1:Subject>
<ns1:Actions>
<ns1:Action>5</ns1:Action>
</ns1:Actions>
</ns1:AuthorizationDecisionStatement>
</ns1:Assertion>
</ns1:AssertionSpecifier>
Discovering Deployed Services
• Our model deploys core services on distributed servers.
– Need to find servers, their deployed services, WSDLs, service
points.
• This model has usual problems of distributed systems.
– Network failures, machine crashes, software failures, topological
defects.
• Information services must be dynamic, up-to-date.
– UDDI has had problems with data aging.
• Such problems can be addressed by peer-to-peer and/or
messaging technologies.
– Let’s try using NaradaBrokering for distributed events
– Narada has more extensive capabilities then used here.
– Shrideep Pallickara, www.naradabrokering.org
Integration of Security into Web Services
•
Authentication through single sign-on.
Users
– Kerberos, PKI
– Distributed ticket system
– Getting assertions about authentication,
authorization, user attribute
•
•
Authenticate
Generating
Assertions
SOAP security should be provided
through standard interfaces to specific
mechanisms.
General methods are
Signing
Assertions
– Message signing.
– Message integrity.
– Message encryption.
•
•
Kerberos, PKI are specific mechanisms.
Assertion is an XML document
describing the information about
authentication acts performed by
subjects, attributes of subjects and
authorization decisions, Created with a
specific mechanism.
Security
Mechanism
Encryption
Assertions
SOAP
HTTP
Web
Service
HTTP
Web
Service
……
……
HTTP
Web
Service
Technical resources
• Modified Apache Axis 1.0: SOAP Engine
• Security assertion
– SAML being standardized at OASIS is an
XML-based security standard for exchanging
authentication and authorization information.
– SAML schema: draft-sstc-schema-assertion27.xsd
• Kerberos: Version 5, Release 1.2.2
Bridging Between Client-Server
and Messaging Services
Browser
Peers register
themselves
to Aggregator
Blah blah blah
Tomcat
Server
Tomcat
Server
Broker
Aggregator
HTTP
Tomcat
Server
SOAP
Dynamic
User Interface
Component
Web service
request for
information
Tomcat
Server
Servers
run Narada
Notifiers
Tomcat
Server
The message structure of the SOAP
request on the client-side.
<soapenv:Envelope
xmlns:soapenv=http://schemas.xmlsoap.org/soap/envelope/
xmlns:xsd=http://www.w3.org/2001/XMLSchema
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance>
<soapenv:Header>
<ns1:Saml xmlns:ns1=http://www.gatewayportal.org/sign.xsd>
<ns1:SignedAssertiion> secure SAML Assertion message
</ns1:SignedAssertion>
<ns1:SecurityMechanism>Kerberos</ns1:SecurityMechanism>
</ns1:Saml>
</soapenv:Header>
<soapenv:Body>
<ns2:SignedBody xmlns:ns2=http://www.gatewayportal.org/signbody.xsd>
secure SOAP body message</ns2:SignedBody>
</soapenv:Body>
</soapenv:Envelope>
The client-side process
•
Convert SAML schema to Java classes
–
•
Develop utility classes for creating assertions, marshalling
them back and forth between Java and SAML.
–
•
•
•
•
•
Castor can be used to easily convert between XML and Java data
objects.
Assertion attributes filled in by the appropriate mechanism.
Login process: the authentication and getting the Kerberos
ticket.
Establish the security context with the server for getting the
shared key.
Generate user’s SAML security assertion.
Sign the user assertion and SOAP Body messages.
Rebuild the SOAP messages.
The server-side process
•
•
Establish the security context with client for getting the shared key.
Handle the SOAP message.
–
–
–
–
•
•
Unwrap the secure assertion
It checks the validity of the assertions.
–
–
–
–
•
•
Secure assertion message.
Secure body message.
Security mechanism name such as Kerberos, PKI.
Message format such as SAML, WS-security.
Issuer name
“Conditions” time limit
Subject name
Authorization for accessing resources
Unwrap SOAP body message
Rebuild the SOAP message.
Schema Wizard for MDWS
(1)
(2)
(3)
Maps Schema to
Java classes, binds
to WSDL
Retrieves WSDL
and generates specific
interface for data
Requests form
pages
Browser
Blah blah blah
HTTP
Gathers
user requests
(4)
Dynamic
User Interface
Component
SOAP
Maps entries to WSDL
expression of schema and
invokes dynamically
(no stubs)
(5)
Metadata
Service
Assigns
elements to
Castor classes
(6)
Context Manager Architecture
Client
SOAP/HTTP
Shared
WSDL
Interface
Axis Servlet
Context
Manager
Internal
Communication
Context Data
FS
XMLDB
Miscellaneous Info
• WSDL-to-Java mappings are defined by
JSR 172. See
http://www.jcp.org/en/jsr/detail?id=172.
• Other language mappings for WSDL
depend on appropriate standards body.