Transcript PowerPoint Presentation - CWSpace: Archiving Course

Technical Introduction To and
Initial Use Of the
Lightweight Network Interface
(LNI) (—to DSpace!)
William Reilly, Larry Stone — MIT Libraries
DSpace User Group Meeting
April 21, 2006 — Bergen, Norway
v. 20060421_0945
©
2006
Today's Talk
1. Context: CWSpace project
2. Web Services
•
"Lightweight Network Interface"
3. WebDAV & DSpace
•
"Technical Introduction To…"
4. MIT OpenCourseWare Submit (example)
•
"Initial Use Of…"
5. Appeal for Use Cases, Future Dev!
DSUG Bergen 2006
Lightweight Network Interface (LNI)
2
Context:
CWSpace project
DSUG Bergen 2006
Lightweight Network Interface (LNI)
3
Project Overview
CWSpace: MIT iCampus project, nearing completion
Vision: Increase
value in Institute
publication: OCW
– Also: Increase
value in Open
Source platform:
DSpace
Goal: Archive OCW to
MIT’s DSpace
– Also: Make content
available to CLEs *
* CLE = Collaborative Learning Environment
Method: Interoperability of systems
– Use Content Packaging standard
– Use Web Services standard
DSUG Bergen 2006
Lightweight Network Interface (LNI)
4
Perceived Need:
Interoperability
• Obvious observation: Higher
Ed. Technologies increasingly
seeking integration with
Library & Repository
holdings
• Not so obvious(?): Libraries
seeing their charter to include
collection, preservation,
dissemination of Teaching &
Learning materials (?)
DSUG Bergen 2006
Lightweight Network Interface (LNI)
5
Interop: What’s Available
1 of 4
• Normalized data model (out of chaos)
– "Sections" fit well to publication organization
• Normalized content files (.PDF)
– Reduced ability to disaggregate, re-purpose
DSUG Bergen 2006
Lightweight Network Interface (LNI)
6
Interop: What’s Achievable
DSUG Bergen 2006
Lightweight Network Interface (LNI)
2 of 4
7
Interop: Challenges
DSUG Bergen 2006
Lightweight Network Interface (LNI)
3 of 4
8
Interop: Possible Resolution?
DSUG Bergen 2006
Lightweight Network Interface (LNI)
4 of 4
9
Web Services:
"Lightweight Network Interface"
DSUG Bergen 2006
Lightweight Network Interface (LNI)
10
DSpace dev: LNI & PKGers
DSUG Bergen 2006
Lightweight Network Interface (LNI)
11
DSpace dev: LNI & PKGers
DSUG Bergen 2006
Lightweight Network Interface (LNI)
12
What was the Problem…?
Content Providers wanted…
Content Provider
Networked Access: Programmatically
access the DSpace object model, but not
running on the DSpace server
Basic
(Lightweight
Network Interface (LNI))
Advanced
[ Access, Deposit,
Limited Admin.
Modifications ]
[ Replace, Withdraw,
Delete, 3rd Party,
Workflow ]
• Submit
• Retrieve Item w. Metadata
• Build own U/I
• Collection Mgt, Map Items
• Specialist metadata
• Automated metadata
• Quality control checks
• Format migration …
DSUG Bergen 2006
DSpace
(possible future
LNI (or other WS!) dev.)
Lightweight Network Interface (LNI)
13
What Is the LNI?
The "Lightweight Network Interface" is:
• A remote API to control DSpace
• It's a separate module at the "application layer" —
peer to the Web U/I, can run in same Tomcat
• It's essentially WebDAV: Extensions to HTTP
– A SOAP (with WSDL) equivalent is provided
• It's a mapping: —
– a "complete and comprehensive" view of the
DSpace public object API…
(the "business logic" layer)
– …as mapped to WebDAV semantics
DSUG Bergen 2006
Lightweight Network Interface (LNI)
14
How Do I Use It?
• LNI is not for direct, interactive access *
– e.g. HTTP browser
• Instead, the LNI is accessed via an application you
develop
• Independence: Language, platform, machine
– Code: Perl, C, Java, other WebDAV libraries
– OSes: "Web Services" all about connecting different platforms
• Further "independence": choice of WebDAV or SOAP
equivalent
• Server side? Minimal — install LNI & let it run.
– Use existing DSpace authentication, e-People
DSUG Bergen 2006
Lightweight Network Interface (LNI)
15
WebDAV & DSpace:
"Technical Introduction To"
a.k.a.
Micro-Intro-Tutorial-Review…
DSUG Bergen 2006
Lightweight Network Interface (LNI)
16
WebDAV & LNI
• WebDAV = "Distributed Authoring and Versioning"
– using a Web server, rendered "DAV-able"
• WebDAV — 1999 spec. from the IETF (Internet
Engineering Task Force) RFC 2518
– http://webdav.org/specs/rfc2518.html
• LNI implementation of WebDAV:
– Compliance Level 1 (no Versioning or Locking)
– To date: No DELETE nor MKCOL; other limitations…
– …room for future development! ;^)
• LNI: DSpace model mapped to WebDAV model …
DSUG Bergen 2006
Lightweight Network Interface (LNI)
17
DSpace: Model
• DSpace digital
archive
• Differs from file
system
– Submit URI not
known
– Workspace
– Workflow (optional)
– Metadata
– Bundles
– Handle to URL
resolution (& vice
versa)
DSUG Bergen 2006
Lightweight Network Interface (LNI)
18
WebDAV: Model, Basics
DAV = "Distributed Authoring and Versioning"
URL
DSUG Bergen 2006
Lightweight Network Interface (LNI)
• Each
Resource
named by
a URL
• Properties
are "name
value"
pairs
• Properties
are
extensible
• Property
values can
be found,
and
modified
19
WebDAV: Model, Fuller
•
•
•
DSUG Bergen 2006
Lightweight Network Interface (LNI)
Resources
can be
Collections
or leaf
nodes
Collections
can nest
Everything
can be
discovered
descending
from the
DAV
server's root
'/' resource
20
DSpace Mapped to WebDAV
DSUG Bergen 2006
Lightweight Network Interface (LNI)
21
DSpace Translated to WebDAV
• When you have a Handle (e.g. from
other services like SRW/U, OAIPMH), then use LNI "lookup"
functionality to convert it to LNI
WebDAV URL.
• To get a Handle (given a LNI URL),
do PROPFIND on that URL and
read the "Handle" property value.
DSUG Bergen 2006
Lightweight Network Interface (LNI)
Handle
LNI URL
22
DSpace Implemented as WebDAV
1. Properties
– WebDAV
– Custom for DSpace
2. DSpace LNI URLs
3. WebDAV Methods
DSUG Bergen 2006
Lightweight Network Interface (LNI)
23
Properties: WebDAV
•
•
•
•
•
•
•
DAV:creationdate
DAV:displayname
DAV:getcontentlength
DAV:getcontenttype
DAV:getlastmodified
DAV:resourcetype
…
Akin to
file system
properties,
with add'l.
metadata
http://www.webdav.org/specs/rfc2518.html
DSUG Bergen 2006
Lightweight Network Interface (LNI)
24
Properties: Custom DSpace (1 of 3)
• ALL DSpace Objects:
– dspace:type
• "BITSTREAM", "ITEM", "COMMUNITY" etc.
– DAV:resourcetype
• "COLLECTION" etc.
– dspace:displayname
– dspace:current-user-privilege-set
DSUG Bergen 2006
Lightweight Network Interface (LNI)
25
Properties: Custom DSpace (2 of 3)
• Community DSpace Objects
– dspace:logo
– dspace:short_description
– dspace:side_bar_text
– dspace:handle
–…
DSUG Bergen 2006
Lightweight Network Interface (LNI)
26
Properties: Custom DSpace (3 of 3)
• Bitstream DSpace Objects
– DAV:getcontentlength = getSize()
– dspace:format = getFormat().getID()
– dspace:sequence_id = getSequenceID()
–…
• Many more DSpace Objects (some 10)
– With many more Properties (some 87)
http://wiki.dspace.org/LightweightNetworkInterface
DSUG Bergen 2006
Lightweight Network Interface (LNI)
27
URLs: DSpace LNI (1 of 3)
• Site
– http://uni.edu/dspace/dav
• Community
– http://uni.edu/dspace/dav/dso_1721.1$46
• Collection
– http://uni.edu/dspace/dav/dso_1721.1$3549
DSUG Bergen 2006
Lightweight Network Interface (LNI)
28
URLs: DSpace LNI (2 of 3)
• Item
– http://uni.edu/dspace/dav/dso_1721.1$3549/
dso_1721.1$5543
– http://uni.edu/dspace/dav/dso_1721.1$5543
• Bitstream
– http://uni.edu/dspace/dav/dso_1721.1$5543/
bitstream_13
– http://uni.edu/dspace/dav/dso_1721.1$5543/
bitstream_13.pdf
DSUG Bergen 2006
Lightweight Network Interface (LNI)
29
URLs: DSpace LNI (3 of 3)
Nota Bene
The LNI URLs for DSpace objects are:
• only for use with the LNI.
– They are therefore not to be used:
• as persistent URLs (e.g. bookmark, store for later use)
• as interactive URLs (e.g. browser)
• opaque — you must use the LNI to resolve to
Handles
• Therefore please to not:
• Roll your own LNI URLs to send
• Infer Handles & Etc. from LNI URLs received
DSUG Bergen 2006
Lightweight Network Interface (LNI)
30
WebDAV Methods in LNI
• PROPFIND - very powerful request with multiple
functions:
– list names of properties
– return property values
– recurse through 1 or more levels of a "collection" resource.
• PROPPATCH - changes value of or deletes properties on
one resource.
• GET - retrieves contents of a resource
• PUT - replace or add a new resource to a collection
• COPY - map a resource ("Item") to a different Collection.
• As noted, equivalent SOAP methods are provided
• Not Implemented: Locking, Versioning.
DSUG Bergen 2006
Lightweight Network Interface (LNI)
31
Initial Use:
OpenCourseWare "Submit"
DSUG Bergen 2006
Lightweight Network Interface (LNI)
32
Current DSpace (pre-LNI)
DSpace
Content Provider, currently
•
Batch import (& export)
•
End user Web U/I
Batch Importer
Web U/I
Workflow
•
dspace-admin Web U/I
Adm
Non Std.
Non Web
Web, but
Non Automated
DSpace Object API
OCW Comm.
•
SRW, OAI-PMH
"Biology (7)"
"Physics (8)"
Item
DSUG Bergen 2006
Lightweight Network Interface (LNI)
33
DSpace with LNI
DSpace
Content Provider using LNI:
•
LNI SOAP end-point URL
•
LNI WebDAV URLs
DSpaceLNI.class
DSpace LNI WebDAV
•
dspace-admin (partial)
- Web.
- Stds.
- Batchable.
DSpace Object API
•
SRW, OAI-PMH
–
•
(Not replicated in LNI)
OCW Comm.
Still available:
–
–
Batch import (& export)
End user Web U/I
"Biology (7)"
"Physics (8)"
Item
DSUG Bergen 2006
Lightweight Network Interface (LNI)
34
Submission: Overview
DSpace
OpenCourseWare Client
Needs 3 Values:
1.
2.
3.
DSpaceLNI.class
LNI SOAP end-point URL
http://userid:password@dsp
ace.mit.edu/dspacelni/lni/DSpaceLNI
DSpace LNI WebDAV
Lang. Indep.
Platform Indep.
DSpace Object API
Handle for OCW
Community =
hdl:1721.1/125
OCW Comm.
1721.1/125
Dept. Name (MIT "Course"
Number) for Course to be
submitted = "Physics (8)"
"Biology (7)"
"Physics (8)"
Item
DSUG Bergen 2006
Lightweight Network Interface (LNI)
35
Submission: 1. Lookup
Request: OCW Comm. HANDLE
LNI lookup() =
http://dspace.mit.edu/dspacelni/dav/lookup/handle/1721.1/1
25
DSpace
DSpaceLNI.class
DSpace LNI WebDAV
DSpace Object API
Response: OCW Comm. LNI URL
http://dspace.mit.edu/dav/dso_
1721.1$125
OCW Comm.
1721.1/125
dso_1721.1$125
Lookup
"Biology (7)"
dso_1721.1$197
"Physics (8)"
dso_1721.1$253
DSUG Bergen 2006
Lightweight Network Interface (LNI)
36
Submission: 2. Collections
Request: PROPFIND on LNI URL for children
DSpace Collections' "displaynames":
DSpace
DSpaceLNI.class
<propfind>
<prop>
<DAV:displayname/>
</prop>
</propfind>
DSpace LNI WebDAV
DSpace Object API
Response: MULTISTATUS
<multiStatus>
<resource>
<href URI="dso_1721.1$197"/>
<displayname>Biology (7)</displayname>
</resource>
<resource>
<href URI="dso_1721.1$253"/>
<displayname>Physics (8)</displayname>
</resource>...
DSUG Bergen 2006
OCW Comm.
1721.1/125
dso_1721.1$125
"Biology (7)"
dso_1721.1$197
Iterate…
"Physics (8)"
dso_1721.1$253
Lightweight Network Interface (LNI)
37
Submission: 3. Submit (PUT)
Request: PUT to DSpace Collection LNI
URL, including specification of
Packager Type (IMS-CP):
PUT
/dspace/dav/dso_1721.1$253?packa
ge=OCW-IMSCP
....package contents in body...
Response: LNI URL for DSpace Item in
"Location" Header:
HTTP/1.1 201 OK
Location:
http://dspace.mit.edu/dspace/dav/dso
_1721.1$836
....other headers....
DSUG Bergen 2006
DSpace
DSpaceLNI.class
DSpace LNI WebDAV
DSpace Object API
OCW Comm.
1721.1/125
dso_1721.1$125
"Physics (8)"
dso_1721.1$253
Item
dso_1721.1$836
Lightweight Network Interface (LNI)
Ingest
38
Submission: 4. Item Handle
Request: PROPFIND on DSpace Item LNI
URL for the DSpace Handle:
<propfind>
<prop>
<dspace:handle/>
DSpaceLNI.class
DSpace LNI WebDAV
</prop>
</propfind>
DSpace Object API
Response: Properties Requested
=== PROPERTIES Successfully returned:
dspace:handle = "hdl:1721.1/836"
OCW Content Mgt. System
Stores DSpace Handle
SQL Server Table: hdl:1721.1/836
DSUG Bergen 2006
DSpace
OCW Comm.
1721.1/125
dso_1721.1$125
"Physics (8)"
dso_1721.1$253
Item
dso_1721.1$836
1721.1/836
Lightweight Network Interface (LNI)
Reverse
Lookup
39
Submission: Summary
In this 4-step OpenCourseWare
submission example, we've
seen:
•
SOAP endpoint URL
•
DSpace Handles
•
DSpace LNI URLs
•
LNI Lookup() function
•
WebDAV Properties
•
WebDAV Methods
•
PROPFIND (LNI request)
•
MULTISTATUS (LNI response)
•
LNI "Reverse Lookup"
•
WebDAV Headers (Location;
Success/Fail)
DSUG Bergen 2006
DSpace
DSpaceLNI.class
DSpace LNI WebDAV
DSpace Object API
OCW Comm.
1721.1/125
"Physics (8)"
1721.1/253
Item
1721.1/836
Lightweight Network Interface (LNI)
40
Subsequent Use:
[ your use case here ]
DSUG Bergen 2006
Lightweight Network Interface (LNI)
41
LNI with Other Protocols
• Search
– SRU/W { + LNI }
• Dublin Core metadata
– OAI-PMH { + LNI }
– METS Packager (DC only) { using LNI }
DSUG Bergen 2006
Lightweight Network Interface (LNI)
42
LNI Security Considerations
• Security
– SSL (see wiki page)
– Stateless
– E-Person userid & pw with each invocation
• Authorization
– E-Person
– <DAV:current_user_privilege_set>
– Area for further development!
DSUG Bergen 2006
Lightweight Network Interface (LNI)
43
Code Status
• http://wiki.dspace.org/LightweightNetworkInterface_2fClient
Downloads
/!\ This code should not be considered stable. It is subject to change at
any time, and may not work with the current development or
released version of DSpace.
Use it for exploration and evaluation only. There are not yet any final
plans for releasing it; much depends on the course of the
AddOnMechanism project.
•
Patch to DSpace 1.4alpha source (28-Mar-06)
•
New files to add to DSpace 1.4 alpha (28-Mar-06)
•
Pre-built client library JAR and WSDL -- not needed when
building from source.
DSUG Bergen 2006
Lightweight Network Interface (LNI)
44
Future Development
• Other DSpace objects
– Bitstream Format Registry
– Metadata Registry …
• Other WebDAV methods
– PUT (for Replacement)
– DELETE
– MKCOL …
• Other Use Cases, Needs
–Basic, Advanced …
DSUG Bergen 2006
Lightweight Network Interface (LNI)
45
Questions, & Etc.
Lightweight Network Interface (LNI)
William Reilly, Larry Stone — MIT Libraries
http://wiki.dspace.org/LightweightNetworkInterface
http://cwspace.mit.edu/docs/ProjectMgt/Reports/DSpaceUserGroupBergen2006/MIT-LNI-DSUG-2006.ppt.htm
DSpace User Group Meeting
April 21, 2006 — Bergen, Norway
©
2006
Abstract
http://dsug2006.uib.no/index.php?p=pres#william_reilly
The "Lightweight Network Interface" (LNI) is a work in progress to provide (yet) another way to gain networked access to
the DSpace application-level API. Initially developed to support the requirement that "Web Services" be used on the
CWSpace project (archiving MIT's OpenCourseWare (OCW) to DSpace), the ensuing vibrant (!) discussion
regarding possible technical approaches (SOAP, RESTful, WebDAV, XML over HTTP) led us to define some high
level goals for how this new interface ought to be shaped (e.g. platform-neutral; based on mature standards; readily
extensible; work with (not replicate) existing remote APIs (SRW, OAI-PMH); comprehensive view of DSpace model;
etc.).
Dubbed the "Lightweight" network interface, the intent has been to largely adopt the robust and proven protocols (and
"verbs") of WebDAV and HTTP, and to establish a proposed mapping of WebDAV's Resource-centric view onto the
DSpace object model. (Note that a set of SOAP methods has also been developed on top of the WebDAV
functionality, such that in fact either approach is supported.) The details of the resulting proposed API were posted
on a lengthy page to the DSpace Wiki; a "smoketest" Java client to the LNI was posted to the CWSpace Wiki (along
with detailed instructions on how to work with SSL and the LNI).
This presentation introduces the technology, the API, the rationale for the approach, and a discussion of the mapping to
the DSpace data model as well as example uses of the LNI to DSpace (e.g. submit; disseminate; copy to another
collection; list collections; etc.).
Closely related DSpace development activity of the Plugin Manager, Packager Plugins, and Crosswalk Plugins will be
addressed briefly as well, as these are used in conjunction with the LNI on the CWSpace project for submission
(and dissemination) of IMS Content Packages of OCW content (courseware websites).
DSUG Bergen 2006
Lightweight Network Interface (LNI)
47