Transcript Document
Tools and Components of the HUMBOLDT Framework
Dr. Christine Giger
(Swiss Federal Institute for Technology and Giger GeoIT) HUMBOLDT Scientific Manager and Member of the Executive Board Swiss National Contact Point for INSPIRE Athens, Greece, March 28, 2011 HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
1
Key components Scenarios • Specifying user needs • Software testing Humboldt tools • Open source • Based on standards Training • E-learning • Dissemination
HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
2
The HUMBOLDT Framework HUMBOLDT Tools GI data integrator/ GIS developer
specify target HUMBOLDT GeoModel Editor configure/store model
End user of geodata or spatial information
Desktop GIS or GIS-Client specify schema translation
GI data integrator GI data integrator/ GIS developer
specify workflow HUMBOLDT Alignment Editor (HALE) HUMBOLDT Workflow Design and Construction Service provide data/ models Database access Database/ Repository provide transformation specification Provide workflow definition HUMBOLDT Mediator Service involve mediator for process execution or directly access single services execute services HUMBOLDT Conceptual Schema Transformer HUMBOLDT Edge Matching Service HUMBOLDT Coordinate Transformation Service HUMBOLDT Language Transformation Service external services
HUMBOLDT Services
HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
3
The HUMBOLDT Framework HUMBOLDT Tools GI data integrator/ GIS developer
specify target HUMBOLDT GeoModel Editor configure/store model
End user of geodata or spatial information
Desktop GIS or GIS-Client specify schema translation
GI data integrator GI data integrator/ GIS developer
specify workflow HUMBOLDT Alignment Editor (HALE) HUMBOLDT Workflow Design and Construction Service provide data/ models Database access Database/ Repository provide transformation specification Provide workflow definition HUMBOLDT Mediator Service involve mediator for process execution or directly access single services execute services HUMBOLDT Conceptual Schema Transformer HUMBOLDT Edge Matching Service HUMBOLDT Coordinate Transformation Service HUMBOLDT Language Transformation Service external services
HUMBOLDT Services
HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
4
Creation of Target
Defining the information product (target schema, SRS, spatial extent, …) to which the processed data needs to be transformed
Target
5 Creation of target
User knowledge
HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
The HUMBOLDT GeoModel Editor
UML editor developed to
support the process of data specification
Main goals:
Easy-to-use
for application experts Collecting all required
information
on the data Providing
formal representation
of the information in order to feed it into a harmonisation process Implemented on a framework (Eclipse) Usage of a “spatial UML” (HUMBOLDT modelling language) Producing/providing a graphical and a textual representation of the data model
basic data model
containing basic spatial data types Support of
vertical mapping
: serialisation possible to transfer standards or other representation (e.g. XMI, GML, ISO19131, ...) HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
6
HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
7
The HUMBOLDT Framework HUMBOLDT Tools GI data integrator/ GIS developer
specify target HUMBOLDT GeoModel Editor configure/store model
End user of geodata or spatial information
Desktop GIS or GIS-Client specify schema translation
GI data integrator GI data integrator/ GIS developer
specify workflow HUMBOLDT Alignment Editor (HALE) HUMBOLDT Workflow Design and Construction Service provide data/ models Database access Database/ Repository provide transformation specification Provide workflow definition HUMBOLDT Mediator Service involve mediator for process execution or directly access single services execute services HUMBOLDT Conceptual Schema Transformer HUMBOLDT Edge Matching Service HUMBOLDT Coordinate Transformation Service HUMBOLDT Language Transformation Service external services
HUMBOLDT Services
HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
8
Process step: Schema to Schema Mapping
?
Source Target 9 Input for transformation (mapping rules) User knowledge HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
The HUMBOLDT Alignment Editor (HALE)
HUMBOLDT Alignment Editor is a tool for geodomain application experts and GI experts, especially data custodians.
It provides means to
create executable mappings on the conceptual level
and to derive the schema transformation on the logical level .
It provides the means to ensure the validity of a mapping on the base of the source and target schema as well as on the base of the source and a reference data set.
It uses schemas created in the HUMBOLDT GeoModel Editor or other Modelling Tools.
It uses the Ontology Mapping Language (OML) to formalize the mappings
Output schema mapping transformed in GML
HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
10
11
HALE
workflow: 1.
2.
3.
4.
Schemas Import (Shapefiles, WFS and GML Application Schemas) Creation of mappings between classes, their attributes and relations. Storing the mapping description either locally or in the HUMBOLDT Model Repository Output schema mapping transformed in GML HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
12
HALE functions
rename attributes change geometry datatypes fill an attribute by values (a given value, numerical expression, NULL) identifiers (specification of the key attribute, geographic name) replace a date replace strings create new spatial objects INSPIRE functions HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
13
Schema mapping operations
1. Filtering : conditional statements applied to source data to filter features (extract sub-sets) Target Schema (INSPIRE HY) Source Schema (VA)
Class „Gewaesser“ Class „Watercourse“ Att. „LEVEL“, values (selected) Definition
Fluss, Bach ( > 20 m)
(river, rivulet)
Fluss, Bach (5-20 m) Fluss, Bach ( < 5 m) Wasserleitung (unterirdisch)
(pipe)
See, Teich (> 10 km)
(lake, pond)
Sumpf, Moorboden
(marsh, bog)
...
Value
1 2 3 5 11 19 ...
Target Schema (INSPIRE HY)
Class „Pipe“
Target Schema (INSPIRE HY)
Class „StandingWater“
Target Schema (INSPIRE HY)
Class „Wetland“
HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
14
Schema mapping operations
2. Reclassification of attribute values Source Schema (BY)
Class „Gewaesser“ Att. „Widmung“, values (selected) Definition
1. Ordnung (Bundeswasserstraße)
(1st order under federal law)
1. Ordnung (Landesgewässer)
(1st order under state law)
Gewässer 2. Ordnung
(2nd order)
Gewässer 3. Ordnung
(2nd order)
Value
1501 1502 1503 1504 Target Schema (INSPIRE HY)
Class „Watercourse“ Att. „waterCourseHierarchy “, values
1st 2nd 3rd 4th 5th other HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
15
Schema mapping operations
3. Renaming of feature classes or attributes Source Schema (VA)
Class „Gewaesser“ Att. „NAME“ (string)
Target Schema (INSPIRE HY)
Class „Watercourse“ Att. „geographicalName/spelling/text“ (CharacterString)
4.
Merge / split of features or attributes Source Schema (XY)
Class „Building“ Att. „NameOfStreet“ (string) Att. „HouseNr“ (string) Att. „City“ (string)
Target Schema (XY)
Class „Building“ concatena tion Att. „Address“ (string)
5.
Change of attribute order HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
16
17
Schema mapping operations
6. Type conversions
Spatial types
– Spaghetti lines to Network – MultiLineString to LineString – Line/Polygon to Point (centroid) – Point/Line to Polygon (buffer) – Polygon to Line (skeleton/centreline)
Other types
– Integer to float – String to numeric Example from German watercourses: < 12 m wide: line > 12 m wide: polygon HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
18
Schema mapping operations
7 Value conversions
Spatial conversions
– – Generalisation, e.g. simplification Conflation (but see: Merge)
Units of measurement
8 Augmentation ’ Deriving values for target schema properties missing in source schema – based e.g. on values of other properties in source schema (ex. using interpolation) Adding derived spatial properties (e.g. centroid of a polygon) Filling in default property values in target schema HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
The HUMBOLDT Framework HUMBOLDT Tools GI data integrator/ GIS developer
specify target HUMBOLDT GeoModel Editor configure/store model
End user of geodata or spatial information
Desktop GIS or GIS-Client specify schema translation
GI data integrator GI data integrator/ GIS developer
specify workflow HUMBOLDT Alignment Editor (HALE) HUMBOLDT Workflow Design and Construction Service provide data/ models Database access Database/ Repository provide transformation specification Provide workflow definition HUMBOLDT Mediator Service involve mediator for process execution or directly access single services execute services HUMBOLDT Conceptual Schema Transformer HUMBOLDT Edge Matching Service HUMBOLDT Coordinate Transformation Service HUMBOLDT Language Transformation Service external services
HUMBOLDT Services
HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
19
WorkFlow Design and Construction Service
Allow users to register processes (WPS) to the system Allow users to visually compose the workflow graph out of geoprocessing components and data sources Manual Workflow Definition, Automated Execution Exports such workflows in different workflow dialects via a WSDL / SOAP Interface 20 HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
WorkFlow Design and Construction Service
Analysis of input data sets
to determine harmonisation requirements
Automated combination
of processing services into
executable workflows
Definition of “Basic Workflows” via Workflow Designer 21 HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
Workflow Design and Construction Service
22 HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
The HUMBOLDT Framework HUMBOLDT Tools GI data integrator/ GIS developer
specify target HUMBOLDT GeoModel Editor configure/store model
End user of geodata or spatial information
Desktop GIS or GIS-Client specify schema translation
GI data integrator GI data integrator/ GIS developer
specify workflow HUMBOLDT Alignment Editor (HALE) HUMBOLDT Workflow Design and Construction Service provide data/ models Database access Database/ Repository provide transformation specification Provide workflow definition HUMBOLDT Mediator Service involve mediator for process execution or directly access single services execute services HUMBOLDT Conceptual Schema Transformer HUMBOLDT Edge Matching Service HUMBOLDT Coordinate Transformation Service HUMBOLDT Language Transformation Service
HUMBOLDT Services
external services 23
Mediator Service
Download Service that encapsulates the capability to execute predefined transformation workflows Execution of workflows retrieved from the HUMBOLDT Workflow Repository Service Ability to read data from Web Feature Services and Web Coverage Services and to feed into transformation services Invoke transformation processes via several standardised interfaces Ability to work with internal transformation services and external transformation services HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
24
The HUMBOLDT Framework HUMBOLDT Tools GI data integrator/ GIS developer
specify target HUMBOLDT GeoModel Editor configure/store model
End user of geodata or spatial information
Desktop GIS or GIS-Client specify schema translation
GI data integrator GI data integrator/ GIS developer
specify workflow HUMBOLDT Alignment Editor (HALE) HUMBOLDT Workflow Design and Construction Service provide data/ models Database access Database/ Repository provide transformation specification Provide workflow definition HUMBOLDT Mediator Service involve mediator for process execution or directly access single services execute services HUMBOLDT Conceptual Schema Transformer HUMBOLDT Edge Matching Service HUMBOLDT Coordinate Transformation Service HUMBOLDT Language Transformation Service
HUMBOLDT Services
external services 25
Process step: Execution of transformation services
Source Executed technical processes Schema Transf.
Edge Matching … Target 26 Analysis of sources Input for transformation (mapping rules) Creation of target User knowledge HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
HUMBOLDT Web Processing Services
Conceptual Schema Transformer
. Web Processing Service for transforming data from one application schema to another
Edge Matching Service
. Web Processing Service implementation of a service that aligns edges and points of vector geometries so that they will be gapless
Coordinate Transformation Service
. WPS implementation of a service that allows to transform coordinates between various geographic reference systems
Language Transformation Service
. Web Processing Service that is capable of transforming/translating all information that becomes visible to a user from one language to another 27 HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
The HUMBOLDT Conceptual Schema Transformer (CST)
Transformation of source data from source schema to target schema ‘Executes’ the schema mappings made in HALE. (Now integrated in the LAST version of HALE) Java platform Makes use of GeoTools libraries Test with OGC CQL (Common Query Language) for filtering and value conditions The CST can be used via request to a Web Processing Service (WPS) Or as part of a data harmonisation workflow chained with other HUMBOLDT services 28 HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
The HUMBOLDT Edge Matching Service (EMS)
WPS implementation Aligns LineStrings and Polygons of vector geometries so that they will be gapless and with no overlaps Three modes of operation:
CleanDataset:
when only one dateset is provided, then the gaps and overlaps will be removed.
Align-to-Reference:
all candidate data sets will be transformed using the reference dataset so that points are moved up to a maximum distance also provided as input.
Distribute-Errors:
In this case, there is no reference data set that can be used as "ground truth", therefore all geometries will be transformed . No point or edge will be moved further than a client-specified amount. 29
EMS Clean Dataset Process – Results
Input: 30
EMS Clean Dataset – Results
Output: 31
Further HUMBOLDT Transformation Services
Coordinate Transformation Service Implemented and in use Language Transformation Service Early implementation Multiple Representation Merging Service Prototype implementation available HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
32
HUMBOLDT Developer Community
Download software Access documentation Capture requirements Report bugs Progress tracing Discuss issues, build a knowledge base of solutions HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
33
HUMBOLDT Open Source Software http://www.community.esdi-humboldt.eu/
HUMBOLDT Model Editor
For the creation of UML application schemas
HUMBOLDT Alignment Editor
Definition of conteptual schema transformation
Workflow Service Service Composition for handling transformation needs Mediator Service
Execution of transformation chains
Transformation Service
Exposed as Web Processing Services (e.g. CTS, Edge Matching)
HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
34
Who is the Community?
You!
The users and developers in the HUMBOLDT scenarios GENESIS: Sharing of Methodology GS-Soil: INSPIRE harmonisation and transformation, tools reuse ESDIN: HALE and CST dissemination, feedback Geoland 2: establish collaboration agreement Plan4all: NDA for document exchange for WP2 results in preparation NatureSDI+: HUMBOLDT training planned eSDI-Net+: Networking, joint meetings GIGAS: Contributions to the Technology Watch activities INSPIRE Data Specification Drafting Teams Snowflake, 1spatial, FME, … HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
35
Thank you for your attention!
HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu
36