Document

Transcript Document

Tools and Components of the HUMBOLDT Framework

Dr. Christine Giger

(Swiss Federal Institute for Technology and Giger GeoIT) HUMBOLDT Scientific Manager and Member of the Executive Board Swiss National Contact Point for INSPIRE Athens, Greece, March 28, 2011 HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu

Key components Scenarios • Specifying user needs • Software testing Humboldt tools • Open source • Based on standards Training • E-learning • Dissemination

The HUMBOLDT Framework HUMBOLDT Tools GI data integrator/ GIS developer

specify target HUMBOLDT GeoModel Editor configure/store model

End user of geodata or spatial information

Desktop GIS or GIS-Client specify schema translation

GI data integrator GI data integrator/ GIS developer

specify workflow HUMBOLDT Alignment Editor (HALE) HUMBOLDT Workflow Design and Construction Service provide data/ models Database access Database/ Repository provide transformation specification Provide workflow definition HUMBOLDT Mediator Service involve mediator for process execution or directly access single services execute services HUMBOLDT Conceptual Schema Transformer HUMBOLDT Edge Matching Service HUMBOLDT Coordinate Transformation Service HUMBOLDT Language Transformation Service external services

HUMBOLDT Services

The HUMBOLDT Framework HUMBOLDT Tools GI data integrator/ GIS developer

specify target HUMBOLDT GeoModel Editor configure/store model

End user of geodata or spatial information

Desktop GIS or GIS-Client specify schema translation

GI data integrator GI data integrator/ GIS developer

HUMBOLDT Services

Creation of Target

Defining the information product (target schema, SRS, spatial extent, …) to which the processed data needs to be transformed

Target

5 Creation of target

User knowledge

The HUMBOLDT GeoModel Editor

   UML editor developed to

support the process of data specification

Main goals: 

Easy-to-use

for application experts  Collecting all required

information

on the data  Providing

formal representation

of the information in order to feed it into a harmonisation process Implemented on a framework (Eclipse) Usage of a “spatial UML” (HUMBOLDT modelling language)  Producing/providing a graphical and a textual representation of the data model 

basic data model

containing basic spatial data types  Support of

vertical mapping

: serialisation possible to transfer standards or other representation (e.g. XMI, GML, ISO19131, ...) HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu

The HUMBOLDT Framework HUMBOLDT Tools GI data integrator/ GIS developer

specify target HUMBOLDT GeoModel Editor configure/store model

End user of geodata or spatial information

Desktop GIS or GIS-Client specify schema translation

GI data integrator GI data integrator/ GIS developer

HUMBOLDT Services

Process step: Schema to Schema Mapping

Source Target 9 Input for transformation (mapping rules) User knowledge HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu

The HUMBOLDT Alignment Editor (HALE)

HUMBOLDT Alignment Editor is a tool for geodomain application experts and GI experts, especially data custodians.

It provides means to

create executable mappings on the conceptual level

and to derive the schema transformation on the logical level .

It provides the means to ensure the validity of a mapping on the base of the source and target schema as well as on the base of the source and a reference data set.

It uses schemas created in the HUMBOLDT GeoModel Editor or other Modelling Tools.

It uses the Ontology Mapping Language (OML) to formalize the mappings

Output schema mapping transformed in GML

HALE

workflow: 1.

Schemas Import (Shapefiles, WFS and GML Application Schemas) Creation of mappings between classes, their attributes and relations. Storing the mapping description either locally or in the HUMBOLDT Model Repository Output schema mapping transformed in GML HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu

HALE functions

rename attributes change geometry datatypes fill an attribute by values (a given value, numerical expression, NULL) identifiers (specification of the key attribute, geographic name) replace a date replace strings create new spatial objects INSPIRE functions HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu

Schema mapping operations

1. Filtering : conditional statements applied to source data to filter features (extract sub-sets) Target Schema (INSPIRE HY) Source Schema (VA)

Class „Gewaesser“ Class „Watercourse“ Att. „LEVEL“, values (selected) Definition

Fluss, Bach ( > 20 m)

(river, rivulet)

Fluss, Bach (5-20 m) Fluss, Bach ( < 5 m) Wasserleitung (unterirdisch)

(pipe)

See, Teich (> 10 km)

(lake, pond)

Sumpf, Moorboden

(marsh, bog)

...

Value

1 2 3 5 11 19 ...

Target Schema (INSPIRE HY)

Class „Pipe“

Target Schema (INSPIRE HY)

Class „StandingWater“

Target Schema (INSPIRE HY)

Class „Wetland“

Schema mapping operations

2. Reclassification of attribute values Source Schema (BY)

Class „Gewaesser“ Att. „Widmung“, values (selected) Definition

1. Ordnung (Bundeswasserstraße)

(1st order under federal law)

1. Ordnung (Landesgewässer)

(1st order under state law)

Gewässer 2. Ordnung

(2nd order)

Gewässer 3. Ordnung

(2nd order)

Value

1501 1502 1503 1504 Target Schema (INSPIRE HY)

Class „Watercourse“ Att. „waterCourseHierarchy “, values

Schema mapping operations

3. Renaming of feature classes or attributes Source Schema (VA)

Class „Gewaesser“ Att. „NAME“ (string)

Target Schema (INSPIRE HY)

Class „Watercourse“ Att. „geographicalName/spelling/text“ (CharacterString)

Merge / split of features or attributes Source Schema (XY)

Class „Building“ Att. „NameOfStreet“ (string) Att. „HouseNr“ (string) Att. „City“ (string)

Target Schema (XY)

Class „Building“ concatena tion Att. „Address“ (string)

Schema mapping operations

6. Type conversions  

Spatial types

– Spaghetti lines to Network – MultiLineString to LineString – Line/Polygon to Point (centroid) – Point/Line to Polygon (buffer) – Polygon to Line (skeleton/centreline)

Other types

– Integer to float – String to numeric Example from German watercourses: < 12 m wide: line > 12 m wide: polygon HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu

Schema mapping operations

7 Value conversions  

Spatial conversions

– – Generalisation, e.g. simplification Conflation (but see: Merge)

Units of measurement

8 Augmentation ’  Deriving values for target schema properties missing in source schema – based e.g. on values of other properties in source schema (ex. using interpolation)   Adding derived spatial properties (e.g. centroid of a polygon) Filling in default property values in target schema HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu

The HUMBOLDT Framework HUMBOLDT Tools GI data integrator/ GIS developer

specify target HUMBOLDT GeoModel Editor configure/store model

End user of geodata or spatial information

Desktop GIS or GIS-Client specify schema translation

GI data integrator GI data integrator/ GIS developer

HUMBOLDT Services

WorkFlow Design and Construction Service

Allow users to register processes (WPS) to the system Allow users to visually compose the workflow graph out of geoprocessing components and data sources Manual Workflow Definition, Automated Execution Exports such workflows in different workflow dialects via a WSDL / SOAP Interface 20 HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu

WorkFlow Design and Construction Service

  

Analysis of input data sets

to determine harmonisation requirements

Automated combination

of processing services into

executable workflows

Workflow Design and Construction Service

The HUMBOLDT Framework HUMBOLDT Tools GI data integrator/ GIS developer

specify target HUMBOLDT GeoModel Editor configure/store model

End user of geodata or spatial information

Desktop GIS or GIS-Client specify schema translation

GI data integrator GI data integrator/ GIS developer

HUMBOLDT Services

external services 23

Mediator Service

Download Service that encapsulates the capability to execute predefined transformation workflows Execution of workflows retrieved from the HUMBOLDT Workflow Repository Service Ability to read data from Web Feature Services and Web Coverage Services and to feed into transformation services Invoke transformation processes via several standardised interfaces Ability to work with internal transformation services and external transformation services HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu

The HUMBOLDT Framework HUMBOLDT Tools GI data integrator/ GIS developer

specify target HUMBOLDT GeoModel Editor configure/store model

End user of geodata or spatial information

Desktop GIS or GIS-Client specify schema translation

GI data integrator GI data integrator/ GIS developer

HUMBOLDT Services

external services 25

Process step: Execution of transformation services

Source Executed technical processes Schema Transf.

Edge Matching … Target 26 Analysis of sources Input for transformation (mapping rules) Creation of target User knowledge HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu

HUMBOLDT Web Processing Services

Conceptual Schema Transformer

. Web Processing Service for transforming data from one application schema to another

Edge Matching Service

. Web Processing Service implementation of a service that aligns edges and points of vector geometries so that they will be gapless

Coordinate Transformation Service

. WPS implementation of a service that allows to transform coordinates between various geographic reference systems

Language Transformation Service

. Web Processing Service that is capable of transforming/translating all information that becomes visible to a user from one language to another 27 HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu

The HUMBOLDT Conceptual Schema Transformer (CST)

     Transformation of source data from source schema to target schema ‘Executes’ the schema mappings made in HALE. (Now integrated in the LAST version of HALE) Java platform Makes use of GeoTools libraries Test with OGC CQL (Common Query Language) for filtering and value   conditions The CST can be used via request to a Web Processing Service (WPS) Or as part of a data harmonisation workflow chained with other HUMBOLDT services 28 HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu

The HUMBOLDT Edge Matching Service (EMS)

   WPS implementation Aligns LineStrings and Polygons of vector geometries so that they will be gapless and with no overlaps Three modes of operation: 

CleanDataset:

when only one dateset is provided, then the gaps and overlaps will be removed.



Align-to-Reference:

all candidate data sets will be transformed using the reference dataset so that points are moved up to a maximum distance also provided as input. 

Distribute-Errors:

In this case, there is no reference data set that can be used as "ground truth", therefore all geometries will be transformed . No point or edge will be moved further than a client-specified amount. 29

EMS Clean Dataset Process – Results

 Input: 30

EMS Clean Dataset – Results

 Output: 31

Further HUMBOLDT Transformation Services

 Coordinate Transformation Service  Implemented and in use  Language Transformation Service  Early implementation  Multiple Representation Merging Service  Prototype implementation available HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu

HUMBOLDT Developer Community

Download software Access documentation Capture requirements Report bugs Progress tracing Discuss issues, build a knowledge base of solutions HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu

HUMBOLDT Open Source Software http://www.community.esdi-humboldt.eu/

HUMBOLDT Model Editor 

For the creation of UML application schemas

HUMBOLDT Alignment Editor 

Definition of conteptual schema transformation

Workflow Service  Service Composition for handling transformation needs Mediator Service 

Execution of transformation chains

Transformation Service 

Exposed as Web Processing Services (e.g. CTS, Edge Matching)

Who is the Community?

You!

The users and developers in the HUMBOLDT scenarios GENESIS: Sharing of Methodology GS-Soil: INSPIRE harmonisation and transformation, tools reuse ESDIN: HALE and CST dissemination, feedback Geoland 2: establish collaboration agreement Plan4all: NDA for document exchange for WP2 results in preparation NatureSDI+: HUMBOLDT training planned eSDI-Net+: Networking, joint meetings GIGAS: Contributions to the Technology Watch activities INSPIRE Data Specification Drafting Teams Snowflake, 1spatial, FME, … HUMBOLDT Workshop HCMR, Athens, March, 2011 © 2011 HUMBOLDT Consortium Fraunhoferstraße 5 64283 Darmstadt http://www.esdi-humboldt.eu

Document

Transcript Document

Tools and Components of the HUMBOLDT Framework

Key components Scenarios • Specifying user needs • Software testing Humboldt tools • Open source • Based on standards Training • E-learning • Dissemination

Creation of Target

Target

User knowledge

The HUMBOLDT GeoModel Editor

support the process of data specification

Process step: Schema to Schema Mapping

The HUMBOLDT Alignment Editor (HALE)

HALE

HALE functions

Schema mapping operations

Schema mapping operations

Schema mapping operations

Schema mapping operations

Schema mapping operations

WorkFlow Design and Construction Service

WorkFlow Design and Construction Service

Workflow Design and Construction Service

Mediator Service

Process step: Execution of transformation services

HUMBOLDT Web Processing Services

The HUMBOLDT Conceptual Schema Transformer (CST)

The HUMBOLDT Edge Matching Service (EMS)

EMS Clean Dataset Process – Results

EMS Clean Dataset – Results

Further HUMBOLDT Transformation Services

HUMBOLDT Developer Community

HUMBOLDT Open Source Software http://www.community.esdi-humboldt.eu/

Who is the Community?

Thank you for your attention!

Directory