Transcript Building a NIEM IEPD: the NIEM IEPD Life Cycle and Tools National Information

National Information
Exchange Model (NIEM)
Building a NIEM IEPD:
the NIEM IEPD Life Cycle
and Tools
6 September 2006
John Wandelt
and
Mark Kindl
Georgia Tech Research Institute
NIEM Builds on GJXDM
NIEM = GJXDM + Expanded Scope + Lessons Learned
• Incremental changes to GJXDM to address specific issues and
expanded scope
• A few examples:
– Expanded Domains (Intel, International Trade, Emergency Mgt,
Infrastructure Protection, etc.)
– Better Modularity
– Improve Reusability - Leveraging work of others – Discovery of IEPDs
and components
– Capturing Business Context
– Incorporating feedback into the model - Model Refinement through use
– Support for components from external standards
– IEPD consistency
2
NIEM Use Cases
• To effectively exchange information, there must be
a common semantic understanding of data among
participants, and the data must be formatted in a
consistent manner.
• NIEM provides reusable components and IEPDs
for exchanges
• Between NIEM domains (inter-domain).
• Within NIEM domains (intra-domain)
(where multiple sub communities exist
within a single NIEM domain, e.g. justice).
3
IEPD Development vs the Model
IEPD Development
• National priorities (e.g. NDEx, TWPDES)
• Authoritative sources (e.g. Rap Sheet)
• Local requirements
The Model –
NIEM reusable
components
0 Scenarios,
BRM taxonomies
1 Info exchg
requirements
5 Publish +
implement
reuse /
revise
add /
replace
4 Assemble
to IEPD spec
IEPD Library
(virtual)
IEPD
Life
Cycle
2 Map +
model
reuse
add /
replace
3 Build +
validate
NIEM
4
NIEM Naming and Design Rules
• Provides a normative technical reference for building NIEM data model
components and IEPDs.
• Draft NIEM NDR will be available for review and comment with the 1.0RC release.
• Specifies some of the following:
–
–
–
–
–
–
–
Use of types, elements, attributes, groups
Enumerations (code lists)
Annotations (documentation and appInfo)
Namespaces
ISO 11179 naming
External standards use (e.g. Geospatial)
Creating new entities (objects)
•
•
•
•
Composition
Specialization w/ inheritance
Associations (relationships)
Roles
– Adding properties to existing entities
• Type augmentation
• Metadata
5
NIEM Architecture (Model View)
NIEM
Total Migration
(special case)
External
domains or standards
NIEM Core
Universal
Structures
Justice
(a) Translate / Map
(b) Extract / Use
Common
(GJXDM)
Justice specific
External
Domain 1
Sc
J
Im
Intel
IT
IP
EM
Z
ZN
NIEM Domains
XN
..
Adopt
External
domains or standards
External
Domain 3
External
Domain 2
Y
Y
CodeLists
Create (boot strap)
Wrap / Refer
6
What is a NIEM IEPD?
Documentation and specification for describing
NIEM information exchanges.
Contains:
• A set of NIEM conforming XML schemas that
– (Re)use or correctly extend NIEM components.
– Define a class of XML exchange instances.
• Documentation detailing
– How to implement the IEP with the schemas.
– Exchange model, business rules, use cases, etc.
– Example instances, stylesheets
• Metadata for indexing, search, discovery,
maintenance, registration, etc.
7
NIEM XML Reference Architecture
XML Schema
*Code list
xsd
NIEM xsd subset
Conformance
Validation
*Constraint xsd
*Extension xsd
Constraint
Validation
*Exchange xsd
Namespace
* Optional
Exchange xml
IEPDs have their
own target
namespaces
8
IEPD Component Perspective
NIEM IEPD schemas are NIEM conforming.
IEPD
Extension
External 0
4
4
5
External 1
6
6
Subset
External domains or standards
External domains or standards
1
External 3
2
External 2
3
3
2
1
2
NIEM
3
1, 5 – create
2, 4 – wrap / refer
3 – translate / map
6 – adopt
9
The IEPD Life Cycle
• General framework for building IEPDs in logical steps.
• Primary process from the practitioner perspective.
• Guide to understanding how IEPDs are ideally built
and published.
• Not prescriptive, IEPD builder …
– Adjusts the scope of the framework to the IEPD level of effort.
– Enters the life cycle at appropriate step.
• The value of formal specifications for input/outputs of the
IEPD Life Cycle.
– Ensure consistency across IEPDs.
– Capture business context to facilitate search and discovery
of NIEM components and IEPDs.
– Machine readable and process-able to automate support
for the IEPD Life Cycle.
– Encourage and facilitate commercial tool development
and value added capabilities.
10
Executive orders,
mission statements,
business requirements,
business context,
policies/procedures
0 Scenarios,
BRM Taxonomy
IEPD for
reuse library
Specific information exchange,
high-level business
requirements, existing IEPDs
NIEM
1 Info exchange
requirements
5 Publish +
implement
Complete IEPD
NIEM IEPD
Life Cycle
2 Map +
model
Existing
IEPD artifacts
IEPD artifacts
to share/reuse,
metadata for
discovery
4 Assemble
to IEPD spec
Data requirements,
business context,
exchange model
3 Build +
validate
Valid IEPD schemas, example
instances, documentation, metadata
Submission
of candidate
NIEM
components,
component
business
context
Gap analysis, specs
for new components
Executive orders,
mission statements,
business requirements,
business context,
policies/procedures
0 Scenarios,
BRM Taxonomy
IEPD for
reuse library
Specific information exchange,
high-level business
requirements, existing IEPDs
NIEM
1 Info exchange
requirements
5 Publish +
implement
Complete IEPD
NIEM IEPD
Life Cycle
2 Map +
model
Existing
IEPD artifacts
IEPD artifacts
to share/reuse,
metadata for
discovery
4 Assemble
to IEPD spec
Data requirements,
business context,
exchange model
3 Build +
validate
Valid IEPD schemas, example
instances, documentation, metadata
Submission
of candidate
NIEM
components,
component
business
context
Gap analysis, specs
for new components
IEPD Life Cycle Feeds the Data
Model Maturity Life Cycle
Top
down
Bottom
up
IEPDs
NBAC / NTAC:
vet, harmonize,
conformance
0 Scenarios,
BRM taxonomies
1 Info exchg
requirements
Staging Area
5 Publish +
implement
NIEM IEPD
Life Cycle
4 Assemble
to IEPD spec
2 Map +
model
Early Awareness
& coordination
NIEM Governance
NIEM
3 Build +
validate
New component candidates
& business context
Groups with common data interests
13
From the
IEPD Life Cycle Perspective
GJXDM Tools
• Justice Information Exchange Model (JIEM) Tool
• JIEM-GJXDM direct mapping capability (through SSGT)
• Schema Subset Generation Tool (SSGT)
• Moving forward
• GJXDM Spreadsheet
• IEPD Clearinghouse
NIEM Tools
• Component Mapping Template (CMT)
• Graphical Browser
• Schema Subset Generation Tool (SSGT)
• Information Exchange Package Documentation (IEPD) Tool
• Toward NIEM 1.0RC1 (30 Sep 2006)
14
GJXDM Tools
Justice Information Exchange Model
(JIEM) Modeling Tool
• JIEM is a Web application that facilitates analysis of business context
and content (payload) requirements for justice information exchanges.
• The analysis is represented in a site database.
16
JIEM Exchanges and Dimensions
A site database defines exchanges in 5 dimensions:
process, event, agency, condition, document (information).
17
JIEM Reference Model
• There is a special site database called the JIEM Reference Model,
which defines exchanges common to most jurisdictions.
• A site database can be created from scratch or copied from another site
database, or the Site Database Builder can be used to create one from
the JIEM Reference Model. Using the Site Database Builder typically
reduces the effort of site database creation by about 75%.
18
JIEM Data Elements
A site database includes data elements, which can be simple
or may be composed of other data elements.
19
JIEM Data Element Usage
Documents contain data elements and define their typical usage.
20
JIEM Reports
• A variety of textual and graphic reports can be generated.
• Reports generally relate to content of the database or usage of data
(comprehension or conceptualization of the data, data admin, etc.)
21
How JIEM and SSGT
Interoperate
SSGT Mapping Tool
JIEM
Schema Subset Generation Tool (SSGT)
.XMI
ArgoUML
Data exchange requirements
(local vocabulary) output in XMI
(i.e. UML-XML)
22
SSGT Mapping Tool
Enables user that has built an AS-IS or TO-BE model with JIEM to map local
data components directly to GJXDM data components through the SSGT.
Schema Subset Generation Tool (SSGT)
23
SSGT Mapping Tool –
Search / Discover
• User local data components are displayed (left)
with convenient access to the SSGT (right).
• User can search with SSGT for GJXDM data components
that match local data components.
Schema Subset Generation Tool (SSGT)
Schema Subset Generation Tool (SSGT)
24
SSGT Mapping Tool –
Select / Build
User maps by
– selecting matching GJXDM data components (check box).
– confirming (clicking) the match to build the map.
Schema Subset Generation Tool (SSGT)
Schema Subset Generation Tool (SSGT)
25
SSGT Mapping Tool –
Generate Subset and Report
When all local data components have been reviewed and
mapped or not, the Mapping Tool can generate:
1. Subset for mapped elements (also contains the wantlist).
2. Report detailing what mapped and what did not.
Schema Subset Generation Tool (SSGT)
26
How JIEM and SSGT interoperate
SSGT Mapping Tool
JIEM
Schema Subset Generation Tool (SSGT)
Mapping Report
Wantlist
SSGT
.XMI
ArgoUML
27
Schema Subset Generation Tool
(SSGT)
• SSGT is a Web application that allows a user to browse the GJXDM
and create GJXDM schema subsets and their wantlists.
• Supports basic and advanced search for GJXDM components.
28
SSGT – Expand a
Type or Property
• A complex type can be expanded to see its properties.
• A property can be expanded to see its type’s properties.
29
SSGT – Types
User clicks a type to see its:
–
–
–
–
–
definition
base types
derived types
properties
properties that it is the type of
30
SSGT – Properties
User clicks a property to see its:
– definition
– types in which it’s contained
– its types.
31
SSGT – Select
Properties and Types
Properties and types can be selected for inclusion in the GJXDM schema subset.
32
SSGT – Generate Subset.zip
Generating a GJXDM schema subset, creates a .zip file containing
both the subset and and its corresponding wantlist.
33
SSGT – Save and Load Wantlists
Wantlists can be saved and loaded so that work can be suspended and resumed.
34
How JIEM and SSGT interoperate
SSGT Mapping Tool
JIEM
Schema Subset Generation Tool (SSGT)
Mapping Report
Wantlist
SSGT
.XMI
ArgoUML
Subset.zip
35
Moving Forward
• Export other dimensions beyond information dimension
in standard open specification (e.g. BPMN, etc.).
• Evaluate applicability to other domains and NIEM.
• Capture additional metadata (security, privacy, etc.)
to support the implementation of an SOA.
• Support generation of extension and constraint schemas.
• Support sharing/reuse of mappings, wantlists, and
schemas across JIEM exchanges and site DBs.
• Support sharing/reuse with other non-JIEM users
through standard import/export specifications
(the NIEM IEPD specification is well suited for this).
36
GJXDM Spreadsheet
37
GJXDM Spreadsheet
38
GJXDM IEPD Clearinghouse
39
GJXDM IEPD Clearinghouse
40
GJXDM Tool Summary
• Justice Information Exchange Model (JIEM) Tool:
– Define/model exchange context and content requirements.
– Generate XMI (UML-XML) representation of local data exchange requirements.
• GJXDM Spreadsheet:
– Off-line textual representation of GJXDM (the model and dictionary).
– Search, navigate, browse, discover GJXDM data components.
• Schema Subset Generation Tool (SSGT):
–
–
–
–
–
On-line textual representation of GJXDM (the model and dictionary).
Search, navigate, browse, discover GJXDM data components.
Select and map GJXDM data components to local components (Mapping Tool).
Generate mapping report spreadsheet (Mapping Tool).
Select data components and generate schema subset and associated Wantlist.
• Manually produce other artifacts required for an IEPD.
– Examples: exchange, extension, and constraint schemas, and example instances.
• IEPD Clearinghouse (CH):
– Register and share a GJXDM IEPD.
41
Available Tools: GJXDM
0 Scenarios,
BRM taxonomies
JIEM
Spreadsheet
1 Info exchg
requirements
IEPD CH
SSGT Map
5 Publish +
implement
IEPD
Life Cycle
4 Assemble
to IEPD spec
SSGT
2 Map +
model
3 Build +
validate
SSGT
42
NIEM Tools
Component Mapping Template
(CMT)
A spreadsheet for mapping local data elements to NIEM elements.
•
•
•
•
•
•
•
Developed to expand GJXDM to NIEM by identifying data requirement gaps.
Developed as submission format for NIEM pilots.
Reflects experience gained from working with key GJXDM customers.
Structures the process of using NIEM for exchange content requirements.
Identifies new data components for addition to NIEM.
Structured format allows for automated processing.
Structured method to capture:
– Local data requirements (vocabulary) and map them to existing NIEM data
components
• exact matches
• partial mapping details
• no mapping
– Information that can provide a metric for modeling refinement and improvement
– Information that can be used for keywords, search hints, mapping assistance, etc.
44
CMT – General Tab
General information on exchange(s) being mapped (business context)
45
CMT – Template Tab
Information on how local components are mapped to NIEM
46
CMT – Codes Tab
Information on how codes are mapped to NIEM
47
NIEM Spreadsheet
48
NIEM Tools for Browse, Navigate,
Search, Discover, Generate/Build
Schema Subset Generation Tool
Graphical Browser
IEPD Tool
49
NIEM Graphical Browser
Java-based GUI to browse, navigate, and generally familiarize with NIEM structure.
3 different
views
8 different
starting locations
50
NIEM Graphical Browser –
Starting Points and Views
• User may start from one of 8 different locations in NIEM (others can be added).
• Select from 3 different views of the NIEM:
(1) Data model, (2) Inheritance, (3) Associations
• The 3 views always correlated to the same location in the model.
Data Model (types + properties)
Associations
Inheritance
51
NIEM Graphical Browser –
Entities and Relationships
• Model is displayed as
a set of nodes and edges
corresponding to entities and
fundamental data modeling
relationships between them:
– parent_of (IS-A)
– contains (HAS-A)
– type_of
• Double-clicking an entity (node)
moves focus to that entity
(by centering and highlighting)
and displays all entities and
relationships to/from it.
52
NIEM Graphical Browser –
Relationships
Relationships (edges) are directional;
to expose a relationship label, hover the cursor over its corresponding edge.
53
NIEM Graphical Browser –
Screen Management
When an entity has >10 relationships, only the first 10 entities are exposed;
next 10 can be seen by clicking the “next 10” button (lower right).
54
NIEM Graphical Browser – Focus
The entity of focus can be dragged around to expose
entities that may be outside the window.
55
NIEM Schema Subset Generation Tool
(SSGT)
• A repository of all NIEM data components with their associated
metadata linked as a consistent, cohesive data model
• Functions: search, find, and select components for,
build and generate a subset schema for an IEPD.
• Linkage of data components is key to the tool’s ability to provide
model navigation and efficient generation of schema subsets.
56
NIEM SSGT – Search
Allows both simple and advanced search for data components
using various means and criteria including:
– Type of component (property, type, facet, external, or association)
– With string, substrings, wildcard, exact/partial, etc.
– Within component name or definition or both
57
NIEM SSGT – Select / Add
• User finds and selects components required by checking boxes
to build a NIEM subset schema.
• A property component can be added to a subset
– as a content bearing element or as a reference to content bearing element.
– as an element within its Subject type or as an independent element.
58
NIEM SSGT – Dependencies
• SSGT adds all other components that
the user requirements depend on.
• User may delete components from
the subset only if he added them.
• Components that user requirements
depend on can only be deleted by
SSGT (when corresponding user
requirements have been deleted).
59
NIEM SSGT – Subset Schema
When user is done adding components, SSGT can automatically generate a
complete NIEM NDR conformant subset schema for a given version of NIEM.
Subset.zip
Wantlist
60
NIEM SSGT – Wantlist
• A Wantlist is an open spec XML file representing user requirements
(without dependencies) and the state of an SSGT session.
• A Wantlist can be saved and reloaded later to resume work
or to reconstitute or modify a subset schema.
61
NIEM IEPD Tool (IEPDT)
• Enables user to easily assemble, document, generate, and share
complete or partial IEPD per NIEM IEPD specification.
• User registers for account and corresponding work space.
62
NIEM IEPDT – Entering an IEPD
User creates an IEPD through a 5 step process:
1. Upload artifact files (optional)
2. Enter basic IEPD metadata.
3. Enter additional IEPD metadata.
4. Enter authoritative source metadata.
5. Verify all information and create the IEPD.
63
NIEM IEPDT –
Validation and Review
• Validates package completeness against NIEM IEPD content specification
and reports results (does not have to pass validation to share).
• Provides a review of metadata and artifact files.
64
NIEM IEPDT –
Auto Format and Assembly
•
•
•
•
Formats metadata file per NIEM IEPD spec (metadata.xml).
Formats catalog file of artifacts per NIEM IEPD spec (catalog.xhtml).
Assembles IEPD as compressed archive (filename.zip).
Installs IEPD in user work space and provides option to share.
65
IEPD Catalog.html
xhtml
IEPD Catalog.html
(source view)
NIEM IEPDT – Download / Upload
• Download IEPD as single, compressed, self-contained archive (<filename>.zip).
• Upload IEPD archive file (created manually or by IEPDT per IEPD spec).
• IEPDs created manually or by any other tool can be uploaded by IEPDT.
74
NIEM Tool Summary
• Component Mapping Template (CMT):
– Map off-line data requirements for content submissions to NIEM.
– Capture information – metrics, business context, POC data,
mapping statistics, etc.
• Graphical Browser
– Visually navigate and familiarize with NIEM.
• Schema Subset Generation Tool (SSGT):
– Repository of NIEM data components and metadata.
– Textually navigate, search, discover NIEM data components.
– Build and generate subset schemas for NIEM IEPDs.
• Information Exchange Package Documentation (IEPD) Tool:
– User work space – assemble, document, download, upload
IEPDs per NIEM IEPD specification.
– Share complete IEPDs or their artifacts.
75
Available Tools: NIEM
0 Scenarios,
BRM taxonomies
Spreadsheet
1 Info exchg
requirements
5 Publish +
implement
IEPDT
IEPD
Life Cycle
2 Map +
model
SSGT
CMT
Graph Brwsr
IEPDT
4 Assemble
to IEPD spec
3 Build +
validate
SSGT
76
Available Tools: NIEM + GJXDM
0 Scenarios,
BRM taxonomies
Spreadsheet
JIEM
Spreadsheet
1 Info exchg
requirements
IEPD CH
SSGT Map
5 Publish +
implement
IEPDT
IEPD
Life Cycle
SSGT
2 Map +
model
SSGT
CMT
Graph Brwsr
IEPDT
4 Assemble
to IEPD spec
3 Build +
validate
SSGT
SSGT
NIEM
+
GJXDM
77
NIEM 1.0RC1 (30 Sep 2006)
Refinements and Improvements
• URI for each NIEM data component. – A universally unique identifier;
can be a URL that resolves to a documentation page defining
semantics, structure, and relationships with other data components.
• Selective Sharing of IEPD artifacts and metadata. – IEPD Tool will allow
user to selectively share parts of an IEPD (rather than simply the entire
IEPD or nothing).
• Support the building of a new IEPD from an existing IEPD. – Capability
to build IEPDs directly by selecting components to be used within the
IEPD, generate, and optionally register/share.
• Improved integration between IEPD Tool, Graphic Browser, and SSGT.
– Intent is to remove extra manual steps between tools (e.g., download
SSGT subset and upload to IEPD Tool).
• Version support for IEPDs. – For bug fixes and changes; will allow for
multiple versions of the base IEPD to exist to avoid possible conflicts.
Track version history and change log in IEPD metadata.
• Basic reporting feature. – usage statistics and management metrics, for
example: components used most often in IEPDs, common search
terms, commonly selected results, etc.
• Export NIEM as a MS Access database
78
Discussion & Questions