Transcript Cloud Computing and NIEM

Future of NIEM
Cloud Computing
and the Importance of Data
Independence
Future of NIEM Vision
What is Cloud
Computing?
What is Data
Independence?
How does NIEM
and Ucore fit?
What is Data Independence?
Data moving to any
authorized place you need
it, in a form that any
authorized application can
use it, by any authorized
user, on any authorized
device.
any application
data
Cloud
on the Internet
any device, any user
any place
What is Cloud Computing?
The cloud is a metaphor for the Internet,
based on how it is depicted in computer
network diagrams, and is an abstraction
for the complex infrastructure it conceals.
Cloud Computing is about the
delivery of computing resources from
a location other than that from the
user.
In its most used context it is Internetbased ("cloud") development and use of
computer technology ("computing").
Most risks of cloud computing associated
with “Data Security” and “Data Privacy”.
“Who controls access to data I put into
the cloud?”
Source: Wikipedia,
http://en.wikipedia.org/wiki/File:CloudComputingStackLarge.svg
Impact of Cloud Computing on Data Independence?
Data moving to any
authorized place you need it,
in a form that any authorized
application can use it, by any
authorized user, on any
authorized device.
data
In addition to the content,
must include information on
the structure, access or
security info, and semantic
understanding.
Source: Wikipedia,
http://en.wikipedia.org/wiki/File:CloudComputingStackLarge.svg
any application, any place
Digital Music: Data Independence Success?
Digital Music
Albums to CDs
First move to digital
yield the capability to
play CDs on a
computer.
File Directory on PC
looked like:
Track 01
Track 03
…
Devices: Personal and
Auto CD Players,
Computers
CDs moved to MP3, a
universal music file
format.
MP3 File Tagging
matured. More data
went with the music.
MP3 File Tagging
standards
File Directory on PC
looked like:
File Directory on PCArtist: Etta James
File Directory on PC
Album Title: Heart
looked like:
of a Woman looked like:
Track#: 5
Genre: Jazz
Track01.mp3
Track02.mp3
…
At last.mp3
Girls Don’t Cry.mp3
Devices: MP3 players,
Video DVD players
Devices: Cell phones,
PDAs
…
At last.mp3
Girls Don’t Cry.mp3
…
Devices: Song
discovery from phone,
Titles show on radios
Social Networking: Data Independence Success?
MySpace
Orkut, Twitter,
…
Facebook
Success Factors For Data Independence
•
•
•
•
•
Market demand pushed
Distinct exchanges for discrete purposes
Open standards-based
Security and Privacy
Applications and services (cloud or not) need to be
built to accept the distinct, open standards-based
exchange
What are NIEM / Ucore / LEXS?
NIEM: information exchange model that is used to
structure of the payload of the information
exchange, using common - shared semantics.
Information Exchange Model
governed by practitioners
Lifecycle Approach to develop
discrete information exchanges
Both are open standards built
by practitioners for information
exchange of distinct packages
of information
Ucore/ LEXS: information
exchange specification and
implementation profile with
shared semantics for a small set
of elements (who, what, when
& where) with security
markings to permit access
control, providing a messaging
framework.
How does NIEM / Ucore fit?
Director of
National
Intelligence
(DNI)
NIEM
(structured
payload)
Ucore (who, what,
when, where with
access control)
Other
Feds…
Department of
Justice
(DOJ)
Dept of Defense
(DoD)
Open standards built
for information
exchange of distinct
packages
Department of
Homeland
Security
(DHS)
State,
Local,
Tribal &
Private
Sector…
SAR: Data Independence Success?
City of
Boston
State of
Florida
NIEM
(structured
payload)
Dept of
Defense
(DoD)
LEXS (basis of Ucore)
City of
Miami
Department
of Homeland
Security
(DHS)
State of
VA
NYC
Police
Dept
Department
of Justice
(DOJ)
Gaps in the Road to Data Independence
using NIEM /UCore / LEXS
• Lack of depth in common data access schemas
addressing privacy and security
• Lack of documented distinct exchanges for discrete
purposes
• Open standards lack capability for element level
security marking
• Vendor adoption of Ucore / NIEM needs
improvement
Current Success with NIEM / Ucore / LEXS
•
DHS adoption at 35% of Major IT programs, 60% for end of FY09
–
–
–
•
FY08 focus on discrete Info Exchanges has increased adoption significantly
Measured compliance in investment review process
Built into Enterprise SOA Guidance and System Engineering Lifecycle
DOJ adoption
–
–
–
–
–
CJIS System of Systems
Terrorism Screening Center (TSC)
FBI’s Guardian
NDEX / One DOJ
Future Development of NGI
•
DOD/DNI adoption through Maritime Domain Awareness and Ucore 2.0
•
PM ISE adoption of the TSC Terrorist Watch Listing and Suspicious Activity
Reporting Functional Standards
•
39 of 50 States using NIEM
–
–
–
39 states represent approx 75% of U.S. population
Mostly in Justice-oriented applications, some state wide( TX, NY, FL)
Bolstered by DOJ/DHS Grants like Real ID implementation
Background on Data Independence
Principles of Data Independence
data independence
Circa 1970’s
The separation of data from the programs that use the data. Nearly
all modern applications are based on the principle of data
independence. In fact, the whole concept of a database
management system (DBMS) supports the notion of data
independence since it represents a system for managing data
separately from the programs that use the data. In contrast, it is
possible to write applications in which the data being processed is
actually represented in the program's source code. This datadependent approach is very inflexible because it makes it difficult to
modify the data and it also makes the data inaccessible to other
programs.
Source: http://www.webopedia.com/TERM/D/data_independence.html
Principles of Data Independence
data independence for data
Circa 1990’s
Techniques that allow data to be changed without affecting the applications
that process it. There are two kinds of data independence. The first type is
data independence for data, which is accomplished in a database
management system (DBMS). It allows the database to be structurally
changed without affecting most existing programs. Programs access data in
a DBMS by field and are concerned with only the data fields they use, not
the format of the complete record. Thus, when the record layout is updated
(fields added, deleted or changed in size), the only programs that must be
changed are those that use those new fields.
data independence for processing
The second type of data independence relates to processing and refers to
miscellaneous data used in programs that might change in the future, such
as discount rates, product descriptions and error messages. Such data
should be stored in a database and not "hard wired" into the code of the
program. When values change, only the database item is altered, a much
simpler task than recompiling numerous programs.
Source: Computer Desktop Encyclopedia, http://www.answers.com/topic/data-independence
Principles of Data Independence
data independence: first level
Circa EARLY 2000’s
The logical structure of the data is known as the schema definition. In
general, if a user application operates on a subset of the attributes of a
relation, it should not be affected later when new attributes are added to the
same relation. Logical data independence indicates that the conceptual
schema can be changed without affecting the existing schemas..
data independence: second level
The physical structure of the data is referred to as physical data
description. Physical data independence deals with hiding the details of
the storage structure from user applications. The application should not be
involved with these issues since, conceptually, there is no difference in the
operations carried out against the data.
Source: Wikipedia, http://en.wikipedia.org/wiki/Data_independence
Principles of SOA
Service Oriented Architecture
Circa 2003
SOA achieves interoperability between different systems and programming
languages provides the basis for integration between applications on
different platforms through a communication protocol. One example of such
communication is based on the concept of messages. Using messages
across defined message channels decreases the complexity of the end
application thereby allowing the developer of the application to focus on true
application functionality instead of the intricate needs of a communication
protocol.
Allows new functionality developed to reference a common business format
for each data element.
Source: Wikipedia, http://en.wikipedia.org/wiki/Service-Oriented_Architecture#Principles
Principles of EDA
Event Driven Architecture
Circa 2003
EDA is a software architecture pattern promoting the production, detection,
consumption of, and reaction to events.
An event driven architecture is extremely loose coupled and well distributed.
The great distribution of this architecture exists because an event can be
almost anything and exist almost anywhere. The architecture is extremely
loose coupled because the event itself doesn’t know about the
consequences of its cause. e.g. If we have an alarm system that records
information when the front door opens, the door itself doesn’t know that the
alarm system will add information when the door opens, just that the door
has been opened
Source: Wikipedia, http://en.wikipedia.org/wiki/Event_Driven_Architecture
Principles of SBA
Space Based Architecture
Circa 2002
SBA is a software architecture pattern for achieving linear scalability of
stateful, high-performance applications, following many of the principles of
Representational State Transfer, Service-Oriented Architecture and EventDriven Architecture, as well as elements of grid computing. With a spacebased architecture, applications are built out of a set of self-sufficient units,
known as processing-units (PU). These units are independent of each other,
so that the application can scale by adding more units.
Source: Wikipedia, http://en.wikipedia.org/wiki/Space-based_architecture
Principles of SN
Shared Nothing Architecture
Circa 1986
SN is a distributed computing architecture in which each node is
independent and self-sufficient, and there is no single point of contention
across the system. People typically contrast SN with systems that keep a
large amount of centrally-stored state information, whether in a database,
an application server, or any other similar single point of contention. While
SN is best known in the context of web development, the concept predates
the web.
Shared Nothing is popular for web development because of its scalability.
As Google has demonstrated, a pure SN system can scale almost infinitely
simply by adding nodes in the form of inexpensive computers, since there is
no single bottleneck to slow the system down. Google calls this sharding.
An SN system typically partitions its data among many nodes on different
databases (assigning different computers to deal with different users or
queries), or may require every node to maintain its own copy of the
application's data, using some kind of coordination protocol.
Source: Wikipedia, http://en.wikipedia.org/wiki/Shared_nothing_architecture
Background on Cloud Computing
Principles of Cloud Computing
Cloud computing
Circa 2000’s
Cloud computing refers to the delivery of computational resources from a location
other than that from the user. In its most used context it is Internet-based ("cloud")
development and use of computer technology ("computing"). The cloud is a metaphor
for the Internet, based on how it is depicted in computer network diagrams, and is an
abstraction for the complex infrastructure it conceals.
It is a style of computing in which IT-related capabilities are provided “as a service”,
allowing users to access technology-enabled services from the Internet ("in the
cloud") without knowledge of, expertise with, or control over the technology
infrastructure that supports them. According to a 2008 paper published by IEEE
Internet Computing "Cloud Computing is a paradigm in which information is
permanently stored in servers on the Internet and cached temporarily on clients that
include desktops, entertainment centers, tablet computers, notebooks, wall
computers, handhelds, sensors, monitors, etc."
Cloud computing is a general concept that incorporates software as a service (SaaS),
Web 2.0 and other recent, well-known technology trends, in which the common
theme is reliance on the Internet for satisfying the computing needs of the users. For
example, Google Apps provides common business applications online that are
accessed from a web browser, while the software and data are stored on the servers.
Source: Wikipedia, http://en.wikipedia.org/wiki/Cloud_computing
Architecture of Cloud Computing
Cloud computing: Architecture
Circa 2000’s
The majority of cloud computing infrastructure currently consists of
reliable services delivered through data centers that are built on
servers with different levels of virtualization technologies.
The services are accessible anywhere in the world, with The Cloud
appearing as a single point of access for all the computing needs of
consumers.
[Commerical] Offerings need to meet the quality of service
requirements of customers and typically offer service level
agreements.
Open standards and open source software are also critical to the
growth of cloud computing.
Source: Wikipedia, http://en.wikipedia.org/wiki/Cloud_computing