Transcript Larry Burns

Redefining Data Management
Larry Burns
Data Architect and Consultant
Copyright (c) 2014 Larry Burns
All Rights Reserved
Session Topics
This session will cover:
• The Data Management paradox: Data Mgt. doesn’t work,
but we still need it!
• The changing data landscape
• The changing business environment
• Data Mgt. vs. Data Gov. – Burns’ Law
• New definitions of Data Management
• The changing nature of data work
2
The Challenge
Traditional concepts of Data Mgt. are being challenged
on all sides:
• Business users expect immediate access to data of all
kinds throughout the organization (Big Data)
• Non-relational DBMSs are on the rise
• There is less distinction between data and content
• Delivery time intervals continue to shorten
• There is increasing acceptance of imperfection, in both
software and data
3
The Challenge
We in Data Management need to face some hard truths:
• Nobody is really interested in Data Management
• The usability of data is more important than quality
• The business wants results now, not months from now
• Business users are more knowledgeable about data
• New tools allow business to bypass the “roadblock” of IT
and get quicker results
• Business users are willing to be instructed by IT, but not
managed or led
4
The Challenge
In sum:
The problem with Data Management is that it represents
those aspects of IT that the business most wants to
avoid!
–
–
–
–
–
An excess of management and control
An overabundance of process
Sluggish response to business needs and changes
Inflexible tools and approaches
Massive time, cost and resource requirements
5
The Paradox
And Yet:
This statement is still as true as it’s always been:
“Organizations that do not understand the overwhelming importance
of managing data and information as tangible assets in the new
economy will not survive.”
-- Tom Peters, 2001
• And data doesn’t magically provision itself!
6
The Problem With Our Current Approach
Our current Data Management approach involves:
• Top-down architecture, design and implementation
• A Waterfall (all-or-nothing) approach
• Rigid and inflexible definitions of data
• Data truncated to fit definitions (the Procrustean bed)
• Absence of data discovery (by the business)
• BI solutions created by IT, not by the business
• An absence of iteration
7
Business Context
Our Current BI Approach
Advanced Analytics
Reporting
Repositories
Extract, Transform
and Load (ETL)
Information
Management
BI Framework
Enterprise Architecture
What Would be a Different Approach?
We need a new definition of Data Management:
• Things become obsolete, not because of what they are,
but because of how they are defined!
• If the turn-of-the-century railroads had thought of
themselves as transportation providers, rather than as
operators of trains, we might all be flying today on Union
Pacific Airlines!
• We in Data Management are in danger of defining
ourselves out of existence!
9
What Would be a Different Approach?
The current definition of Data Management:
“Data management is the function of planning for, controlling and
delivering data and information assets.”
-- DAMA DMBOK (2010), p. 4
10
What Would be a Different Approach?
A proposed new definition of Data Management:
“Data management is the function of provisioning data in support of
effective business information processes.”
-- Larry Burns
11
Characteristics of a New Approach
The following principles should characterize a new
approach to Data Management:
• Focus on information processes, not data or databases
• Emphasize facilitation and collaboration, not control
• Let business (not IT) direct the process
• Adopt an Agile (iterative) methodology
• Accept (and manage) risk
12
Take an Information-Based Approach
The nature of data work is changing!
• Data professionals are becoming information managers,
not just database designers
• There is now less distinction between data and content,
and much more unstructured data
• The focus has shifted from designing and building
databases to designing and building information delivery
systems (Information Architecture)
• Less need to design data structures; more need to
design information views
13
Take An Agile Approach
We can (and should) apply Agile principles to Data
Management:
• Do the right amount of the right work at the right time
• Empower people (not just technology) to get work done
• Let the business direct the solution
• Deliver results continually and incrementally
• Design (and automate) efficient processes
• Accept “just good enough” (80%) solutions
14
Take a Process-Based Approach
Effective solutions are process-based, not technologybased:
• Underlying business processes must be understood (by
both business and IT) before technology can be
introduced
• Not all processes need to be automated
• People must be empowered to support these processes
• We want people to be doing the right kind of work!
15
Take a Process-Based Approach
All business processes involve:
• Work Management (what needs to be done, and when?)
• Risk Management (what outcomes must be forestalled)
• Opportunity Management (what outcomes must be
encouraged)
• Cost Management (how can we achieve our goals most
effectively?)
All Information Management processes involve one or
more of the above business processes!
16
Take a Risk-Management Approach
Understand the difference between Data Management
and Data Governance:
• Data management is the IT work of provisioning data to
the business, including platform and storage
management, virtualization, integration and delivery.
• Data governance is the business management of data
meaning, content and information.
17
Burns’ Law:
“The less data management you have, the more data
governance you need”:
• Traditional data management tries to eliminate the risk of
the business using bad data.
• The less we do of this work on the IT side, the more we
need to do risk management (i.e., data governance) on
the business side.
• The business needs to understand and assume this risk!
18
Data Mgt.
Traditional
EDW
Self-Service
BI
Risk
Data Gov.
Cost/Effort
The BI “Magic Quadrant”
19
Data Governance is a Necessity!
Data governance helps the business address the
following questions during data discovery:
• “Where did this data come from?”
• “What is the business meaning of this data?”
• “How current is this data”?
• “What do I do if I think this data is incorrect?”
• “How can I share my analysis with others?”
• “How do we resolve disagreements about the meaning
or the value of the data?”
• “What is the risk of using this data?”
20
Unstructured Data
Data Governance
Structured Data
Data Acquisition
Data Discovery
Data Analysis
Master Data / EDW
Metadata
Data Virtualization
Data Visualization
Proposed New BI Approach
How is this Different?
A more flexible approach to BI would include:
• Accommodation for data discovery and “Big Data”
analytics
• Accommodation of unstructured as well as structured
data
• Master data can be created/updated through the process
of data discovery (as, for example, Groupon)
• Master data can inform the process of data discovery
• Metadata becomes hugely important!
22
In Summary:
We need a new paradigm for Data Management:
• One that recognizes the ways in which data is currently
being used to create and manage information
• One that is flexible, Agile, and responsive to the needs of
our business users and customers
• One that is less dependent on traditional notions of
databases and database technology
• One that accepts (and manages) risk, rather than trying
to eliminate it
• One that is not IT-centric
23
Speaker Bio
Larry Burns is the Domain Architect for Data and
BI at a global Fortune 500 company, where he
teaches data and database concepts to application
developers. He was a contributing author and
editor of DAMA International’s Data Management
Body of Knowledge (DAMA-DMBOK), and writes a
quarterly series of articles for TDAN.com.
24
Speaker Bio
He has been an instructor and advisor in the
certificate program for Data Resource
Management at the University of Washington in
Seattle. He is also the author of "Building The
Agile Database", published by Technics
Publications (www.technicspub.com).
25