Transcript Practical Approaches to Sharing Information at Raytheon

Practical Approaches to Sharing
Information at Raytheon
Taxonomies, Metadata and Beyond
Presented by
Christine JM. Connors
[email protected]
KMPro / KMForum, Bentley College, Waltham, MA
June 30, 2004
Data Discovery –
What we Learned
•
•
•
Level of information management varies
• 85% of our information is unstructured
• Over 90% of information is not tagged
• High proportion of tagged documents result of templates, and
therefore relay bad data
• Ethan Frome – over 200 documents
• Automatically extracted data imprecise
• “The flight to Dallas is cancelled”
Greater than 13% of information is exactly duplicated
• “Near” duplication harder to determine but potentially more costly
• Worst duplication in File Servers / Shared Drives
Difficult to determine true age of document due to web scripting, date
of publication to public drive
• Over 23% of sample data not modified in previous 5 years
• Over 68% of sample data not modified in previous 2 years
2
Intranet Search and Browse Survey
June 2003
• 16 multiple choice questions, 1 optional free-text comment field
• 516 surveys started (clicked into)
• 199 responses over 3 weeks
• 39% completion rate
• 101 comments on “How can we improve the intranet search and
•
browse capabilities?”
51% comment rate by survey participants
3
How can we improve the intranet search
and browse capabilities?
101 user comments frequently included:
• Qualify searches by function, organization, and business
• Qualify searches by date
• Qualify searches by document type (especially web pages)
• Provide sorting of results by date, document type
• Provide category search
• Do not change URLs of pages (users bookmarked)
• Reduce number of search results
• Google (mentioned 32 times in comments)
4
Results Summary
Search/browse
•
•
•
•
•
About ¼ (26%) of respondents find the current capabilities “Good” or
“Excellent”
About ¼ (24%) of respondents consistently locate helpful information
About ¼ (22%) of respondents indicate they are consistently successful using
keyword searching
About ¼ (26%) of respondents find it consistently easy to browse
The advanced search page is used sparingly
Categories
•
62% of respondents would find categories consistently useful
Bookmarks
•
•
•
36% of respondents consistently return to previously found information
71% of respondents use bookmarks “Most of the time” or “Always”
65% of respondents consistently organize their bookmarks
5
OUCH!
•
“The search engine is poor to inadequate. I needed to find an appropriations
data sheet and was returned 366 entries, none which had anything to do with
appropriations. I spend far too much time looking through the search results
for this engine to be effective. If I could find this document on the INTERNET
I would do so, but this is an internal Raytheon document that is successfully
hidden somewhere in the archives with the Ark of the Covenant.”
• Unidentified search and browse survey participant, June, 2003
•
“Who gets more hits: www.amazon.com or
www.thequaintbookstoredownthestreet.com? Listen up people: Our intranet
is a wasteland of information. We need to unify - we need to standardize.
Information is power - but only if it is on my desktop, not hidden away in
some server waiting for a lucky adventurer to uncover it like some lost
continent.”
• Another unidentified search and browse survey participant, June, 2003
6
Usability Testing
• Bentley College’s Design and Usability Testing Center
• 4 Focus Groups of 8-10 people each
• They told us:
• Want to filter searches
• Didn’t want long list of items to select from
• Liked “Suggested…” boxes
• Didn’t understand the taxonomy when presented like Yahoo!
• Liked taxonomy as file folder metaphor
• Liked thesaurus
• Were confused by relational taxonomies
• Liked “Categories” as the tab label, over Topics, Taxonomy,
Thesaurus, Subjects or Browse.
7
Taxonomies – Who?


The Dream Team
 Information Scientists
 Cognitive Scientists
 Linguists
 Programmers
 Database Experts
 Network Specialists
 Verity Administrators
 Human Computer Interaction / Usability Experts
 Subject Matter Experts
 Organizational Change Management
What we got
 Information Scientists (in-house)
 Cognitive Scientist / Linguist (from Verity)
 Programmers, Database Experts, Network Specialists, and Verity
Administrators (both in-house and from Verity)
 HCI / UI Experts (in-house, Verity and Bentley College)
 Subject Matter Experts (in-house)
8
Taxonomies – What for?
•
•
•
•
•
Browse & Navigation
Relational Taxonomies
Refine Search
Parametric Search
Federated Search
•
•
•
Dynamic taxonomies
Profiling
Compliance Engine
•
•
•
Classification & Categorization
Provide controlled vocabularies to use with Metadata Schema(s)
Easy selection to minimize angst over having to fill out file properties
9
Taxonomies 2003

Deployed 5 taxonomies
 Defense Technologies (based on DTIC)
 Purchased DTIC taxonomy
 Revised to fit Raytheon’s data
 Removed several categories including agriculture that are not needed
 Raytheon Products
 Revised our products listing into a hierarchical approach
 Enlisted Raytheon Communicators as Subject Matter Experts
 IPDS
 Built using data from the IPDS web site
 Enlisted IPDS experts as Subject Matter Experts
 Engineering
 Implemented taxonomy built by Raytheon’s Engineering Technology
Network – Needs revising and enhancing
 Information Technology
 Purchased from Verity
 Revised to fit Raytheon’s data
 Enlisted members of Corporate IT as Subject Matter Experts
10
Taxonomies 2004




Human Resources taxonomy
 Deployed March 2004
 Purchased from Verity
 Cross-functional team of HR representatives reviewed as Subject Matter
Experts
Six Sigma
 Will be deployed June 2004
 Built based on Raytheon Six Sigma data
Legal Taxonomy
 Will be deployed June 2004
 Purchased from Verity
 Able to create additional taxonomies for Ethics, Environmental Health &
Safety, and Export/Import Compliance from the purchase of this ONE
taxonomy
Will be restructuring our top level categories :
 Business Units – domestic & international
 Functions
 Processes
 Products
 Relationships
11
Taxonomies – How?
•
•
•
•
•
•
•
•
•
•
Card sorting
• EZ Sort
• 3x5 cards
Review search engine logs
• Internal logs
• Webtrends
Review organic systems
• Web and file share navigation
Review existing taxonomies/thesauri
Concept Mapping
• Linguistic algorithm
Intelligent Classifier – lots of query building behind each node
MultiTes
Mind Manager
TextPad
BUY!
12
Build vs. Buy
•
•
•
•
•
Build to suit users
• Reflect corporate vernacular
• Internal acronyms
• Corporate culture
• How is the business structured and portrayed?
Can take a long time
• Time estimates depend on type/use of taxonomy/tools available
• Simple = 5 minutes to build term, 5 minutes to build category and
map the topic to the taxonomy
• Complex = 75 minutes to build term
• PLUS Quality Assurance testing!
Buy
• Industry standard
• Rapid implementation
Need customization
Both decisions require maintenance
13
Benefits - Increased Productivity
Delphi Group 2003 – as reported by Gartner
• Business professionals spend more than 2 hours per day searching for information
• Half of that time – 1 hour per day is wasted by failure to find what they seek
• The single factor most attributed to the large amount of time wasted was
• data changes (location 35%) and
• bad tools (ineffective search and lack of labeling 28%)
• If we conservatively assume only 1 hour per year would be saved per general employee and
1 hour per month per engineer, then:
Dollars Returned to the Business for Growth
Hours Returned to Engineering by Business
(1 hour per year per general employee)
(1 hour per month per engineer)
(1 hour per year per employee)
(1 hour per month per engineer)
$4,000,000
7000
$3,500,000
6000
$3,000,000
5000
$2,500,000
4000
$2,000,000
3000
$1,500,000
2000
$1,000,000
1000
$500,000
0
$0
Corp
RSL
RTSC
SAS
RAC
NCS
RMS
IIS
IDS
Dollars
Hours
69
542
2872
6020
661
5377
4956
4319
Corp
RSL
RTSC
SAS
RAC
NCS
RMS
IIS
IDS
$123,000 $378,100 $2,099,60 $3,869,75 $716,950 $3,529,55 $3,254,25 $2,771,25 $2,962,55
4407
14
Benefits – Reduce Storage Costs
Data growth assuming 60% annual growth rate
$90
2500
T1 Only
General tiered move
Unintelligent Move
Policy based Move
$80
Millions (Annual Cost)
$80.2
$70
2000
$60
$55.3
$50.1
$50
1500
$44.3
$40
$31.3
1000
$33.2
$30
$20
$19.6
$12.2
$0
500
$17.3
TB
$10
$27.7
$10.8TB
TB
TB
340
544
870
1393
2003
2004
2005
2006
TB
2228
0
2007
Relative of starting point, growth curves represent storage acquisition cost increases over time.
15
Is it working?
• “New” search launched September 29
• Latest survey results show improvement
• Neutral rating upgraded to Good
• Metrics show increased usage of search
• 17% increase in unique users per day
• 25% increase in searches per day
• Metrics show increasing use of categories in search
• Since launch, the categories have been used 50,000 times
• ITLT approved project funding
• Knowledge Representation team recipients of 2003 IT
Excellence in Collaboration and Knowledge Management Award
16
17
17
18
18
19
19
20
20
21
21
22
22