Transcript PPT Overview of SharePoint Search

SharePoint Search
Mirjam van Olst
Information Worker Solutions
Macaw
Making Serious Sites
Agenda
•
•
•
•
•
•
•
•
Enterprise Search Challenges
Microsoft Search Vision
SharePoint End-user improvements
Knowledge Network
Administration Improvements
Performance and Capacity Planning
Customizing the Search UI
Develop Search Solutions
Enterprise Search Challenges
Making Serious Sites
Enterprise Search Challenges
Finance Manager
Reviewing a Budget
People often know
more than systems
Information access is
controlled
Information is stored in
many places and in
many formats
Sales Executive
Responding to an RFP
Business Analyst
Preparing a Briefing
Microsoft Search Vision
Making Serious Sites
Microsoft Search Solutions
Enable professionals to find, use, and share relevant
business data quickly, easily and securely
• Find information across the
desktop, intranet and Internet
Find
Use
• Use information via intuitive,
familiar interfaces
• Share information among teams
and connect people in real-time
Share
Search As an Infrastructure Service
• Scenario spectrum
–
–
–
–
Within-site search– WSS or MOSS
Search-driven business applications - MOSS
Intranet Search – MOSS
Internet point of presence search - MOSS
Enterprise Search Architecture
User Requests
X
Load Balancer
...
Web front ends
...
Query servers
Propagation
of indexes
External
content
...
Indexer
Crawling
...
Content
databases
SharePoint Server 2007 SKUs
SKU
Description
Offers all functionality. Requires server
license and Client Accent License.
Offers all functionality. Requires only a
server license.
for Internet Sites
Offers only Search functionality. Requires a
server license.
Enterprise Edition
Offers only Search functionality. Requires a
server license. Limited to 500.000 content
Standard Edition items.
Offers only forms functionality. Requires
server license and Client Accent License.
for Internet Sites
Offers only forms functionality. Requires
only a server license.
Search Feature Comparison
Microsoft®
Office
SharePoint®
Server 2007 for
Search (Standard
Edition)
Microsoft®
Office
SharePoint®
Server 2007 for
Search
(Enterprise
Edition)
Microsoft®
Office
SharePoint®
Server 2007
(Standard CAL)
Microsoft®
Office
SharePoint®
Server 2007
(Enterprise CAL)
Search File shares, Web Sites,
SharePoint Sites, Exchange PFs, Notes
DBs OOB
Search 3rd Party Document Repositories
Indexes 200 File Types OOB (extensible)
Secure Content Access Control
Search for People and Expertise
Enhanced Search Center UI
Search Structured Data Sources
Document Limit
500,000
No Limit
No Limit
No Limit
Search Feature Comparison
WSS
MOSS
Local
SharePoint
content
SharePoint Web,
Exchange, File
shares, Notes,
LOB…
Rich, relevant results
X
X
Alerts, RSS, DYM, Dup collapsing
X
X
Can Index
Scopes, Managed Properties
Best Bets, Result Removal, Query Reports
X
X
Tabs
X
People Search, KN
X
BDC Search
X
APIs Provided
Query
Query + Admin
Secure & Manage Your Search Deployment
Control access to information with permissions
– Perform standard query time security trimming
– Implement custom security trimming
Manage your search solutions
– Analyze usage patterns using powerful out-of-the-box reporting
– Tune relevance parameters for better results
– Control deployment of desktop search with group policies
Scale your deployment
– Cluster web front ends, query servers, and index databases
– Improve indexing performance with a dedicated index server
– Index up to 50 million items per index
Unlock Unstructured & Structured Data
Find information in unstructured repositories
– Find content in web sites, file shares, SharePoint sites, public folders & Notes DBs
– Use Protocol Handlers to index content in proprietary repositories
– Support any file type with iFilters (over 200 supported formats out-of-the-box)
Use search as an interface for line-of-business systems
– Index structured data sources using the Business Data Catalog (BDC)
– Customize search results using out-of-the-box Business Data Web Parts
– Make results actionable with Business Data Actions
Refine results and save important queries
– Refine results using the keyword query syntax
– Filter and sort result lists using relevant attributes
– Save important queries and subscribe to RSS feeds or email alerts
IFilters
iFilters available for lots of different file types
http://www.citeknet.com/ for:
• CAB
• CHM
• EXE
• HLP
• RAR
• ZIP
https://support.cadcompany.nl/ifilter/default.aspx for:
• DWG (Looking for Beta users)
http://www.adobe.com/support/downloads/detail.jsp?ftpID=2611
• PDF
SharePoint End-user
Improvements
Making Serious Sites
Relevance Improvements
• Best results on first page, every time
• Enterprise Search is different from Internet Search
– Far less cross-linking of content
– Lots of non-Web content: Office docs, line-of-business data,
etc.
– Security is paramount
• Revamped relevance & ranking engine
– Extensive collaboration with Microsoft Research, MSN,
Windows
– Algorithms tuned for web and enterprise content
– New search ingredients
Relevance Improvements
• New ingredients added, including
–
–
–
–
–
Click Distance
Anchor Text
URL Depth
Missing Metadata Creation
Automatic Language Detection
• Result is noticeably more relevant search
– 100% better on all queries
– 500% better on common queries
Influencing Ranking
• Best Bets
• Additional Metadata
• Authoritative Sites
Enhanced User Experience
• Simple yet powerful user interface
–
–
–
–
–
–
–
Query correction – “Did you mean?”
Hit highlighting
Duplicate Collapsing
Alerts
RSS Feeds
Security-trimmed search results
Actionable results through integration with Microsoft Office
applications, Windows Desktop Search, Live Communication
Server
Enhanced Search Center
• Search Center
– Web part-based query and results pages
– Many options for customization
• Search tabs
• OOB Search Web Parts
–
–
–
–
–
–
–
Search Box
Search Core Results
Search Best Bets
Search High Confidence Results
Search Statistics
Search Paging
Search Action Links
Anatomy Of Results Page UI
SEARCH BOX
HIDDEN SHARED OBJECT
ACTION LINKS
STATISTICS
PAGINATION
HIGH CONFIDENCE MATCH
CORE SEARCH RESULTS
MATCHING
KEYWORDS
AND BEST BETS
Use of Search functionality in MOSS
•
•
•
•
•
•
Content Query Web Part
Login Name and Display Name
My Team Sites Web Part
My Links
Colleague Tracker Web Part
Alerts
Search Center
The Search Center,
the home for Enterprise Search
Business Data Search
Search data, not just documents
• Information in LOB systems is often hard to access
• MOSS 2007 can bring that data to your users
– Data is accessed through the Business Data Catalog
– Exposed to many features in SharePoint
• Search can easily index the data
– No need to write code
– Highly customizable results
– Integrated with scopes and search center
LOB Search
Find People and Expertise
Getting your job done means working with
the right people
Build rich personal profiles
– Customize personal profile attributes
– Manually update profile information
– Automatically populate profiles using AD, LDAP directories, or LoB systems
Publish personal profiles using SharePoint My Sites
– Create personal portals
– Share files, links, and other useful information with others
– Control access to information using security and privacy controls
Identify people who can help => Knowledge Network
– Find people based on keyword matches with SharePoint personal profiles
– Find “in-common” connections—managers, memberships, and colleagues
– Group results by social distance and filter by common attributes
Vragen?
Making Serious Sites
Knowledge Network
Making Serious Sites
Knowledge Network
•
•
•
•
•
•
Knowledge Network Overview
Client Profile Creation
SharePoint Server Integration
Privacy and Anonymous Brokering
Deployment and Administration
Extensibility
What is Knowledge Network?
Knowledge Network is software for enterprise social networking that helps
users collaborate more effectively by automating the discovery and sharing
of undocumented knowledge and relationships
Who knows whom?
Who knows what?
Better Decisions
More Quickly
Why Is Microsoft Creating KN?
• Three Core Beliefs
– Most information is undocumented
– It’s difficult to connect to the right person
– “Weak ties” deliver significant value
Core Belief #1
Most knowledge is undocumented
• Employees are more likely to turn to
colleagues for information*
• Not stored in documents
• Not stored in databases
• It’s Stored in peoples’ heads
Documented
Knowledge
20%
80%
Individual
Knowledge
Source: Delphi Consulting Group
Core Belief #2
It’s difficult to connect to the right person
• Finding the right person often involves a referral by
an intermediary
Core Belief #3
“Weak ties” deliver significant value
• People in our inner circle know basically the same people and
the same things that we know
• People that they know have connections that we don’t have
• By leveraging “weak ties” we can learn about new
opportunities, share new ideas or meet a new person or group
of people
You
1 Degree
Your Inner
Circle
2 Degrees
Their Extended
Circle
3+ Degrees
Their Extended
Circle
The KN Solution
5. People Results
Ranked by
4. Seeker
Searches for people
Social distance to seeker
Expertise and relationship
relevance
Who knows what
Who knows whom
2. Publish Profile
Incremental
updates
1. KN Client
Analyzes email to
create profile of
Keywords
Colleagues
External Contacts
2. Publish Profile
Incremental
updates
3. KN Server
Aggregate profiles
Expertise information
Social network
1. KN Client
Analyzes email to
create profile of
Keywords
Colleagues
External Contacts
Privacy Concerns
Striking the Right Balance
• The goal is to strike the right balance between
Utility, Simplicity and Privacy
– Utility is how useful will this software
be to me?
– Simplicity is how easy will this software be to
install, upload, maintain and use?
– Privacy is how much personal information will this
software reveal and how much control do I have?
The Simple Privacy Model
• Notification
– Communicate steps of the profile creation and publishing process
– Customers can expose privacy policy in the client profile wizard
• Control
– User can choose which items to include/exclude
– User can choose from 5 levels of privacy to apply to each
profile item to control who is allowed to view that information
on the server
– Administrators can configure the default operation of the client,
including opt-in/opt-out and the default privacy visibilities for profile
items
– Administrators can determine which aspects of the product
functionality to leverage including external contacts, anonymous
results, and DL keywords
• Consent
– KN sends no data to the server before the user has approved it
• Caveat is keywords associated with “public” Distribution Lists – aggregated
into DL profiles on the server
Top Customer Questions…
• Why client-side mining?
– Privacy – Nothing leaves user’s machine until they “publish”
– Access to information – PSTs, future data sources
– Distribute the processing
• Why not mine sources other than email?
– E-mail is by far the richest and most pervasive source today
– Calculating strength across different data sources adds complexity
• How long does analysis take?
– Depends on a number of factors
• On the amount of email and unique contacts
• On disk performance, RAM, and CPU
• User activity – since analysis process runs at low priority
Client Profile Creation
Select Information
Run Analysis
Review Profile
1.
2. Contact Resolution
3. Update
SetSync
Data Sources
each
emailto
Lookup contacts against Aggregate counts
• Read
Outlook
folders
GAL (MAPI)
analyze key
Capture
Check thresholds
data”
Internal or external?
• “interaction
Import Outlook
Calculate strength
in
MDB
contacts
Is it a DL? Discard
inInstant
contacts
• Read
Import
Capture key
from
Outlook
and
properties
Messaging contacts
IM clients
Sync Colleagues
from SPS profile
Publish Profile
4. Recommendation
“Exclusion lists”
Special rules
Limits applied
Organization name
mapping for
External Contacts
KNClient.log file and MDB files
C:\Documents and Settings\<user>\Local Settings\Application Data\Microsoft\Knowledge Network
SharePoint Integration – Overview
Extract and Publish
Active
Directory
People Profiles
Index and Search
Index
KN Client
MDB
Search Center
- Find People -
SharePoint Integration – Data
KN Client Data
Keywords
Colleagues
External Contacts
Client
Client
Server
Server
Publish
KN Data
to Server
KN Data
Keywords (public)
Colleagues
SPS User
Profiles
KN Profile
Management
Web Service
Crawl from SPS
AD property changes
SPS property changes
KN Data
Keywords (nonpublic)
External Contacts
KN Store
SharePoint Integration – My Site
Profile Detail Tabs
 Colleagues
 External Contacts
 Keywords
KN Member
Indicator
Keywords Property
 Summary of
public keywords
Keywords In Common
Desktop Search Integration
Anonymous Brokering
 Seeker Initiates
Search
 Reply to
Help or
Decline
 Request
 Anonymous
Results
 Brokered
E-mail
Deleted
Knowledge Network
Client Deployment
System Requirements
Configuration Options
• Windows XP SP2 or Vista
• Office 2007 or 2003
• Client policies (GPO)
– Strongly recommend
“Cached Exchange Mode”
• Exchange 2007 or 2003
– With Active Directory
• .NET Framework 2.0
–
–
–
–
–
–
Update interval
Recommendation limits
Default “Show To” privacy
Opt-in/out first experience
Anonymous brokering
Retention “clean up”
• Custom privacy policy
• KN Server settings
– Organization mapping
– Exclusion lists
Server Deployment And Admin
Deployment
Administration Options
• Leverages Office Server 2007
administration, provisioning,
and deployment
infrastructure
• View and manage jobs
• User management
– Manage KN “members”
– Manage External Contacts
• Client configuration
– Organization mapping
– Exclusion lists
• Server statistics
• E-mail templates
KN Extensibility
• Client-side
– Managed API exposing data in KN DB
– Retrieve user Keywords, Colleagues,
and External Contacts
• Server-side
– Web service exposing a full-fledged query
language for expertise/social network people search
• Can go beyond UI scenarios – e.g., multi-target queries
– Web service to retrieve (read) and augment (write) profile data
• Bootstrap the profile or add custom
keywords and contacts
Vragen?
Making Serious Sites
Search Administration
Improvements
Making Serious Sites
Search Administration
• Design goals
– Address SPS 2003 admin UI pain points
– Unify WSS and MOSS search
– Enable full programmability via the OM
– Even better scalability and performance
Improved Admin & Management
• Consistent search system across
WSS and SPS
• Revamped indexing management
– Simpler UI
– Better indexing logs
• Flexible scoping
– Scopes decoupled from
content sources
– Simple and compound scopes
• Schema
– Map foreign schema into search
system
– Streamlined UI
• Query and Result Reporting
– Get visibility into user
search behavior
– Logging for more
detailed custom reports
• Best Bets
– Editorial results for
specific queries
– Streamlined UI
• Continuous propagation
– Indexed content is immediately
searchable
• Updated object model and APIs
– Custom tools and services
Indexing Management
• Streamlined experience and more control
• One index per shared service  no need
to worry about managing discrete indexes
• Multiple start addresses per content source
• MOSS indexes can drive the WSS search
experience
– Allow upgrade from WSS to MOSS
Security
• Query-time security trimming in SPS 2003
– File shares, WSS/SPS 2003, Exchange, Lotus Notes
(via mapping)
• Now support pluggable authentication
for content in WSS/MOSS sites
– Based on ASP.NET 2.0 model
• Minimum required crawler permission is now
just Full Read, not administrator
– Still provides the same security trimming functionality
• Ability to remove Single Items
Search Administration
Customization
• Search in your company is different
– Different metadata might matter
• Documents: Title, Author, File location, size
• Records: Patient, Doctor, Healthcare provider, SSN…
– How users meaningfully scope searches differs
• “All finance documents”
• “All patient records”
• “All published documents”
• Customize results to “pop” metadata that matters
• Customization offered at many levels
– Webparts, XSLT/CSS, full object model…
– SDK available
Search Administration
Adding a tab
Customizing the Search results
Query Reporting
• Best way to improve search
is to understand current usage
• New out of the box usage reporting
– Query volume trends, top queries, click through
rates, queries with zero results, etc.
– At both site and service provider levels
– Export data for extended reporting in Excel
– Respond to feedback with configuration changes or
editorial results
Query Reporting
Query Reporting
Performance and Capacity Planning
Making Serious Sites
Performance Improvements
• Key new features make the crawls faster so the
content is fresher
– More efficient SharePoint Crawling
(Change Log Crawl)
– Continuous Propagation
– Unified WSS and MOSS search
– Security Change Only Crawl
• Maximum scale is 10s of Millions of documents per
indexer
Search Performance Planning
• Key learnings
– Think about freshness goals, change rate
– Evaluate remote server responsiveness
– Upgrade WSS 2003 sites to WSS 2007 sites to index them
faster
– Balance results freshness with load on servers
– In most cases, your search index is on one server
• Scale strategies might include
–
–
–
–
Dedicated WFE for crawling
Separate indexer machine
Separate SQL DB for Search
Fast disk for SQL, fast CPU for Indexer, more memory
Vragen?
Making Serious Sites
Customizing the Search UI
Making Serious Sites
Search UI Programmability
• Easy and powerful:
– Customize using web part properties
– Customize using XSLT, XML, and script
• Manipulate URL parameters
– Build web parts and apps using the query OM
• keyword and SQL syntax
– Remote apps using the query web service
Customize UI With XSLT
•
•
•
•
Web part property specifies query columns
Search web parts get results XML
Web parts apply transform to XML nodes
Search page URL parameters enable
query manipulations
• Appropriate for scenarios requiring
– Change results layout, look and feel
– Keyword syntax queries
Specify Properties To SELECT
• Properties must have been added as Managed
Properties
• XSLT uses them for display or logic
• Use the core search results web part 'Columns'
property in the tool pane:
<root>
<SelectColumns>
<Column Name=“ShirtSize”/>
<Column Name=“ShirtColor"/>
<Column Name=ShirtImage"/>
<...>
</SelectColumns>
</root>
Displaying Custom Properties
<xsl:call-template name="DisplayString">
<xsl:with-param name="propval" select=“ShirtSize">
<xsl:with-param name="proplabel“>Shirt Size:
</xsl:with-param>
<xsl:call-template>
.
.
(Later…)
.
.
<xsl:if test='string-length($propval) &gt; 0'>
<xsl:value-of select="$proplabel" />
<xsl:text> </xsl:text>
<xsl:value-of select:"$propval"/>
</xsl:if>
Property Filtering
• Pivot or drill-down on a property to offer
different queries
• Construct URL, re-post to page with
new query
<xsl:template name="DisplayPivot">
<xsl:param name="value" />
<xsl:param name="fieldname" />
<xsl:param name="label" />
- <a href="http://co113/Search/Pages/results.aspx?k={$fieldname}%3a{$value}">
<xsl:value-of select="$value"/>
<xsl:text> <xsl:text>
<xsl:value-of select="$label"/></a>
</xsl:template>
Property Filtering
<xsl:call-template name="DisplayPivot">
<xsl:with-param name="value" select="fileextension"/>
<xsl:with-param name="fieldname">
FileExtension
</xsl:with-param>
<xsl:with-param name="label">
File
</xsl:with-param>
</xsl:call-template>
Keyword Syntax
• Established search UI element
– Properties
monty -actor:"John Cleese"
monty -actor:John
• same-property clauses are ORed
• Equality only, prefix or exact phrase match
– And / Or / Not / Phrase
"chocolate covered" -frog
• No elaborate groupings
– Scopes:
• scope:movies
URL Request Parameters
• Content
– results.aspx?k=fish
• Scopes
– results.aspx?k=fish&s=BBC
• Sort
– results.aspx?v=date
– results.aspx?v=relevance
• Page
– results.aspx?start=21
Develop Search Solutions
Making Serious Sites
Using The Query OM
• Use query OM:
– To build custom search UI web parts
– To have direct access to query and results
properties
– When you need to manipulate results on the server
• Same flow, keyword or SQL syntax
– Set up query text and options
– Execute()
– Retrieve a set of IDataReader objects
Process Results
• 'ResultsTableCollection': IDataReaders
• Custom properties provide information on query
and query execution
• One table for each requested result type
– RelevantResults
– SpecialTermResults
– HighConfidenceResults
– DefinitionResults
Using The Query Web Service
• http://<server>/_vti_bin/search.asmx
• Two query methods:
– Query
• Research and Reference Servicecompatible XML
• Use in the Office Research pane
– QueryEx
• Returns .NET DataSet
• Full set of result types, features
Customizing With XSLT
Putting it all together
Vragen?
Making Serious Sites
Macaw
Beech Avenue 140 1119 PR Schiphol-Rijk Tel +31 (0)20 8 510 510 Fax +31 (0)20 8 510 511