Transcript Solr facets in Alfresco 4

Solr Facets in Alfresco 4
Dan Tuffery
Who Am I?
About Me
• Senior Software Engineer at Ixxus
• Working with Alfresco & Solr for 2.5 years
• Alfresco Certified Engineer and Solr Certified
• Winner of Alfresco’s 2012 Dashlet Challenge
Agenda
Talk Outline
• Alfresco Solr facets demonstration
• Solr facets overview
• Walkthrough of implementation
• Plans for the module
Alfresco Solr Facets Demonstration
Solr Facets Overview
3 Different flavours of facets
• Field facets
• Date facets
• Number range facets
Indexing Facet Fields
• Indexed
• Not Tokenised
• Not Stored
Facet Field
Parameters
• facet = enables faceting
• facet.field = the field that you want to facet on
• facet.mincount = constraint count >= facet.mincount
• facet.limit = limit the number of constraints that are
displayed
Facet Field
Parameters
• facet = enables faceting
• facet.field = the field that you want to facet on
• facet.mincount = constraint count >= facet.mincount
• facet.limit = limit the number of constraints that are
displayed
q=myfield:solr&facet=true&facet.field=myfield&facet.mincount=
1&facet.limit=20
Date Facet
Parameters
• facet.date = name of date field
• facet.date.start = start of the date range
• facet.date.end = End of the date range
• facet.date.gap = period of time in between each date
range
Date Facet
Parameters
• facet.date = name of date field
• facet.date.start = start of the date range
• facet.date.end = End of the date range
• facet.date.gap = period of time in between each date
range
q=myfield:solr&facet=true&facet.date=created&facet.limit=20
&facet.date.start=2012-0101T00:00:00.0Z&facet.date.gap
=1MONTH&facet.date.end=2012-10-22T00:00:00.0Z
Solr Facets In Alfresco 4
What Needed To Be Done?
Alfresco’s Facet Implementation
Out Of The Box
• Facet field POJO
• Logic to add facet field params to Solr request URL
• Facet results are mapped to JSON object
My Implementation
Five Main Tasks
1. Add facets to Alfresco (Facet Manager)
2. Solution to add facets to search request
3. Solution to return facets in the search response
4. Modify search results page to display facets
5. Add support for filter queries
1)Add Facets to Alfresco
Facet Manger
• Tool in Admin Console
• Based on current Tag Manager
• Create, Read, Update and Delete facets
• REST API in Alfresco (POST, GET, PUT, DELETE)
• Custom content model – ix:facetField
1)Add Facets to Alfresco
What facets to display
• Content Type, Mime type, Property, Date
How the facet should be displayed
• Display Name, Minimum Count, Limit
When the facet should be displayed
• Individual Site, All Sites, Repository Search
Indexing Facet Fields in Alfresco
Best Practices
• Indexed
• Not Tokenised
• Not Stored
Indexing Facet Fields in Alfresco
Best Practices
• Indexed
• Not Tokenised
• Not Stored
Indexing Facet Fields in Alfresco
Best Practices
• Indexed
• Not Tokenised
• Not Stored
Indexed Field Name
Indexed Value(s)
@{http://www.ixxus.com/model/1.0}location
{en}new
{en}york
@{http://www.ixxus.com/model/1.0}location.__
new
york
@{http://www.ixxus.com/model/1.0}location.u
{en}New York
@{http://www.ixxus.com/model/1.0}location.__.u
New York
@{http://www.ixxus.com/model/1.0}location.sort
enNew York
Search Flow of Execution OOTB
Search Flow of Execution
2) Add Facets to Search Request
• Search request sent from Share
• search.lib.js
var queryDef = {
query: ftsQuery,
language: "fts-alfresco",
page: {maxItems: params.maxResults * 2},
templates: getQueryTemplate(),
defaultField: "keywords",
onerror: "no-results",
sort: sortColumns,
searchScope: searchScope,
siteId: params.siteId,
filterQueries: filterQueries
};
nodes = search.query(queryDef);
2) Add Facets to Search Request
• Extended Search.java
String searchScope = (String)def.get("searchScope");
String siteId = (String)def.get("siteId");
..
..
if(searchScope != null) {
List<FieldFacet> fieldFacets =
facetService.getFieldFacets(searchScope, siteId);
searchParameters.setFieldFacets(fieldFacets);
}
..
results = this.services.getSearchService().query(searchParameters);
2) Add Facets to Search Request
• Extended SolrQueryHttpClient.java
• Generates Solr Request URL
• Add parameters (including facets) to URL
List<FieldFacet> fieldFacets = searchParameters.getFieldFacets();
for(FieldFacet facet : fieldFacets) {
url.append("&facet.field=" + facet.getField());
url.append("&facet.mincount=" + facet.getMinCount());,
url.append("&facet.limit=" + facet.getLimit());
}
2) Add Facets to Search Request
• Extended SolrQueryHttpClient.java
• Generates Solr Request URL
• Add parameters (including facets) to URL
List<FieldFacet> fieldFacets = searchParameters.getFieldFacets();
for(FieldFacet facet : fieldFacets) {
url.append("&facet.field=" + facet.getField());
url.append("&facet.mincount=" + facet.getMinCount());,
url.append("&facet.limit=" + facet.getLimit());
}
facet.field=@{http://www.ixxus.com/model/1.0}location.__.u
3) Return Facets in Search Response
• Results mapped to SolrJSONResultSet.java
• Response returns set of ScriptNode objects
Collection<Object> set = new LinkedHashSet<Object>(results.length(), 1.0f);
..
//OOTB code to add scriptNodes to the set
..
if(searchParameters.getFieldFacets().size() > 0) {
List<FieldFacet> fieldFacets = sp.getFieldFacets();
for(FieldFacet facetField : fieldFacets) {
List<Pair<String, Integer>> facets =
((SolrJSONResultSet)results).getFieldFacet(facetField.getField());
set.add(new ScriptFacet(facetField, facets, getScope(), this.services));
}
}
return set;
3) Return Facets in Search Response
• ScriptFacet Implements Scopable
• Represents a facet field and its properties
• Constraint filter query is constructed here
public ScriptFacet(IxxusFieldFacet facetField, List<Pair<String, Integer>>
facetList, Scriptable scope, ServiceRegistry services) {
this.services = services;
this.facetField = facetField;
this.scope = scope;
this.constraints = getFacetResults(facetList);
}
public Boolean isFacet() {
return isFacet;
}
3) Return Facets in Search Response
• Back up to search.lib.js to process results
var results, i, j, facetResults;
for (i = 0, j = nodes.length; i < j; i++) {
if(nodes[i].isFacet) {
var facet = nodes[i];
if(facet.facetHasAnyHits) {
facetResults.push(facet);
}
} else {//it is a ScriptNode object, so process it in the usual way
..
//OOTB code to add search results to results list
}
}
return ({
items: results,
facetItems: facetResults
});
3) Return Facets in Search Response
• search.get.json.ftl
"items":
[
...
],
"facets":
[
{
"facetDisplayName":"Location",
“constraints" :
[
{
"constraint": "San Jose (7)",
"fq":"@{http://www.alfresco.org/model/content/1.0}title.__.u:San Jose"
},
{
"constraint":"Berlin (4)",
"fq":"@{http://www.alfresco.org/model/content/1.0}title.__.u:Berlin"
},
...
]
}
]
4) Modify Search Results Page
• Search Results page layout changed dynamically
• Nested divs to display facet column
4) Modify Search Results Page
• search.js (YUI)
var facets = oFullResponse.facets, resultsContainer,
facetsDiv, results, body;
if(facets && facets.length > 0) {
resultsContainer = new Element(document.createElement("div"), {
id: “results-container"
}).addClass("results-container");
facetsDiv = new Element(document.createElement("div"), {
id: "facetsDiv"
}).addClass("search-facets");
Ixxus.search.solr.generateFacets(me, facets, facetsDiv);
body = new Element(Dom.get("body"));
results = new Element(Dom.get("results"));
results.addClass("results-right");
resultsContainer.appendChild(facetsDiv);
resultsContainer.appendChild(results);
body.appendChild(resultsContainer);
}
5) Add Support For Filter Queries
• Each facet constraint is a link
• Original search is executed + filter query/queries
• Same flow of execution as facets
• Filter queries added to the Solr request URL
5) Add Support For Filter Queries
• SolrQueryHttpClient.java
List<String> filterQueries = searchParameters.getFilterQueries();
for(String filterQuery : filterQueries) {
filterQuery = filterQuery.replace("{", "\\{");
filterQuery = filterQuery.replace("http", "http\\");
filterQuery = filterQuery.replace("}", "\\}");
filterQuery = filterQuery.replace(" ", "*");
filterQuery = filterQuery.replace("TO", " TO ");
url.append("&fq=").append(encoder.encode("{!lucene}" + filterQuery, "UTF8"));
}
Module Plans
Ixxus Solr
• Code has been contributed back to Alfresco
• Alfresco Enterprise 4.2 (Spring 2013)
• Maybe other community release before then
Module Plans
TODO
• Number range facets
• Multi field faceting
• Hierarchical faceting
Additional Information
• Search Terms Dashlet
• Uses faceting in Alfresco 4
•
https://github.com/ixxus/ix-search-terms-dashlet
• Ixxus blog
•
http://www.ixxus.com/blog/2012/10/solr-facets-alfresco-walkthrough/
Summary
• Easy and configurable way to add facet fields
• Solr facets
• Field Facets
• Date Facets
• Support for filter queries
Questions?