The power of faceted search in alfresco

The Power of Faceted Search in Alfresco

Roxana AnghelutaWillem Van den Eynde

This presentation

● Who are we ?● Alfresco● Faceted Search● Demo

Willem Van den Eynde

● 2004 - 2007○ Bachelor applied informatics, KHLeuven

● 2006 - 2007○ LiU Erasmus Sweden

● 2007○ internship in Paris

● 2007 - 2010○ Master in applied informatics KULeuven

● 2010 - current○ Software Engineer XeniT Leuven

Roxana Angheluta

● 1995 - 1999○ Bachelor informatics, University of Bucharest

● 2000 - 2001○ Erasmus student KULeuven

● 2001-2004○ Assistant researcher KULeuven

● 2003-2004○ Master in Artificial Intelligence KULeuven

● 2004 - 2012○ Software Engineer Attentio Brussels

● 2012 - 2013○ Software Engineer XeniT Leuven

2009 - Proprietary and Confidential Information of Xenit Solutions

Introducing XeniT

Managing content in a smart way

2009 - Proprietary and Confidential Information of Xenit Solutions

From our home base

In collaboration with our customers

With an enthusiastic and experienced team

2009 - Proprietary and Confidential Information of Xenit Solutions

The corporate story of XeniT

2007 2008 2009 2010

IWT projectConcurrent

collaboration

3.5 M docs

8 M docs

2011 2012

Alfresco-As-A-

Service

2013

Alfresco is the largest private, pure-play open source software company in the world.

4 million+ downloads of Alfresco community75,000+ sites running community2000+ Enterprise customers from 43+ countries200+ channel partners20 consecutive quarters of revenue growthfounded in 2005

Maidenhead, UK Global Headquarters Atlanta, US Headquarters

What is Alfresco

Alfresco is an open source enterprise content management system

What is Alfresco ?

● Enterprise Content Management (ECM)

is a formalized means of organizing and storing an organization's documents, and other content, that relate to the organization's processes. The

term encompasses strategies, methods, and tools used throughout the lifecycle of the

content.

Classification and Retrieval

● Classification● Retrieval

FAQ

● How does an open-source company like Alfresco generate revenue ?

● Alfresco vs Microsoft SharePoint

2009 - Proprietary and Confidential Information of Xenit Solutions

Alfresco demo

Search in Alfresco

● Many search engines out there, few engines really good, fewer open source

● Requirements:○ accurate○ performant○ flexible○ cross-platform○ scalable○ mature

● Lucene○ https://lucene.apache.org/

● Starting with Alfresco 4.0 => Solr○ http://lucene.apache.org/solr/

Lucene

● Java-based indexing and search library, as well as spellchecking, hit highlighting and advanced analysis/tokenization capabilities

● HistoryDoug Cutting originally wrote Lucene in 1999.[2] It was initially available for download from its home at the SourceForge web site. It joined the Apache Software Foundation's Jakarta family of open-source Java products in September 2001 and became its own top-level Apache project in February 2005.

● Many projects based on Lucene: Solr, Nutch, Elasticsearch

Lucene

Indexing

● over 150GB/hour on modern hardware● small RAM requirements -- only 1MB heap● incremental indexing as fast as batch indexing● index size roughly 20-30% the size of text indexed

Searching● ranked searching -- best results returned first● many powerful query types: phrase queries, wildcard queries, proximity queries, range queries and more● fielded searching (e.g. title, author, contents)● sorting by any field● multiple-index searching with merged results● allows simultaneous update and searching● flexible faceting, highlighting, joins and result grouping● fast, memory-efficient and typo-tolerant suggesters● pluggable ranking models, including the Vector Space Model and Okapi BM25● configurable storage engine (codecs)

2009 - Proprietary and Confidential Information of Xenit Solutions

Lucene in Alfrescohttp://www.slideshare.net/JM.Pascal/alfresco-search-tutorial-presentation

2009 - Proprietary and Confidential Information of Xenit Solutions

Lucene in Alfrescohttp://www.slideshare.net/JM.Pascal/alfresco-search-tutorial-presentation

2009 - Proprietary and Confidential Information of Xenit Solutions

Lucene in Alfrescohttp://www.slideshare.net/JM.Pascal/alfresco-search-tutorial-presentation

2009 - Proprietary and Confidential Information of Xenit Solutions

Lucene in Alfrescohttp://www.slideshare.net/JM.Pascal/alfresco-search-tutorial-presentation

2009 - Proprietary and Confidential Information of Xenit Solutions

Lucene in Alfrescohttp://www.slideshare.net/JM.Pascal/alfresco-search-tutorial-presentation

2009 - Proprietary and Confidential Information of Xenit Solutions

Lucene in Alfrescohttp://www.slideshare.net/JM.Pascal/alfresco-search-tutorial-presentation

The way to preserve information in the Lucene index is specified in Alfresco's data modelsMain concept: tokenization

2009 - Proprietary and Confidential Information of Xenit Solutions

Lucene in Alfrescohttp://www.slideshare.net/JM.Pascal/alfresco-search-tutorial-presentation

with

without tokenization with tokenization

2009 - Proprietary and Confidential Information of Xenit Solutions

Lucene in Alfresco

● Out of the box search:○ search in all items, in a certain property or in the

content (full text search)

○ additionally: PATH, ASPECT, CATEGORY searches

○ Lucene syntax allowed:■ boolean queries■ wildcard queries■ range queries

Solr

● Standalone full-text search server within a servlet container such as Tomcat. Uses Lucene library and has REST-like HTTP/XML and JSON API. Has an extensive plugin architecture.

● In 2004, Solr was created by Yonik Seeley at CNET_Networks and in January 2006 the source code was donated to the Apache Software Foundation under the Lucene top-level project. In March 2010, the Lucene and Solr projects merged and consequently in 2011, the Solr version number scheme was changed in order to match that of Lucene.

● Many users:○ http://wiki.apache.org/solr/PublicServers

Solr

● Uses the Lucene library for full-text search● Faceted navigation● Hit highlighting● Query language supports structured as well as textual search● JSON, XML, PHP, Ruby, Python, XSLT, Velocity and custom Java binary output formats over HTTP● HTML administration interface● Replication to other Solr servers - enables scaling QPS● Distributed Search through Sharding - enables scaling content volume● Search results clustering based on Carrot2● Extensible through plugins● Pluggable relevance - boost through formula● Caching● Embeddable in a Java Application

Faceted Search in Alfresco

● A way to navigate through the documents, showing counts per property value and offering the possibility to drill down in the data

● Faceted search supported by Lucene/Solr, not yet supported by Alfresco

● Implemented by Xenit in Fred

Faceted Search in Alfresco

Faceted Search in Alfresco

● Questions○ which fields should be facetable?

■ only the ones with a limited set of possible values■ only the ones which are untokenized■ plus ranges: dates and numbers

○ how to navigate inside facets?

● Current implementation○ facetable fields configurable in a file○ date ranges and number ranges not supported yet○ drilling-down in a single value possible

Faceted Search in Fred: mockup

Demo