Make your gui shine with ajax solr

MAKE YOUR GUI SHINE WITH AJAX SOLR

Troy D. Thomas Product Engineering Manager

Koorosh Vakhshoori Technical Architect

Make Your GUI Shine With AJAX Solr

Setting the Stage

Company Backgrounder

Project History

Business & Technical Requirements

Inspiration

Demo

Why AJAX Solr?

AJAX Solr Framework DEEP DIVE

Themes

Widgets

Challenges

Q & A

Agenda

Company - Background

Synopsys

25 year old company / 1.8B 2012 revenue

Electronic Design Automation (EDA)

Electrical engineers design computer chips

High level design

Simulation

Test

Place and route

IP blocks

Nearly every semiconductor built uses Synopsys software

microprocessors, RAM, etc.

Synopsys – What?

Company Background

SolvNet ® - online knowledge base system used by

customers and employees

Dedicated engineering team

20 year history

1993 Email

1995 A “Patchy” NCSA Web server + PERL CGI

1997 Verity Netscape search

2001 Java – Netscape Iplanet Portal + Verity

2005 Apache Lucene

2007 Pure Apache

2012 AJAX Solr / Solr 4

Synopsys – SolvNet ®

Lucene

Moved to Lucene - 2005

Custom tokenization helped results

+delay_mode_zero

Autocomplete - 2008

Custom Yahoo UI Widget

Relatively Complex

Tomcat w/ RMI callback

HTML parser

PDF text extraction using PDFBox

Generate Lucene documents

Separate collections – Articles, Docs, etc.

Open Source wins…

Requirements

Voice of the customer

Make it faster

Easier to use

Better results

Can you just use Google?

Technical team concerns

Custom code

Lucene aging

JSF foundation - GUI complexity

Business / Technical

Project Inspiration

Advanced Full-Text Search Capabilities

Optimized for High Volume Web Traffic

Standards Based Open Interfaces - XML,JSON and HTTP

Comprehensive HTML Administration Interfaces

Server statistics exposed over JMX for monitoring

Scalability - Efficient Replication to other Solr Search Servers

Flexible and Adaptable with XML configuration

Extensible Plugin Architecture

Solr Uses the Lucene Search Library and Extends it!

A Real Data Schema, with Numeric Types, Dynamic Fields, Unique Keys

Powerful Extensions to the Lucene Query Language

Faceted Search and Filtering

Advanced, Configurable Text Analysis

Highly Configurable and User Extensible Caching

Performance Optimizations

External Configuration via XML

An Administration Interface

Monitorable Logging

Fast Incremental Updates and Index Replication

Highly Scalable Distributed search with sharded index across multiple hosts

XML, CSV/delimited-text, and binary update formats

Easy ways to pull in data from databases and XML files from local disk and HTTP sources

Rich Document Parsing and Indexing (PDF, Word, HTML, etc) using Apache Tika

Multiple search indices

Apachecon 2010

Solr 4

Solr

Faceting!

Modernize GUI

Deprecate custom code base

Autocomplete using Yahoo UI

Did you mean?

Tika

ExtractingRequestHandler (Solr Cell)

Extract more mime types

Solr 4 (Trunk) DirectSolrSpellChecker

More like this

Synonym list

Future migration path

Why?

Front-End

Screenshot / Demo

Front-End

Screenshot / Demo / 2

Requirements

Dynamic GUI Interface AJAX Solr / Solr 4

Faceting

Autocomplete

Spell checker

Did you mean?

Deeper search (attachments) Tika

Better hit highlighter Solr 4

Technical Translation

Why AJAX Solr?

Limited resources / no budget

No GUI designers on staff

Limited specialization

“Enough” JavaScript – but not experts

Dynamic GUIs are commonplace (AJAX)

Autocomplete

Faceting

AJAX Solr framework helps with all of these

Considerations

AJAX Solr

Modern Standards JQUERY / JSON friendly

Widget Framework Autocomplete

Query submit

Save settings

Sort /display results

Pagination

Facet by product

Facet by doc type

Challenges Session management

Strength of community (Google Group)

Proxy Server

Benefits / Challenges

AJAX Solr DEEP DIVE

Solr Search Server

Overall system architecture

Index

Browser Client

Proxy Server

Web Application

Server

Why AJAX Solr?

• Dynamic GUI

– Faceting

– Autocomplete

• Faster GUI

– Less data traffic between server and client

– Less area to refresh

– Less mouse clicks

• Intuitive

What is AJAX Solr?

It is JavaScript framework-agnostic, but

requires an AJAX implementation for

communication with Solr. As such, you may

use the library whether you develop using

jQuery, MooTools, Prototype, Dojo, or any

other framework.

AJAX Solr Wiki

Intro AJAX Solr Framework

• MVC on top of an Ajax framework

– Our implementation uses Jquery

• Main components

– Manager

– Parameters

– Widgets

– Theming

Search Query

Search Black Box View

Browser Client

index.jsp

solr/select?...&q=clock%20tree%20power&..

Web Application

Server

Solr Server

Search hits JSON

Response

Search Results Response (JSON) { "responseHeader":{ "status":0, "QTime":960, "params":{ ... "facet":"true", "facet.field":[... "{!ex=products}products"], ... "fq":[... "{!tag=products}products:\"Synplify FPGA\""], ... "q":"clock tree power", ... "spellcheck":"true", ....}}, "response":{"numFound":12,"start":0,"maxScore":1.5953931,"docs":[ { ... "id":"913646", ... "title":" Clock tree fan out using syn_maxfan", ... "products":["Synplify FPGA"], "teaser":"... Question: I have a design and I am targetting Spartan devices. The clock is not meeting timing in a design, ..."},

... }, "facet_counts":{ ... "facet_fields":{ ... "products":{ ... "Synplify FPGA":12, ...}}, ....}, "highlighting":{ "913646":{ "title":[" Clock tree fan out using syn_maxfan"], "indexed_content":["The clock is not meeting timing in a design,..."]}, ...}}, "spellcheck":{ "suggestions":{}}}

Autocomplete

Autocomplete Black box view

Browser Client

solr/terms?...&terms.regex=.*[\_-]clo.*|^[+|\.|%60|-]?clo.*&...

Solr Server

autocomplete JSON Response

Autocomplete response

{ "responseHeader":{ "status":0, "QTime":377}, "terms":{ "indexed_content":{ "run_clock_create_tree":41, "clock_crossing":15, "timing_crpr_remove_muxed_clock_crp":6, "report_clock_crossing":5, "crpr_remove_muxed_clock_crp":3, ...} } }

AJAX Solr Architecture

Web Browser

Widget (View)

Manager (Controller)

ParameterStore (Model)

Solr Server

Theme (Helpers)

Manager

• Set solrUrl

Manager = new AjaxSolr.Manager({ solrUrl: 'http://www…com/solr/‘ });

Solr Search Server

More on Manager

Index

Browser Client

Proxy Server

Web Application

Server

Direct or Proxy

Manager Life Cycle

Manager ParameterStore Widget Widget Widget

On index.jsp page load

Instantiate

setStore()

Instantiate

Instantiate addWidget() For each widget

init()

init()

init()

For each widget

For each widget

doRequest() beforeRequest()

executeRequest()

For each widget afterRequest()

Ajax call

ParameterStore

• Solr parameters – Single: q, start

– Multi valued: fq

• Methods – get, add, addByValue, …

Manager.store.addByValue('q', '*:*');

• Derived classes – ParameterHashStore

– ParameterHistoryStore

– ParameterYUIStore

ParameterHashStore

• Esoteric Requirements

– Bookmarking

– Back button

– URL masking

Manager.setStore(new AjaxSolr.ParameterHashStore()); Manager.store.exposed = [ 'fq', 'q', 'start', 'sort']; Manager.init();

Widgets

• What is a Widget?

• AbstractWidget

– id, target

• Derived Widgets

– AbstractFacetWidget

– AbstractTextWidget

– AbstractSpellcheckWidget

– AbstractSpatialWidget

– PagerWidget

AbstractWidget AjaxSolr.AbstractWidget = AjaxSolr.Class.extend( { //Attributes

id: null, target: null, //Methods init: function () { … }, beforeRequest: function () { … }, afterRequest: function () { … }

}

Manager Life Cycle

Manager ParameterStore Widget Widget Widget

On index.jsp page load

Instantiate

setStore()

Instantiate

Instantiate addWidget() For each widget

init()

init()

init()

For each widget

For each widget

doRequest() beforeRequest()

executeRequest()

For each widget afterRequest()

Ajax call

AbstractTextWidget

• The widget that manipulates

the ’q’ parameter

• Example implementations:

– TextWidget.q.js

– AutocompleteTermWidget.js

AutocompleteTermWidget

First term Two or more terms

AutocompleteTermWidget AjaxSolr.AutocompleteTermWidget = AjaxSolr.AbstractTextWidget.extend( { //Attributes

field: null, tokenized: true, … //Methods init: function () { //Event binding such as Keydown, <CR> and Autocomplete. }, beforeRequest: function () { //Example, blur & disable input field. }, afterRequest: function () { //Example, enable focus input field. }

}

AutocompleteTermWidget

• Suggestion algorithms

– First term: TermsComponent

– Second, third, etc. : facet.prefix

• Analytics

– Search query

– Autocomplete

ResultWidget

ResultWidget AjaxSolr.ResultWidget = AjaxSolr.AbstractWidget.extend( {

//Methods init: function () { //Event binding such manipulating results, say hide details. }, beforeRequest: function () { //Example task done here, loading image. }, afterRequest: function () { //Main task of showing the hit results. }

}

ResultWidget

• Function: Show the search results. Based

on tutorial example

• Extends AbstractWidget

• Analytics

– Total hit count

– Which filters and facets

– Selected content type

PagerWidget

• Function: Pager footer for going to next,

previous result pages, etc.

• Extends AbstractWidget

PagerWidget AjaxSolr.PagerWidget = AjaxSolr.AbstractWidget.extend( { //Attributes

… //Methods init: function () { //Empty }, beforeRequest: function () { //Empty }, afterRequest: function () { //Main task of building the pager. }

}

AbstractFacetWidget

• For implementing

search faceting

• Product, DocType

AbstractFacetWidget AjaxSolr.ProdTypeWidget = AjaxSolr.AbstractFacetWidget( { //Single vs. multiValue fq

multiValue: false … //Methods init: function () { //Initializes the facet field and related parameters. }, beforeRequest: function () { //Empty}, afterRequest: function () { //Setup click handler for the selected products. } clickhandler: function() { doRequest for selecting a product.} unclickhandler: function() { doRequest when deselecting a product.}

}

Theme

• Separate presentation layer

– Result widget: Display metadata and

summary

– Facet widget: Attach click handler

AjaxSolr.theme.prototype.facet_link = function (value, handler) { return $('<a href="#"/>').text(value).click(handler); };

AjaxSolr.theme('facet_link', facet_value,facetHandler)

Other Requirements

• Interaction with other applications

– Third party JavaScript • Analytics, Survey

– Backend applications • Recent searches

• Personalized search profile

• Error handling

– Ajax Exception

– Application logging

Security concerns

• Coarse vs. Granular

– Proxy blocking

– Role based filter

• Session management

– Error handling

– Timeout

Debugging

• Browser side debugging

– Firebug on Firefox

– Chrome developer tools

• Raw Solr JSON/XML

– Learn it!

Performance Tuning

• Reduce Traffic between Server and

Browser

– Filter on server

– Minimize browser side computation

– Solr schema consideration

– Compress JavaScript & CSS

• Proxy Server Caching

– JavaScript, CSS

Housekeeping

• Jquery & Jquery-UI upkeep

• Browser compatibilities

– NO IE 6 support

– IE 10

– Plugins

Summary

• Make your GUI Shine with AJAX Solr

• Understanding of the Framework

• Best practices

• Challenge and possibilities

Acknowledgements

• AJAX Solr creator James McKinney

• Lucene / Solr community

• SolvNet Development team

References

• AJAX Solr Wiki

https://github.com/evolvingweb/ajax-solr/wiki

• Google group ‘ajax-solr’

https://groups.google.com/forum/?fromgroups#!

forum/ajax-solr





https://groups.google.com/forum/?fromgroups




CONFERENCE PARTY

The Tipsy Crow: 770 5th Ave

Starts after Stump The Chump

Conference badge gets you in

TOMORROW

Breakfast starts at 7:30

Keynotes start at 8:30

CONTACT US

Education

Make your gui shine with ajax solr