Embed Size (px)
Citation preview
Amazon CloudSearch
& Future Search Architectures Iain Fletcher [email protected]
The expert in the search space
The expert in the search space
Agenda
A brief introduction to Search Technologies
Search Application Architectures
Common challenges, emerging architectures
Our thoughts about Amazon CloudSearch
Search Technologies Overview
San Diego, CA
San Jose, CR
Herndon, VA
Ascot, UK
Cincinnati,
OH
Karlsruhe, DE
• The leading IT Services company dedicated to Enterprise Search & Search-based Applications
• Implementation, Consulting, Managed Services
• 120 employees and growing
• Independent, working with all of the leading software vendors and open source alternatives
500+ Customers
Future Search Architectures
The expert in the search space
The expert in the search space
A Traditional Integrated Architecture
Integrated Search Engine Integrated Search Engine Content
Sources Connectors Connectors Index Pipeline Index Pipeline
Search Index
Search Index Employee
Directory Employee Directory
CMS CMS
File Share File Share
UI UI
Etc. Etc.
The year is 2013, but we still have:
Dissatisfaction with search results
Scalability issues (various)
High initial investment
Inflexibility in the model
Data integration issues…
The expert in the search space
Reasons for Change
Today’s leading “indexing engines” are very capable
The challenges to success almost always lie in peripheral tasks. On the indexing side of the house, this means:
Gathering and feeding data to the search engine
Preparing data for indexing (normalization, metadata enrichment. Etc.)
The expert in the search space
Reasons for Change
The need to cope with continuous and rapid data growth
Things usually break first on the data gathering side of the house
E.g. The Government department and the 7TB File Server
The expert in the search space
A Traditional Integrated Architecture
Integrated Search Engine Integrated Search Engine Content
Sources Connectors Connectors Index Pipeline Index Pipeline
Search Index
Search Index Employee
Directory Employee Directory
CMS CMS
File Share File Share
UI UI
Etc. Etc.
RE-INDEX
The expert in the search space
A Better Search Architecture
Search Engine Search Engine Content
Sources Connectors Connectors
Index
Pipeline
Index
Pipeline Search Index
Search Index Employee
Directory Employee Directory
CMS CMS
Etc. Etc.
RE-INDEX
Content
Processing
Content
Processing
Secure xml
Cache
Secure xml
Cache
Iterative
Development
Iterative
Development
Better operational agility
The expert in the search space
Amazon CloudSearch Amazon CloudSearch Content
Sources Connectors Connectors
Index
Pipeline
Index
Pipeline Search Index
Search Index Employee
Directory Employee Directory
CMS CMS
Etc. Etc.
RE-INDEX
Content
Processing
Content
Processing
Secure xml
Cache
Secure xml
Cache
Iterative
Development
Iterative
Development
Better operational agility
Much improved cost of ownership
A Better Search Architecture
The expert in the search space
Let’s take a look at Amazon CloudSearch
If you’ve ever switched on an EC2 server, then you can switch on a CloudSearch server
And of course, switch it off just as easily
Xml or JSON in, for indexing
RESTful / xml out, for searching
Has the bases covered in terms of core search functionality
The expert in the search space
Amazon CloudSearch: Important Search Functions
Full-on Search Navigators
Results sorting by property
Plenty of relevancy tuning controls
Indexing speed is impressive
We’ve tested up to 500 docs/sec
Very long results lists can be served
The expert in the search space
Cost Effective & Flexible
The usual benefits of AWS Only use what you need
Elastic scalability
Maintenance inclusive
Compare that to the cost of a Service Level Agreement for an on-premise alternative
Then add the labour costs….
From less than £50 per month
The expert in the search space
What do you need to get started
Repository Connectors
Capture content
Convert to xml
XSLT. Etc.
Content Processing
Java code, Groovy. Etc.
A framework product, E.g. Aspire
Commercial software to suit the purpose
A User Interface
Amazon S3
Atlassian Confluence
Documentum
eRoom
Secure File Systems
IBM Connections
JIVE
Lotus Notes
RDBMS
SharePoint 2010, 2013
SocialCast
TeamForge
Web Crawling
The expert in the search space
Search-specific UI/UX builders are available
The expert in the search space
What Applications Suit Amazon CloudSearch?
In principle any
CloudSearch is an option that everyone should consider for search-based applications
Especially where elastic scalability is an advantage
Or where the data already lives in the cloud, including S3 buckets
The expert in the search space
To conclude
If you like Amazon Web Services, then you’ll like Amazon CloudSearch for the same reasons
It is a full-function indexing engine, ideally suited to agile, flexible search architectures
If you are looking for a search-based application, it should be on your shortlist
For many applications, Amazon CloudSearch will prove to be highly cost effective
My email: [email protected]
Useful URLs:
Connectors for Cloudsearch
www.searchtechnologies.com/cloudsearch-data-connectors.html
Aspire Content Processing Framework
www.searchtechnologies.com/aspire.html
A Java API for Amazon Cloudsearch
www.searchtechnologies.com/java-api-amazon-cloudsearch.html
Enterprise Search Surveys
www.searchtechnologies.com/enterprise-search-surveys.html
Blog: Searching Wikipedia with Amazon CloudSearch
www.searchtechnologies.com/cloudsearch-data-connectors.html
White Paper: A Big Data Architecture for Search
www.searchtechnologies.com/big-data-enterprise-search-architecture.html
The expert in the search space
