Delivering Search for Today's Local, Social, and Mobile Applications

Jon Handler

Delivering Search for Today's Local, Social, and Mobile

Applications

CloudSearch Solution Architect

Agenda

• Search for social and mobile applications

• Amazon CloudSearch

• Customer stories and panel

– Tim Ramsey, CTO/Co-founder for Ziplist / Conde Nast

– Pat Binkley, VP Engineering for Zumobi

– Ken Chung, CTO for Viddy

Time Spent Online (source: go-gulf.com)

Social Activities

• People are talking!

• Word of mouth is the primary factor behind

20% to 50% of all purchasing decisions

(Jonah Berger, "Contagious")

• Search brings relevance to the process of

pulling in friends

The Rise of Mobile Search (Source: BIA / Kelsey)

• In 2011, Search was 75% desktop,

25% mobile

• In 2013, they project 60% / 40%

• By 2016, 57% mobile, 43% desktop

In the Mobile Space, Local Is Growing

The Path to Search

• Search is central to user experience

• Options for building a search experience: BYO, open source, legacy enterprise

• Challenges

Amazon CloudSearch: Simple Search Simply

• Pay for infrastructure as you need it

• Lower total cost of operation

• No need to guess at capacity

• Increase innovation – low risk, inexpensive, simple experimentation

• Does the undifferentiated heavy lifting

• Available in 5 regions: go global in minutes

Amazon CloudSearch Overview

DNS / Load Balancing AWS Query

Search API Console Config

API

Command

Line Interface Console Doc

Svc API

Command

Line Interface Console

SEARCH SERVICE Search Documents

DOCUMENT SERVICE Add Documents

Update Documents

Delete Documents

Create Domains

Configure Domains

Delete Domains

CONFIG SERVICE

Search Domain

CloudSearch in Context

Automatic Scaling

SEARCH INSTANCE Index Partition n

Copy 1

SEARCH INSTANCE Index Partition 2

Copy 2

SEARCH INSTANCE Index Partition n

Copy 2

SEARCH INSTANCE Index Partition 2

Copy n

SEARCH INSTANCE

DATA Document Quantity and Size

TRAFFIC Search Request Volume and Complexity

Index Partition n Copy n

SEARCH INSTANCE Index Partition 1

Copy 1

SEARCH INSTANCE Index Partition 2

Copy 1

SEARCH INSTANCE Index Partition 1

Copy 2

SEARCH INSTANCE Index Partition 1

Copy n

Pricing

• Get started for just $2.40/day; $75/month

• AWS Calculator http://calculator.s3.amazonaws.com/calc5.html

Get started now – 30 days free

Customer Stories

Searching Recipes and Recipe Boxes on ZipList

Tim Ramsey

Chief Architect and Co-Founder,

Ziplist Inc.

ZipList.com - High-Level Description and Context

• Shopping List

• Recipes and Recipe Box

o 1.6M Recipes in Global Index

o 25M Recipes in Boxes

• 3.5M Users

• iPhone and Android Mobile Apps –

315K Downloads

• Partners

o 300+ White Label Partners

(SkinnyTaste.ZipList.com)

o 1,500+ Recipe Partners

(DarcyDiva.com)

• Hardware/Software Platform

o Amazon EC2

o Amazon RDS

o Amazon CloudSearch

o Ruby on Rails

ZipList Recipes

Indexed from Around the Web

• Structured (AllRecipes.com) or

Unstructured (Saveur.com)

• Recipes collected with and without

website cooperation

• Pushes to API or JIT with the Recipe

Clipper

* Attributes used in recipe box search/filtering

Attributes Collected and Indexed

• Title*

• Description

• Photo* (yes/no)

• Publisher* (Martha Stewart, Serious

Eats, etc.)

• Ingredients (text and number*)

• Instructions

• Publisher Tags*

• Course* (main, dessert, breakfast, etc.)

• Season (Thanksgiving, summer,

birthday, etc.)

• Etc.

Searching ZipList Recipes

• First try – Ferret o Ability to index on any attribute without DB changes

o Zero latency between insert and available in index

o Runtime load and stability

• Next try – Sphinx o Incremental indexing load and time

o Indexing DB load

o Search daemon load and latency

• Current – CloudSearch o SDF allows for synthetic fields (like ferret)

o Fast query results

o Indexing latency issues

ZipList Platform

ZipList Recipe Indexing

WWW Recipes

ZL REcipes

User A Recipe Box

User B Recipe Box

Recipes CloudSearch

SDF Documents

Save Recipe

Save Recipe

Recipe Box Cloudsearch

SDF Documents

Recipes Clipped or Pushed

Recipe Box Indexing Latency

• Desire to unify the UX for Global Recipe Search and Recipe Box Search

o Filtering

o Facet Counts

o Pagination

• Scenario

o User finds a recipe on Epicurious.com and adds it to their box

o They then go immediately to ZipList.com to view their Recipe Box and filter by

Epicurious publisher

o The time it takes for the recipe to appear is LTOT = LSDF + LCSI

o Even 10s is unacceptable

ZipList Platform

Handling Indexing Latency

ZL Recipes

Recipe Box

Recipe Box Cloudsearch

SDF Documents

Save Recipe

Query Module

RB Cache

Search Query

Search Results

Summary and Conclusions

• With the latency issues resolved, CloudSearch works very well for our

searching needs

• Searching, filtering, and faceting provided by CloudSearch fit in well with our

application

• We have a consistent interface / capabilities between Global Recipe Search

and Recipe Box Search

• Positions ZipList well for moving to NoSQL solutions

Zumobi

Pat Binkley, VP Engineering

Zumobi is the leading mobile media company that partners with top media brands to publish premium applications

and provide integrated native advertising experiences on smartphones and connected devices.

The Zumobi Brand Integration (ZBi) native advertising platform offers an SDK to drive a wide array of features and formats enabling seamlessly integrated brand experiences on mobile.

Zumobi’s Long History of Mobile Innovation

CloudSearch Example: THE WEEK App

• Why? Parity with Desktop experience

– Most news apps are not yet integrated with search

capabilities, but desktop is. Mobile is quickly catching up

with desktop functionalities

• Problem? Volume of data on device

– Zumobi apps store content locally for performance and

offline experience

– Too much data to store on phone to provide meaningful

search results

• Solution? AWS CloudSearch

– Store documents in the cloud and query

New Capabilities

• Keyword Searching

• Relevant Results Display

• Article Viewing

• Social sharing

– Facebook & Twitter

• Save

– Save your favorite articles to review later

Architecture

Zumobi

Platform

(Rails App)

CloudSearch DOCUMENT

SERVICE RSS

Optional: >2K items

Document Request

Response

CloudSearch SEARCH

SERVICE Step 1: Search for Item/List Returned

Step 2: Fetch optional > 2K items

App Content SQLite/S3

CDN

** Implementation Time: - 5 days

Impact

• Feature parity with THEWEEK.com website

• Enhanced App Performance

– Increased Usage

• Amplified User Engagement

• More Pageviews

• New Users

• Data Collection

– Find out what your audience is searching for in order to provide more relevant

content

Impact

Zumobi App Network Future Search Integrations…

Learn More – Check Out

• Zumobi.com

• Zumobi.com/AdLab

Viddy

Ken Chung, CTO

Viddy

Viddy is a simple way to capture,

create, and share short, 30-second

social videos with friends.

Viddy – Search Use Cases

• Users want to discover content around them

• Users want to find their friends and celebrities

• Users want to discover content with certain #hashtags and keywords

• Users want to discover more relevant content by language, country, and more

context

Viddy – Unique Problem? No

• Problems not unique to Viddy

• With rise of social, mobile, and geo-aware mobile apps, search is basically

everyone’s challenge

• Why re-invent the wheel?

• Let the platform handle the headaches and let us focus on innovations!

Viddy – B.C. aka “Before CloudSearch”

• Custom managed Lucene + Indexers + Search API Servers

• A full dedicated backend development effort for search product (out of two

backend engineers)

• Lack of tooling for optimizing queries and improving search relevancy by

non-engineers

• Lack of monitoring and reporting around search

… and then 300K users to 40 million users in less than 6 months

Viddy – Hello, CloudSearch!

• Migrated entirely from Lucene to CloudSearch in 3 days

• Simple CloudSearch REST API for Indexing and Querying

• 2–5 hrs/wk of maintenance effort for search product

• Interns improving search result through CloudSearch console

• Better monitoring and report

Viddy – Aftermath

• Operation cost: $5–6K/mo to $1600/mo

• More development resources for innovation

• 5x increase in search usage

• Faster incremental index: avg. delay from 15 mins to 30 secs

Customer Panel

Thank You!

• Learn more at http://aws.amazon.com/cloudsearch

• Please fill out your evaluation forms