ElasticSearch Introduction

ElasticSearch Introduction

Minh Hoang TO

Semptember 18, 2014

Agenda

Introduction

Server-side

Client-side

Demo

IntroductionServer-sideClient-side

Demo

OverviewGetting Started

ElasticSearch?

Open source searchplatform:

Written in Java

Bases on Lucene &Netty

Released asstandalone Javaapplication

Latest StableRelease: 1.3.1

3 / 19

IntroductionServer-sideClient-side

Demo

OverviewGetting Started

Features

Full Text Search

Scroll Search

Facet Search

Scripting

Near-Real-Time (NRT) Index

Dynamic Configuration

Pluggable Architecture

Distributed Design in Mind

Built-in Support for Installation on AWS, Azure

Multiple community-backed Management Tools

4 / 19

IntroductionServer-sideClient-side

Demo

OverviewGetting Started

Install & Start

Extract elastic search

u n z i p e l a s t i c s e a r c h −1.3.1. t a r . gz

t a r −x v f e l a s t i c s e a r c h −1.3.1. t a r

Run Elastic Search:

${ES HOME}/ b i n / e l a s t i c s e a r c h

5 / 19

IntroductionServer-sideClient-side

Demo

OverviewGetting Started

Data Indexing

Data indexing request:

http://localhost:9200/{index}/{type}/{id} POST

Indexing article in demoindexindex with article type and idequals to 123456 :

curl http://localhost:9200/demoindex/article/123456-d ’${data in JSON format}’

6 / 19

IntroductionServer-sideClient-side

Demo

OverviewGetting Started

Full Text Search

Search request:http://localhost:9200/{index}/{type}/ search?q=GEThttp://localhost:9200/{index}/{type}/ searchPOST

Examples:

c u r l −XGET h t t p : / / l o c a l h o s t :9200/demoindex / a r t i c l e / s e a r c h ?q=c o n t e n t : Ebola

c u r l −XPOST h t t p : / / l o c a l h o s t :9200/demoindex / a r t i c l e / s e a r c h −d

’{” q u e r y ” : {

” term ” : {” c o n t e n t ” : ” Ebola ”}}

} ’

7 / 19

IntroductionServer-sideClient-side

Demo

OverviewGetting Started

Scroll Search

’Iterate over items matching search query to find the first item matching certaincriteria might be expensive’

Scroll Search:

1 Send search request to get scroll id

h t t p : / / l o c a l h o s t :9200/ s e a r c h / s c r o l l POST

2 Fetch items progressively with scroll id

h t t p : / / l o c a l h o s t :9200/ s e a r c h / s c r o l l /{ s c r o l l i d} GET

8 / 19

IntroductionServer-sideClient-side

Demo

OverviewGetting Started

Facet Search

facets parameter insearch query:

{. . . . .” f a c e t s ” : {

” t a g s ” : { ” terms ” : {” f i e l d ” : ” t a g s ”} }

}. . . .

}

9 / 19

IntroductionServer-sideClient-side

Demo

OverviewGetting Started

Administration Tool

Admin APIexposed butno built-inAdministra-tionGUI

Tools fromElasticSearchCommunity:Marvel,Kibana,...

10 / 19

IntroductionServer-sideClient-side

Demo

ArchitectureData ModelConfigurationPlugins

ElasticSearch Instance

11 / 19

IntroductionServer-sideClient-side

Demo

ArchitectureData ModelConfigurationPlugins

ElasticSearch Cluster

Node Elastic Search Instance

Cluster One or many nodes sharing common cluster name

Index Storage of ElasticSearch data, index might span over multiple nodes

Primary Shard Node stores index data into Primary Shard

Replica Shard Copy of Primary Shard

12 / 19

IntroductionServer-sideClient-side

Demo

ArchitectureData ModelConfigurationPlugins

ElasticSearch Document

Data in JSONformat

One-to-one mappingto Lucene Document

Stored in onePrimary Shard andits Replicas

13 / 19

IntroductionServer-sideClient-side

Demo

ArchitectureData ModelConfigurationPlugins

Configuration

Declarative settingin:${ES HOME}/config/elasticsearch.yml

Setting could besetup on-the-fly viaRESTful API:http://localhost:9200/ settings

14 / 19

IntroductionServer-sideClient-side

Demo

ArchitectureData ModelConfigurationPlugins

ElasticSearch Plugin

Building plugin is the canonical way to extend or to customise the core services

15 / 19

IntroductionServer-sideClient-side

Demo

Client APIRouting

ElasticSearch Client

Applications talk to ElasticSearch server via Native or HTTP-based clients:

Native: establish connection to node(s) in cluster:Fully asynchronousOnly available in Java

HTTP-based: make calls to REST API:Not fully asynchronousAvailable in multiple Programming Languages

16 / 19

IntroductionServer-sideClient-side

Demo

Client APIRouting

Shard Routing

App: I need the list of articles containing’Maria Ozawa’

ES: No hint on shard ID? I need tobroadcast to all shards, then aggregateresponses and return the list to you. Bepatient

App: Same request, but i rememberassigning ’routing=jap adult’ while askingyou to index those articles

ES: ’routing=jap adult’ ? Those articlesare in Shard 2, i am returning you the listsoon

17 / 19

IntroductionServer-sideClient-side

Demo

DemoQ&A

Demo

Local ElasticSearch cluster with:3 nodes.5 shards per node.Marvel installed.demo-plugin installed

ElasticSearch integration into Service+

18 / 19

IntroductionServer-sideClient-side

Demo

DemoQ&A

19 / 19