Search & Analyze Data in Real Time

ELASTICSEARCH

agenda1- Full Text Search

2- Elasticsearch Overview

3- Basic Concepts

4- Elasticsearch Users

5- Real World Use Cases

6- Plugins

full text search

full text searchLike

Select * from Article nolock where ArticleTitleManset Like '%kitap%' or ArticleText Like '%kitap%'

It controls all rows

full text searchWhy Full Text Search?

Making Text Searchable

full text search

full text search

full text search

INVERTED INDEX

full text searchFull Text Search Databases

full text searchFull Text Search Engines

full text searchFull Text Databases versus Full Text Search Engines

CAP THEOREM

Distributed systems are not guarantee to provide these three terms :

full text searchFull Text Databases versus Full Text Engines

what ıs elastıcsearch?

what ıs elastıcsearch?Real Time Search & Analytics Engine

what ıs elastıcsearch?Open Source - Apache Licence

https://github.com/elastic/elasticsearch

what ıs elastıcsearch?Written In Java - Support Cross Platform

Elasticsearch Requires Java!!!

what ıs elastıcsearch?Built On Top Of Lucene ● Very Popular, Full Text Search Library

● Project Based

what ıs elastıcsearch?Document Oriented(JSON) - Restful Api

what ıs elastıcsearch?Distributed

what ıs elastıcsearch?History

Elasticsearch created by Shay Bannon. Before Elasticsearch, Shay Bannon ‘s first product is Compass.Compass created on 2004 to “scalable search solution” and also built on Lucene.

In February 2010, Compass development was stopped by Shay Bannon and Elasticsearch released.

basıc concepts

BASIC CONCEPTSELASTICSEARCH STRUCTURE

BASIC CONCEPTSElasticsearch Flow

At first, Data should index to Elasticsearch

Then you can search!

BASIC CONCEPTSAnalyzers

BASIC CONCEPTSAnalyzers

Character Filter

+

Tokenizer

+ =

Token Filter

+

Token Filter

ANALYZER

BASIC CONCEPTSMapping Data

Unlike Solr and Sphinx, Elasticsearch is scheme-free.

Mapping can change on the fly.

BASIC CONCEPTSTypes

Others

IPV4

Attachment

BASIC CONCEPTSRelevance

By Default, results are returned in descending order of Relevance.

Term Frequency

Inverse Document Frequency

Field - Length Norm

BASIC CONCEPTSScoring & Boosting

For custom scoring, increase boost value

elastıcsearch users

who uses?

real world use cases

use casesALL DATA TYPES SEARCHING

use cases

https://qbox.io/blog/boston-elasticsearch-meetup-scoring-images-by-color

IMAGE SEARCHING

use casesGEOLOCATION SEARCHING & DISTANCE CALCULATION

https://github.com/varunpant/AroundMe

use casesCENTRALIZED LOGGING

Loggly built in Elasticsearch and Kibana

use casesDATA ANALYZING

use casesALERT SYSTEM

plugıns & products

PLUGINSHEAD

Web Front End Browsing to Elasticsearch Data and Cluster Management

PLUGINSSENSE

Web Querying Plugin

PLUGINSKOPF

Web Administration Tool for Elasticsearch

ProductsLOGSTASH

● Logging● Event sourcing● Archiving● Alert Data● Monitoring Data

ProductsKIBANA

Browser based search and analytics dashboard for Elasticsearch.

Open Source data visualization platform, powerful graphics.

ProductsLOGSTASH & KIBANA

ProductsWATCHER

Alerting For Elasticsearch Data & Cluster

● Command● Email● JIRA● OpsGenie● SNS● Hipchat● Slack● Debug

ProductsSHIELD

Protect your data with Shield

Role Based Security, Encyrption,IP Filtering

Mustafa Dağdelen

happy searchıng!