Java clients for elasticsearch

Java Clients for

Florian Hopf@fhopf

elasticsearch

Elasticsearch?

„Elasticsearch is a distributed, JSON-based search and analytics engine, designed for horizontal scalability, maximum reliability, and easy management.“

Elasticsearch?


Elasticsearch?

Installation

# download archivewget https://artifacts.elastic.co/downloads/

elasticsearch/elasticsearch-5.0.0.zip

unzip elasticsearch-5.0.0.zip

# on windows: elasticsearch.batelasticsearch-5.0.0/bin/elasticsearch

Accessing Elaticsearch

curl -XGET "http://localhost:9200"

{ "name" : "LI8ZN-t", "cluster_name" : "elasticsearch", "cluster_uuid" : "UvbMAoJ8TieUqugCGw7Xrw", "version" : { "number" : "5.0.0", "build_hash" : "253032b", "build_date" : "2016-10-26T04:37:51.531Z", "build_snapshot" : false, "lucene_version" : "6.2.0" }, "tagline" : "You Know, for Search"}

Indexing

curl -XPOST "http://localhost:9200/food/dish" -d'{ "food": "Hainanese Chicken Rice", "tags": ["chicken", "rice"], "favorite": { "location": "Tian Tian", "price": 5.00 }}'

Indexing

curl -XPOST "http://localhost:9200/food/dish" -d'{ "food": "Ayam Penyet", "tags": ["chicken", "indonesian"], "spicy": true}'

Search

curl -XGET "http://localhost:9200/food/dish/_search?q=chicken"...{"total":2,"max_score":0.3666863,"hits":[{"_index":"food","_type":"dish","_id":"AVg9cMwARrBlrY9tYBqX","_score":0.3666863,"_source":{ "food": "Hainanese Chicken Rice", "tags": ["chicken", "rice"], "favorite": { "location": "Tian Tian", "price": 5.00 }}},...

Search using Query DSL

curl -XPOST "http://localhost:9200/food/dish/_search" -d'{ "query": { "bool": { "must": { "match": { "_all": "rice" } }, "filter": { "term": { "tags.keyword": "chicken" } } } }}'

Elasticsearch?


Distributed

Distributed

Recap

● Java based search server● HTTP and JSON● Search and Filtering Query-DSL● Faceting, Highlighting, Suggestions, …● Nodes can form a cluster

Transport-Client

Transport-Client

dependencies { compile group: 'org.elasticsearch.client', name: 'transport', version: '5.0.0'}

Transport-Client

TransportAddress address = new InetSocketTransportAddress(

InetAddress.getByName("localhost"), 9300);

Client client = new PreBuiltTransportClient(Settings.EMPTY) addTransportAddress(address);

Search using Query DSL

curl -XPOST "http://localhost:9200/food/dish/_search" -d'{ "query": { "bool": { "must": { "match": { "_all": "rice" } }, "filter": { "term": { "tags.keyword": "chicken" } } } }}'

Search using Search Builders

SearchResponse searchResponse = client.prepareSearch("food")

.setQuery(boolQuery().

must(matchQuery("_all", "rice")). filter(

termQuery("tags.keyword", "chicken"))) .execute().actionGet();

assertEquals(1, searchResponse.getHits().getTotalHits());

SearchHit hit = searchResponse.getHits().getAt(0);String food = hit.getSource().get("food").toString();

Indexing

XContentBuilder builder = jsonBuilder() .startObject() .field("food", "Roti Prata") .array("tags", new String [] {"curry"}) .startObject("favorite") .field("location", "Tiong Bahru") .field("price", 2.00) .endObject() .endObject();

IndexResponse resp = client.prepareIndex("food","dish") .setSource(builder) .execute() .actionGet();

Indexing

Transport-Client

● Connects to an existing cluster● Uses the binary protocol also used for inter

cluster communication

Transport-Client

Sniffing

Sniffing

● State of the cluster can be requested from elasticsearch

● Client side loadbalancing

TransportAddress address = new InetSocketTransportAddress(

InetAddress.getByName("localhost"), 9300);

Settings settings = Settings.builder().put("client.transport.sniff", true).build();

Client client = new PreBuiltTransportClient(settings) addTransportAddress(address);

Transport-Client

● Full API-Support● Efficient communication● Client side loadbalancing

Transport-Client Gotchas

● Compatibility between elasticsearch versions● Elasticsearch dependency

REST-Client

Elasticsearch 5 REST-Client


dependencies { compile group: 'org.elasticsearch.client', name: 'rest', version: '5.0.0'}


+--- org.apache.httpcomponents:httpclient:4.5.2+--- org.apache.httpcomponents:httpcore:4.4.5+--- org.apache.httpcomponents:httpasyncclient:4.1.2+--- org.apache.httpcomponents:httpcore-nio:4.4.5+--- commons-codec:commons-codec:1.10\--- commons-logging:commons-logging:1.1.3


RestClient restClient = RestClient.builder( new HttpHost("localhost", 9200, "http"), new HttpHost("localhost", 9201, "http")).build();


HttpEntity entity = new NStringEntity( "{ \"query\": { \"match_all\": {}}}", ContentType.APPLICATION_JSON); // alternative: performRequestAsync Response response = restClient.performRequest("POST",

"/_search", emptyMap(), entity); String json = toString(response.getEntity()); // ...


● Less dependent on elasticsearch version ● Clean separation network application/cluster● Minimal dependency● Sniffing● Error handling, timeout config, basic auth,

headers, …● No query support (for now)

Jest – Http Client

Jest

dependencies { compile group: 'io.searchbox', name: 'jest', version: '2.0.0'}

Client

JestClientFactory factory = new JestClientFactory();factory.setHttpClientConfig(new HttpClientConfig .Builder("http://localhost:9200") .multiThreaded(true) .build());

JestClient client = factory.getObject();

Searching Jest

String query = jsonStringThatMagicallyAppears;

Search search = new Search.Builder(query) .addIndex("library") .build();

SearchResult result = client.execute(search);assertEquals(Integer.valueOf(1), result.getTotal());

Searching Jest

JsonObject jsonObject = result.getJsonObject();JsonObject hitsObj = jsonObject.getAsJsonObject("hits");JsonArray hits = hitsObj.getAsJsonArray("hits");JsonObject hit = hits.get(0).getAsJsonObject();

// ... more boring code

Searching Jest

public class Dish {

private String food;private List<String> tags;private Favorite favorite;

@JestIdprivate String id;

// ... getters and setters}

Suche mit Jest

Dish dish = result.getFirstHit(Dish.class).source;

assertEquals("Roti Prata", dish.getFood());

Jest

● Alternative HTTP implementation● Queries as Strings or via Elasticsearch-Builder● Indexing and searching Java beans● Node Discovery

Spring Data Elasticsearch

Spring Data

● Abstractions for different data stores● Speciality of each store available● Dynamic Repository implementations● Popular modules

● Spring Data JPA● Spring Data MongoDB

Dependency

dependencies {compile group: 'org.springframework.data',

name: 'spring-data-elasticsearch', version: '2.0.4.RELEASE'}

Entity

@Document(indexName = "spring_dish")public class Dish {

@Idprivate String id;private String food;private List<String> tags;private Favorite favorite;

// more code

}

Repository

public interface DishRepository extends ElasticsearchCrudRepository<Dish, String> {

}

Configuration

<elasticsearch:transport-client id="client" />

<bean name="elasticsearchTemplate" class="o.s.d.elasticsearch.core.ElasticsearchTemplate"> <constructor-arg name="client" ref="client"/></bean>

<elasticsearch:repositories base-package="de.fhopf.elasticsearch.springdata" />

Indexing

Dish mie = new Dish();mie.setId("hokkien-prawn-mie");mie.setFood("Hokkien Prawn Mie");mie.setTags(Arrays.asList("noodles", "prawn"));

repository.save(Arrays.asList(hokkienPrawnMie));

Searching

Iterable<Dish> dishes = repository.findAll();

Dish dish = repository.findOne("hokkien-prawn-mie");

Repository

public interface DishRepository extends ElasticsearchCrudRepository<Dish, String> {

List<Dish> findByFood(String food);

List<Dish> findByTagsAndFavoriteLocation(String tag, String location);

List<Dish> findByFavoritePriceLessThan(Double price);

@Query("{\"query\": {\"match_all\": {}}}")List<Dish> customFindAll();

}

Recap

● High level abstraction● Entity beans● dynamic repository implementation● HTTP support in the making● Slower feature development

Recap

● Transport-Client● Full API support● Elasticsearch dependency

● REST-Client● Uses HTTP● currently lacking search API

Recap

● Jest● Http client ● Support for Java beans

● Spring-Data-Elasticsearch● High level abstraction● Dynamic repositories

Links

● http://elastic.co● https://github.com/searchbox-io/Jest● https://github.com/spring-projects/spring-

data-elasticsearch● https://github.com/fhopf/singajug-examples● http://blog.florian-hopf.de

Technology

Java clients for elasticsearch