Embed Size (px)
Citation preview
Use Cases of Elasticsearch AND
Considerations in Multi-tenancy
2016-09-14
eurie Inc. Takahiro Ikeuchi
AgendaAbout us and our product
Our use cases of Elasticsearch
Combination with RDB
Considerations in Multi-tenancy
How do we search private documents ?
Self-Managed vs Amazon ES
2
Conclusion
3
ConclusionIn the case of Multi-tenancy model, processingAuthentication and Authorization should behandled by your backend applications.
If a strict synchronous execution is not needed,an asyncronous execution using AWS Labmdais a good option.
You may want to implement wrapper logics tostandardize interfaces.
I would recommend that you adopt "self-managed" until Amazon ES supports VPC.
4
About us and our product
5
AuthorTakahiro Ikeuchi @iktakahiro
Company / Community
eurie Inc. Founder & CEO
SQUEEZE Inc. Tech Adviser
PyData.Tokyo Organizer
Specialties (or just a dabbler :-D
Go lang, Python, React.js, TypeScript
Cloud Infrastructure, UI Design etc...
6
7
A Customer Support Tool for Innovative Teams
8
ArchitecturesRESTful API
Go lang (framework: echo)
Single Page Application
React.js + Typescript
Elasticsearch
Cloud Native
AWS Aurora, Lambda, CloudFront, WAF...
Microservice Architecture
9
Our use cases of Elasticsearch
11
Searching Message
12
13
Why Elasticsearch ?
14
Why Elasticsearch?Full-Text Search
Morphological Analysis
High-Performance / Scalable
Powerful weighted-search, sort, �lter
Similar documents Recommendations
Pretty REST API, Client Library etc...
15
Architecture
16
Architecture
17
GET docs from Elasticsearch
18
The �ow of searching messages
1. A user enters a keyword in the client app. A client sends a GET Request to a REST API
on the backend app.
2. The REST API receives the GET Request. Authentication / Authorization
3. The backend app sends a GET Request toElasticsearch Receiving search-results.
4. The backend app builds / returns a response.
5. The client app receives the response from theREST API. Rendering User-Interfaces.
19
GET docs from Elasticsearch
20
Web API Design
21
Web API DesignGET /meesage/search?q=word&page=1&per_page=10
Response Body
{ "took": 36, "_scroll_id": "", "hits": { "total": 1,
The above JSON is a responce of Elasticsearchitself. It was not changed.
22
Web API DesignResponse Header
Link:
Createing a Response Header: Link based on total (and the others) value in a search-result.
RFC 5988 - Web Linkinghttps://tools.ietf.org/html/rfc5988
</messages/search?q=gmail&page=1&per_page=10>; rel="first",</messages/search?q=gmail&page=2&per_page=10>; rel="next", </messages/search?q=gmail&page=2&per_page=10>; rel="last"
23
Combination with RDB
24
Considerations in combination with RDB
Would you really need RDB?
Would you need a strict synchronous?
Is UPDATE/DELETE executed frequently?
In the case of eurie Desk :
At the �rst, all documents are inserted intoRDB (Amazon Aurora).
Just after inserting, documents are also put toElasticsearch asynchronously.
Elasticsearch is used on write-only situations.25
POST docs to RDB
26
PUT docs to Elasticsearch (Async)
27
Considerations in Multi-tenancy
28
The case of Multi-tenancy model
â Issues
A user or role based access controll system fordocuments is needed.
Documents should not be able to referred byother tenants.
How do we search private documents?
29
The case of Multi-tenancy model
Solutions
You should build Elasticsearch in a internal(private) network.
A processing Authentication and Authorizationshould be handled by your backend apps.
Of couse use Index for each tenant.
30
RDB and ES instances boot in the Private Network
31
Amazon ES Pros/Cons
Amazon ES = Amazon Elasticsearch Service
https://aws.amazon.com/elasticsearch-service/
Pros
Very easy, even High-Available structures.
By default, Amazon ES has Kuromoji-plugin. So you can use Elasticsearch immediately.
32
Amazon ES Pros/Cons
Cons
It's only booted in a global netowrk area.
IP Restrictions OK, but we can't use "Secuirty Groups".
Or there is an option to develop a systemusing "IAM"...
Is the newest version of Elasticsearch releasedimmediately?
We don't have permission to install plug-infreely. 33
Self-Managed vs Amazon ES
If you want use Elasticsearch to search none-private documents, It might be that adoptingAmazon ES is a wise way.
On the other hand, in the case of privatedocuments, consider to adopt Self-Managed.
It's layer is same as RDB.
I hope Amazon ES supports soon.
34
ConclusionIn the case of Multi-tenancy model, processingAuthentication and Authorization should behandled by your backend applications.
If a strict synchronous execution is not needed,an asyncronous execution using AWS Labmdais a good option.
You may want to implement wrapper logics tostandardize interfaces.
I would recommend that you adopt "self-managed" until Amazon ES supports VPC.
35
AppendixElasticsearch o�cial site
Elasticsearch client library for Go lang:olivere/elastic
eurie.io
36
