Upload
tomas-jansson
View
19.897
Download
0
Embed Size (px)
DESCRIPTION
Slides for my presentation about how to get started with Elasticsearch and .NET using Nest.
Citation preview
GETTING STARTED WITH ELASTICSEARCH ON WINDOWS
AND .NET WITH NEST
A short introduction
Oslo/NNUG Meetup
Tomas Jansson
29/01/2014
THIS IS ME
Tomas Jansson
Manager & Group Lead .NETBEKK Oslo
@[email protected]/mastojblog.tomasjansson.com
AUDIENCE
N00b
N00b Expert
Expert
BACKGROUND
This is the data and we need this new application
THE MASTERPLAN
WHAT I WANT TO SHOW YOU IS...
Elasticsearch is awesome
Indexing using NEST
Querying using NEST
... not about advanced elasticsearch hosting
INSTALLATION
Great news, install as a service added in 0.90.5
Powershell to the rescue
NEST
Abstractionover
Elasticsearch
There is an low level abstraction as well called RawElasticClient
Abstractionover
Elasticsearch
NEST
Fluent & Strongly
typed
Functional C#
FUNC DEMO
C:\Dev\git> scriptcs
scriptcs (ctrl-c or blank to exit)
> Func<int, int, int> add = (x, y) => x + y;
> add(1, 3)
4
Func executable
SIMPLE EXPRESSION DEMO
> using System.Linq.Expressions;
> Expression<Func<int, int, int>> addExpr = (x, y) => x + y;
> addExpr(1, 3)
(1,1): error CS1955: Non-invocable member 'addExpr' cannot be used like a method.
> var binExpr = addExpr.Body as BinaryExpression;
> Console.WriteLine(binExpr);
(x + y)
> var add2 = addExpr.Compile();
> add2(3, 1);
4
Expression ”function description”
MORE COMPLEX EXPRESSION DEMO
> public class SomeClass { public string MyString { get; set; } }
> Expression<Func<SomeClass, object>> propExpr = y => y.MyString + y.MyString;
> var compExpr = propExpr.Compile();
> var obj = new SomeClass { MyString = "Hello world" };
> compExpr(obj)
Hello worldHello world
> var body = propExpr.Body as BinaryExpression;
> Console.WriteLine(body);
(y.MyString + y.MyString)
> var left = body.Left as MemberExpression;
> Console.WriteLine(left.Member.Name);
MyString
MORE COMPLEX EXPRESSION DEMO
> public class SomeClass { public string MyString { get; set; } }
> Expression<Func<SomeClass, object>> propExpr = y => y.MyString + y.MyString;
> var compExpr = propExpr.Compile();
> var obj = new SomeClass { MyString = "Hello world" };
> compExpr(obj)
Hello worldHello world
> var body = propExpr.Body as BinaryExpression;
> Console.WriteLine(body);
(y.MyString + y.MyString)
> var left = body.Left as MemberExpression;
> Console.WriteLine(left.Member.Name);
MyString
Enables us to translate from one domain to another in an ”easy” manner
Show me the code!
ELASTICSEARCH CONNECTION
public class ElasticClientWrapper : ElasticClient{
private static string _connectionString = Settings.ElasticSearchServer;
private static ConnectionSettings _settings =new ConnectionSettings(new Uri(_connectionString)) //http://demoserver:9200
.SetDefaultIndex(Settings.Alias) //"customer_product_mapping"
.UsePrettyResponses();
public ElasticClientWrapper(): base(_settings)
{}
}
//usagevar client = new ElasticClientWrapper();
MAPPING
public class Product{
public double UnitPrice { get; set; }public int TotalQuantity { get; set; }[ElasticProperty(Index = FieldIndexOption.not_analyzed)]public string ProductName { get; set; }[ElasticProperty(Index = FieldIndexOption.not_analyzed)]public string CategoryName { get; set; }
}
public class Customer{
public string CustomerID { get; set; }public string CompanyName { get; set; }public string Address { get; set; }public string City { get; set; }public string Country { get; set; }[ElasticProperty(Type = FieldType.nested)]public Product[] Products { get; set; }
}
MAPPING & INDEXING
_client = new ElasticClientWrapper();_client.CreateIndex("indexName", s =>
s.AddMapping<Customer>(m => m.MapFromAttributes()));
var customers = _customerRepo.GetCustomers();_client.IndexMany(customers, "indexName");
Mapping created from attributes
Indexing will use the mapping for the specified index
There is async versions of the methods
ALIAS
_client = new ElasticClientWrapper();_client.Alias("indexName", "aliasName");
Index_01
Alias
SWAPPING
_client = new ElasticClientWrapper();_client.Swap("aliasName", new [] { "Index_01" }, new [] { "Index_02" } );
Index_01 Index_02
Alias Alias
1. Create new index
2. Swap
3. Delete old index
MY QUERY OBJECT (WILL BE USED IN THE EXAMPLES)
public class SearchModel{
private int? _numberToTake;public string Query { get; set; }public Dictionary<string, IEnumerable<string>> Filter { get; set; }
public int? NumberToTake{
get { return _numberToTake.HasValue ? _numberToTake.Value : 25; }set { _numberToTake = value; }
}}
QUERYING
{"query": {
"query_string": {"query": "tomas"
}}
}
_client.Search<Customer>(sd => sd.QueryString(Input.Query));
Elasticsearch NEST
FUZZY
{"query": {
"fuzzy": {"_all": {
"min_similarity": 0.6,"prefix_length": 1,"value": "tomas"
}}
}}
_client.Search<Customer>(sd => sd.Query(q => q
.Fuzzy(fd => fd
.OnField("_all")
.MinSimilarity(0.6)
.PrefixLength(1)
.Value(Input.Query))));
Elasticsearch NEST
Will enable us to search for both «Thomas» and «Tomas» when writing «Tomas»
FUZZY IMPROVED (USING BOOL QUERY) - ELASTICSEARCH
{"query": {
"bool": {"should": [{
"match": {"_all": {
"query": "tomas"}
}},{
"fuzzy": {"_all": {
"boost": 0.1,"min_similarity": 0.6,"prefix_length": 1,"value": "tomas"
}}
}]}
}}
FUZZY IMPROVED (USING BOOL QUERY) - NEST
_client.Search<Customer>(sd => sd.Query(q => q
.Bool(b => b.Should(new Func<QueryDescriptor<Customer>, BaseQuery>[]
{_ => _.Match(m => m
.OnField("_all")
.QueryString(Input.Query)),_ => _.Fuzzy(fd => fd
.OnField("_all")
.MinSimilarity(0.6)
.PrefixLength(1)
.Value(Input.Query)
.Boost(0.1))}))));
HIGHLIGHT RESULT - ELASTICSEARCH
{"query": {
// see previous example},"highlight": {
"pre_tags": ["<span class='highlight'>"
],"post_tags": [
"</span>"],"fields": {
"companyName": {"fragment_size": 100,"number_of_fragments": 1
}}
}}
HIGHLIGHT RESULT - NEST
_client.Search<Customer>(sd => sd.Query( /* See previous example */ ).Highlight(h => h
.PreTags("<span class='highlight'>")
.PostTags("</span>")
.OnFields(new Action<HighlightFieldDescriptor<Customer>>[]{
_ => _.OnField(c => c.CompanyName).NumberOfFragments(1).FragmentSize(100)
})));
FACETS - ELASTICSEARCH
{"query": { /* See previous example */ },"highlight": { /* See previous example */ },"facets": {
"products.productName": {"nested": "products","terms": { "field": "products.productName", "size": 1000 }
},"products.categoryName": {
"nested": "products","terms": { "field": "products.categoryName", "size": 1000 }
},"country": {
"terms": { "field": "country", "size": 1000 }}
}}
FACETS - NEST
_client.Search<Customer>(sd => sd.Query( /* See previous example */ ).Highlight( /* See previous example */ ).FacetTerm(f => f
.Nested(c => c.Products)
.OnField(c => c.Products[0].ProductName)
.Size(1000)).FacetTerm(f => f
.Nested(c => c.Products)
.OnField(c => c.Products[0].CategoryName)
.Size(1000)).FacetTerm(f => f
.OnField(c => c.Country)
.Size(1000)));
http://go-gaga-over-testing.blogspot.no/2011/09/solution-to-warning-in-quality-center.html
FILTERS - ELASTICSEARCH
{"query": {
"filtered": {"query": { /* See previous example */ },"filter": {
"bool": {"must": [{
"terms": { "country": ["usa"] }},{
"nested": {"query": { "terms": { "products.categoryName": ["Condiments", "Seafood"] } },"path": "products"
}},{
"nested": {"query": { "terms": { "products.productName": ["Chai"] } },"path": "products"
}}
]}
}}
},"facets": { /* See previous example */},"highlight": { /* See previous example */ }
}
FILTERS – NEST – PART 1, THE CUSTOMERS FILTER
private static BaseFilter AddCustomerFilter(IEnumerable<string> items, Expression<Func<Customer, object>> propExpr)
{return Filter<Customer>.Terms(propExpr, items.ToArray());
}
FILTERS – NEST – PART 1, THE PRODUCTS FILTER
private static BaseFilter AddProductsFilter(IEnumerable<string> items,Expression<Func<Customer, object>> propExpr)
{return Filter<Customer>.Nested(sel => sel
.Path(c => c.Products)
.Query(q => q.Terms(propExpr, items.ToArray())));}
FILTERS – NEST – PART 1, THE MAGIC DICTIONARY
public Dictionary<string, Func<IEnumerable<string>, BaseFilter>> FilterDesc =new Dictionary<string, Func<IEnumerable<string>, BaseFilter>>(){
{"products.productName", ps => AddProductsFilter(ps, c => c.Products[0].ProductName)},
{"products.categoryName", cs => AddProductsFilter(cs, c => c.Products[0].CategoryName)},
{"country", cs => AddCustomerFilter(cs, c => c.Country)}};
FILTERS – NEST – PART 1, ALL THE HELPERS
private static BaseFilter AddCustomerFilter(IEnumerable<string> items, Expression<Func<Customer, object>> propExpr)
{return Filter<Customer>.Terms(propExpr, items.ToArray());
}
private static BaseFilter AddProductsFilter(IEnumerable<string> items,Expression<Func<Customer, object>> propExpr)
{return Filter<Customer>.Nested(sel => sel
.Path(c => c.Products)
.Query(q => q.Terms(propExpr, items.ToArray())));}
public Dictionary<string, Func<IEnumerable<string>, BaseFilter>> FilterDesc =new Dictionary<string, Func<IEnumerable<string>, BaseFilter>>(){
{"products.productName", ps => AddProductsFilter(ps, c => c.Products[0].ProductName)},
{"products.categoryName", cs => AddProductsFilter(cs, c => c.Products[0].CategoryName)},
{"country", cs => AddCustomerFilter(cs, c => c.Country)}};
FILTERS – NEST – PART 2, THE QUERY
_client.Search<Customer>(sd => sd.Query(q => q
.Filtered(fq =>{
fq.Query(qs =>{
if (!string.IsNullOrEmpty(Input.Query)){
qs.Bool( /* See previous example */ ));}else{
qs.MatchAll();}return qs;
});if (Input.Filter.Count > 0){
var filters =Input.Filter.Select(_ => FilterDesc[_.Key](_.Value)).ToArray();
fq.Filter(fs => fs.Bool(bf => bf.Must(filters)));}
})).Highlight( /* See previous example */ ).FacetTerm( /* See previous example */ ).FacetTerm( /* See previous example */ ).FacetTerm( /* See previous example */ );
Easy installation
Awesome search engine
Strongly typed client
Fluent
Abstraction over Elasticsearch
Elasticsearch NEST
SUMMARY
Demo code: https://github.com/mastoj/NestDemo
Nest documentation: http://nest.azurewebsites.net/
Nest source code: https://github.com/Mpdreamz/NEST
Slideshare: http://www.slideshare.net/mastoj/getting-started-with-elasticsearch-and-net
Sense (great tool to query elastic search in the browser): https://github.com/bleskes/sense
RESOURCES
Questions?
Thank you!
@TomasJansson