O C T O B E R 1 3 - 1 6 , 2 0 1 6 • A U S T I N , T X

Understanding the Solr security framework Anshum Gupta

Senior Software Engineer, IBM Watson

3

01Who am I?

• Anshum Gupta, Apache Lucene/Solr PMC member and committer, IBM Watson Search team.

• Interested in search and related stuff.

• Apache Lucene since 2006 and Solr since 2010.

• Organizations I am or have been a part of:

4

02What is ?

• Apache Lucene is a free open source information retrieval software library

• Originally written in Java by Doug Cutting.

• It is supported by the Apache Software Foundation and is released under the Apache Software License.

5

03What is ?

• Solr (pronounced “solar”, not “solaar”) is an open source enterprise search platform

• Written in Java,

• For a while now, a part of the Apache Lucene project.

• Search on Lucene & Resin (SoLR)

• SolrCloud - Distributed feature set

6

"The only truly secure system is one that is powered off, cast in a block of concrete and sealed in a lead-lined room with

armed guards." —Gene Spaford

7

01

Apache Solr is the most widely-used search solution on the planet.

Solr has tens of thousands of applications in production.

You use everyday.

8,000,000+Total downloads

Solr is both established and growing.

250,000+Monthly downloads

2,500+Open Solr jobs and the largest

community of developers.

8

01SolrCloud Logical Architecture

Shard 1 (leader)

Followers

Shard 2 (leader)

Followers

ZooKeeperZooKeeper instance

Solr Instance

9

01SolrCloud - Physical Architecture

ZooKeeper

Node 1 Node 2

LoadBalancer

Client

Client

Client

Client

Client

Client

Client

Client

Client

Lots

Of

Interaction

Coins by Creative Stall from the Noun Project

10

01Why do we need it?

• Multi-tenant systems

• Access control

• Solr resources

• ZooKeeper

• Authentication

• Authorization

• Until recently: Nothing out of the box. Only locked-in, third party!

11

01Solr security - available options

• SSL support

• ZooKeeper ACLs

• Authentication framework

• Authorization framework

12

01Security Framework Architecture

Incoming RequestAuthenticate andattach metadata Authorize Process Request

Error

Servlet Filter

Authentication PluginAuthorization Plugin

Error

Return Responsewith result

13

01Enabling a plugin

• /security.json file in ZooKeeper

• Contains security config

• Custom plugins: More meta-data can be provided

{

"authentication" : {

     "class": “class.that.implements.authentication",

"other_data" : "..."

 },

 "authorization": {

     "class": “class.that.implements.authorization",

"other_data" : "..."

  }

}

14

01Authentication Framework

Authentication Plugin

Client side componentServer side component

Reconfigurer*Request Reconfigurer*Inter-nodeRequests

Inter-nodeRequests

Inter-nodeRequests

15

01Available Authentication Plugins

• BasicAuth

• Kerberos

16

01Kerberos Plugin

• Prerequisites:

• Enabled via /security.json before Solr instance starts

• Each Solr node must have:

• A Kerberos service principal and key tab file

• Client principal and a corresponding keytab file. Can be the same as the service principal.

• Recommendation:

• Kerberized ZooKeeper

• Start Solr with the correct host-specific parameters.

Kerberized Solr

Kerberized ZooKeeper

KerberosEnabled C

lient

17

01Authorization Plugins

• Rule based authorization plugin

• Apache Ranger plugin (external)

• DIY!

18

01Rule based Authorization Plugin

• Rule based permissions - stored in ZooKeeper

• Editable using API

• Auto-reload on update

• Well known permissions eg: config-edit, collection-edit, security-edit

SolrCloud Cluster

ZooKeeper Ensemble

19

Getting started with Basic Auth & Rule based plugin is easy!

• Upload security.json

• Add users and permissions using API calls!

curl --user solr:SolrRocks http://localhost:8983/solr/admin/authorization -H 'Content-type:application/json' -d '{"set-permission" : {"name":"collection-admin-edit", "role":"admin"}}'

20

01Benefits of the frameworks

• Secure operations in a multi-tenant setup

• Allows for features that couldn’t be added due to lack of security e.g. registering handlers, and uploading configs via API calls.

21

Writing your own plugin

• Authentication

• Extend AuthenticationPlugin

• Optional: extend HttpClientInterceptorPlugin

• Authorization

• Implement AuthorizationPlugin

• Optionally implement ConfigEditablePlugin

22

01The eco-system

HDFS

ZooKeeper

And more…

23

01"Security is not a product, but a process". —Bruce Schneier

• Better inter-node communication standard

• First-class support in SolrJ and bin scripts for basic auth.

• More plugins!

24

01Connect@

http://www.twitter.com/anshumgupta

http://www.linkedin.com/in/anshumgupta/

anshum@apache.org

Love Solr? We are hiring!