Bringing spatial love to your python application

OPENSHIFTWorkshop

PRESENTEDBY

ShekharGulati

Bringing Spatial Love to your Python Application

WHO AM I

• Shekhar Gulati – Works at Red Hat

• Principal OpenShift Developer Evangelist

• Java / Python/ JavaScript / NoSQL / Cloud Guy

• Twitter Handle : shekhargulati

• Github https://github.com/shekhargulati

• Slides http://www.slideshare.net/shekhargulati

AGENDA

➔ Learn a bit about Flask

➔ Learn some MongoDB basics

➔ Build a location aware Job search application

➔ Go live with a Flask MongoDB powered app in ~ 50 mins

Write an application and host it on internet

http://localjobsdemo-shekhargulati.rhcloud.com/

GOAL

CODE DU JOUR

https://github.com/shekhargulati/localjobs-python2.7

● You know Python● You know Git● You like to write code rather than managing

servers.

ASSUMPTIONS

The ToolsFlask , MongoDB , and OpenShift

FLASK

● Flask is a microframework for web application development in Python

● Microframework refers to simplicity and small size of framework

● It is inspired by Sinatra(Ruby framework to create web apps with minimum fuss)

● Built on Werkzeug , Jinja2 , WSGI

● Good sensible defaults

FLASK

● Easy to learn – dead simple

● Simple but extensible

● Excellent documentation spread over 280 pages with lots of examples

● Very active community

● It has minimal footprint

● Does not make choice for you

● Unit testing support

WHY FLASK?

$ mkdir flask-demo

$ cd flask-demo

$ virtualenv venv --python=python2.7

$ . venv/bin/activate

$ pip install flask

$ touch app.py

$ python app.py

FLASK : GETTING STARTED

● Look under templates for templates (default jinja2)

● Look under static for assets (CSS , JS , etc.)

FLASK DEFAULTS

RESTFUL REQUEST DISPATCHING

MONGODB

15

Open Source NoSQL document datastore– JSON style documents

Schema-less– Each document is heterogeneous, and may have completely unique

structure compared to other documents

Fast and horizontally scalable Rich query language Rich documents Easy to get running Geospatial indexing

WHAT IS MONGODB

Database → Database

Table → Collection

Row → JSON Document

Index → Index

Join → Embedding

STEP 4 : MONGODB TERMINOLOGY

17

What is it for? Find all the MongoDB jobs near me – Proximity Queries Find all the MongoDB jobs within Tokyo – Bounded Queries Find all the MongoDB job at this location – Exact Queries

● Supports only two dimensional indexes. You can only have one geospatial index per collection. By default, 2d geospatial indexes assume longitude and

latitude have boundaries of -180 inclusive and 180 non-inclusive (i.e. [-180, 180))

GEOSPATIAL INDEXING BASICS

18

1) Put your coordinates into an array

{ loc : [ 50 , 30 ] } //SUGGESTED OPTION

{ loc : { x : 50 , y : 30 } }

{ loc : { foo : 50 , y : 30 } }

1) { loc : { lon : 40.739037, lat: 73.992964 } }

2) Make a 2d index

db.jobs.ensureIndex( { loc : "2d" } )

3) If you use latitude and longitude as your coordinate system, always store longitude first. MongoDB’s 2d spherical index operators only recognize [ longitude, latitude] ordering.

HOW TO MAKE IT WORK

19

// Find all the jobs with skill as mongodb

db.jobs.find({"skills":"mongodb"})

// Find all the jobs with python as skill and near to given location

db.jobs.find({"lngLat":{$near : [139.69 , 35.68]}, "skills":"python"})

// Find all the python or mongodb jobs near to given location

db.jobs.find({"lngLat":{$near : [139.69 , 35.68]}, "skills":{$in : ["mongodb","python"]}})

SOME QUERIES

Platform as a Service delivers

• Application run-time environment in the cloud

• Configures & manages both the cloud & stack for your application

“The cloud is now useful!”

origin

Public Cloud

Service

On-premise

or Private Cloud

Software

Open Source Project

FLAVORS OF OPENSHIFT

23

OUR STACK

24

https://openshift.redhat.com/app/account/newPromo code is PYCONAPAC

CREATING OPENSHIFT ACCOUNT

● Free! No time limit

● 3 gears (like servers) - each 512 Mb RAM, 1 Gb disk

● Auto-scaling

● Simple pricing

BUT WAIT – THERE's MORE

1) Create Python 2.7 OpenShift Application

2) Add MongoDB database

3) Pull source code from github

4) Import data into MongoDB

5) Push the changes to Application

DEMO

rhc app create localjobs python-2.7

CREATE APPLICATION

rhc cartridge add mongodb-2.2 --app localjobs

ADD CARTRIDGE

git remote add upstream -m master https://github.com/shekhargulati/localjobs-python2.7.git

git pull -s recursive -X theirs upstream master

PULL SOURCECODE

● Step 1 : Application Setup , User Registration , User login– git checkout 005ce14038f489b16249a28be84c035cad99b2b8

● Step 2 : Create New Job– git checkout e96e5d19f28ae8aaa33c1cce283daef98c4ff124

● Step 3 : Search with Near– git checkout 06a388f003fbfec51c6cee1763bae3cf96b58185

● Step 4 : Search with GeoNear– git checkout 653e830dfecf5e7df31016be14e89d5daadb84ee

● Step 5 : OpenShift Application– git checkout master

CODE WALKTHROUGH

$ rhc show-app$ scp jobs-data.json <ssh url>:app-root/data$ rhc ssh$ cd app-root/data$ mongoimport -d localjobs -c jobs --file jobs-data.json -u $OPENSHIFT_MONGODB_DB_USERNAME -p $OPENSHIFT_MONGODB_DB_PASSWORD -h $OPENSHIFT_MONGODB_DB_HOST -port $OPENSHIFT_MONGODB_DB_PORT

IMPORT DATA INTO MONGODB

git push

DEPLOY APP

rhc create-app localjobs python-2.7 mongodb-2.2 --enable-jenkins -s --from-code

https://github.com/shekhargulati/localjobs-python2.7.git

LOT MORE GOODIES

AutoScaling + Jenkins

QUESTIONS?

DONE!