node.js workshop- node.js databases

node.js databases

Thomas Dunajskigfnork webentwicklungsunternehmen

Redis, NoSQL, MongoDB, drivers

2

Redis

3

What is Redis?NoSQL key-value data store"memcached with built-in persistence" open source

4

Why should I use Redis?fast (saves data in RAM if possible)keeps a copy on Disc scales goodrarely used values are swapped out to disk if RAM is lowcan save complex javascript objectseasy to usegreat for Apps with small reqirements

5

What is SQL/NoSQL? (very simplified)SQL databases store data in tables all data needs to match the format of the tabletend to be more stabelare scaled by using better hardware(vertical scaling)NoSQL does not use tablesitems in the database do not need to have the same formatare scaled by using more hardware(horizontal scaling)depending on what you want to do one will be better than the other

6

NoSQL database typesKey-value storesitems are keys(names) with one valueDocument databases each key has a complex data structure instead of a valueGraph storesstore information about networks (i.e . social connection)Wide-column storesstore columns of data together instead of rows

7

How to get Redis running?download it on http://redis.io/downloadcompile itrun src/redis-servernpm install redis

http://redis.io/download

8

How to use Redis?var redis = require("redis") , client = redis.createClient();

client.set("key", "Your Data", function (err, reply) { //do something }); client.get("key", function (err, reply) { console.log(reply.toString()); });=> Your Data

9



10


Import the redis driver

make an Instance of

a connection

11



12

How to use Redis?

client.set("key", "Your Data", function (err, reply) { //do something });

Insert Command Value CallbackKey

13



14

How to use Redis?

client.get("key", function (err, reply) { console.log(reply.toString()); });

Load Command Key Callback

15

Questions?var redis = require("redis") , client = redis.createClient();


16

MongoDB

17

What is MongoDBNoSQL document-orientated-database cross platform open source

18

Document databasesdocument databases pair keys with complex data structuresthis complex data structures are called documentsdocuments can contain key-value pairskey-array pairsnested documents

19

Why should i use MongoDBfast (saves data in RAM if possible)scales great(sharding)failure resistent(replica sets)easy to usehigh development speedunderstands JSONsuports geo-querys

20

What is a Replica Set?multiple machines have the same dataif one crashes the data is still accessiblesuch a group of machines is called Replica Set

21

Replica Sets

Node 1

Primary

Node 1

Secondary

Node 1

Secondary

22

Replica Sets

Node 1

Node 1

Secondary

Node 1

SecondaryElection

23

Replica Sets

Node 1

Node 1Node 1

Secondary Primary

24

Replica Sets

Node 1

Node 1

Secondary

Node 1

Secondary Primary

25

What is Shardingspliting data on multiple machinesdata packages are called chunkscunks that exeed 64MB are split in twosingle machines are called shardsthe shard key is the value that decides to witch chunk a entry belongsWARNING: the shard key can only be set once

26

Sharding

a

c

d

Shard1

b e

Shard2

27

Sharding

a

c1

d

Shard1

b e

Shard2

c2

28

Sharding

a

c1

d

Shard1

b e

Shard2

c2

29

Real Deployment

mongod

mongod

mongod

Replica Set

mongod

mongod

mongod

Replica Set

mongod

mongod

mongod

Replica Set

mongod

mongod

mongod

Config Servers

mongos

Client

30

How to get MongoDB running(Ubuntu)?sudo apt-get install mongodb sudo service mongodb start

31

How to get MongoDB running(Windows)?go to http://www.mongodb.org/downloadsdownload the setup install itrun mongod.exe

http://www.mongodb.org/downloads

32

What is ORM?Object-Relational Mapper is a piece of software that transforms objects to relations that can be saved in a database and vice versa

33

What Driver should you use?depends on your needs...Mongoose – Object Relational Mapper MongoskinNode MongoDB Native Driver

34

MongooseProsschema-basedbuilt-in type castingbuild-in validation and other usefull features

Consslow

35

MongoskinProsfastlightweight

Conslightweight

36

Node MongoDB Native DriverProsfastest DriverConsLeast comfortable

I only use Node MongoDB Native Driver when I need maximum performance Mongoskin is fast enough for almost anything

37

How to use Mongoskin?//npm install mongoskin var db = require('mongoskin').db( 'localhost:27017/Workshop');

db.collection('participants').insert({name: "Noob", skills: []}, function(err, result) { if (err) throw err; if (result) console.log('Added!');});=>Added!

db.collection('participants').insert({name: "Geek", skills: ['Node.js' , 'MongoDB']}, function(err, result) { if (err) throw err; if (result) console.log('Added!');});=> Added!

38




39

How to use Mongoskin?

var db = require('mongoskin').db('localhost:27017/Workshop');

assigning Connection to

dbimport Connectiong

to db

Host Adress Port

Database Name

40




41


db.collection('participants').insert({name: "Noob", skills: []}, function(err, result) { if (err) throw err; if (result) console.log('Added!');});

Get a Collection

Collection Name

Insert Command

A Java Script Object

Callback

42

Questions?//npm install mongoskin var db = require('mongoskin').db( 'localhost:27017/Workshop');



43


db.collection('participants').find().toArray(function(err, result) { if (err) throw err; console.log(result); }); =>[ { _id: 5375ea17898372a81425297a, name: 'Noob', skills: [] }, { _id: 5375ea17898372a81425297b, name: 'Geek', skills: [ 'Node.js', 'MongoDB' ] } ]

44


db.collection('participants').find().toArray(function(err, result) { if (err) throw err; console.log(result); }); =>[ { _id: 5375ea17898372a81425297a, name: 'Noob', skills: [] }, { _id: 5375ea17898372a81425297b, name: 'Geek', skills: [ 'Node.js', 'MongoDB' ] } ]

45


db.collection('participants').find().toArray(function(err, result) { if (err) throw err; console.log(result); });

Returns a Collection

Search Command

Converts a Cursor to an

Array

46


db.collection('participants').findOne({name:'Geek'}, function(err, result) { console.log('Geeks skills:'); console.log(result.skills);});=>Geeks skills: [ 'Node.js', 'MongoDB' ]

Returns a single Object

can be used like any other

Object

47

How to use Mongoose?var mongoose = require('mongoose').db( 'localhost:27017/Workshop'), db = mongoose.connection;

db.on('error', console.error.bind(console, 'connection error:'));db.once('open', function callback () { console.log('Connected!')});

var participantSchema = mongoose.Schema({ name: String, skills:[String]});var Participant = mongoose.model('Participant', participantSchema);

48




49

How to use Mongoose?

var mongoose = require('mongoose').db( localhost:27017/Workshop'),

import Connectiong to db

Host Adress Port

Database Name

assigning Connection to Variable db

50




assigning Connection to Variable db

51




52


db.on('error', console.error.bind(console, 'connection error:'));

Reactiong to Event Event Name Handling

ErrorWhere the

Ountput gets piped to

String attached to

Error Message

53




54


db.once('open', function callback () { console.log('Connected!')});

Event that will only be done once

Event Name

55




56


var participantSchema = mongoose.Schema({ name: String, skills:[String]});

Schema Name

Mongoose Object

Function for defining Schemas

Variables

57




58


var Participant = mongoose.model('Participant', participantSchema);

Model NameFunction for

creating Models

Model Name Schema Name

59

Questions?var mongoose = require('mongoose').db( 'localhost:27017/Workshop'), db = mongoose.connection;



60

How to use Mongoose?var noob= new Participant({ name: 'Noob', skills:[]});noob.save(function( err, result) { if( err) return console.error(err); if (result) console.log('Added!');});=> Added!var geek= new Participant({ name: 'Geek', skills:['Node.js' , 'MongoDB']});geek.save(function (err, result) { if (err) return console.error(err); if (result) console.log('Added!');});=> Added!

61


62


var noob= new Participant({ name: 'Noob', skills:[]});

Creating new Instance of

Model

JS Object matching Schema

63


64


noob.save(function( err, result) { if( err) return console.error(err); if (result) console.log('Added!');});

Function for saving

65

Questions?var noob= new Participant({ name: 'Noob', skills:[]});noob.save(function( err, result) { if( err) return console.error(err); if (result) console.log('Added!');});=> Added!var geek= new Participant({ name: 'Geek', skills:['Node.js' , 'MongoDB']});geek.save(function (err, result) { if (err) return console.error(err); if (result) console.log('Added!');});=> Added!

66

How to use Mongoose?Participant.find(function (err, result) { if (err) return console.error(err); console.log(result)});=> [ { _id: 5375fb170ccdac981527120e, name: 'Noob', __v: 0, skills: [] }, { _id: 5375fb170ccdac981527120f, name: 'Geek', __v: 0, skills: [ 'Node.js', 'MongoDB' ] } ]

67

How to use Mongoose?Participant.find(function (err, result) { if (err) return console.error(err); console.log(result)});=> [ { _id: 5375fb170ccdac981527120e, name: 'Noob', __v: 0, skills: [] }, { _id: 5375fb170ccdac981527120f, name: 'Geek', __v: 0, skills: [ 'Node.js', 'MongoDB' ] } ]

68


Participant.find(function (err, result) { if (err) return console.error(err); console.log(result)});

Returns An Array of Objects

69

How to use Mongoose?Participant.findOne({ name: 'Geek' }, function (err, result) { if (err) return console.error(err); console.log('Geeks skills:'); console.log(result.skills);});

=> Geeks skills: [ Node.js, MongoDB]

70

How to use Mongoose?Participant.find({ name: 'Geek' }, 'skills',function (err, result) { if (err) return console.error(err); console.log('Geeks skills:'); console.log(result); });=> Geeks skills: [ { _id: 5375f79aaa515144156ee3eb, skills: [ 'Node.js', 'MongoDB' ] } ]

71


72


Participant.find({ name: 'Geek' }, 'skills',function (err, result) {

Key

Searches Objects with the Name 'Geek' and returns the Value for the Key

'skills'

73


74

What are indexes and why do i need them?indexes are similar to a table of contents in a bookit keeps you from having to read the whole book to get an answer on a questioninstead you can find the right page by checkin the indexmore technically said you speed up the query by allowing it to search a smaller subset of data

75

MongolabDemotime!

76

How to make Indexes?db.collection("participants").ensureIndex( {name: 1 }, function(err) { if (err) throw err; else console.log('done writing indexes'); });

db.participants.ensureIndex( {name: 1 })

77

How to make Indexes?

db.collection("participants").ensureIndex( {name: 1 }, function(err) { if (err) throw err; else console.log('done writing indexes'); });

db.participants.ensureIndex( {name: 1 })

Get a Collection

Command for making Indexes

Name of the Key that has to be indexed

1 = Ascending

2 = Descending

78

Questions?

79

Exercise Datadb.collection('querys').insert({name: "a", number: 1}) db.collection('querys').insert({name: "a", number: 2}) db.collection('querys').insert({name: "a", number: 3}) db.collection('querys').insert({name: "b", number: 1}) db.collection('querys').insert({name: "b", number: 2})db.collection('querys').insert({name: "b", number: 3})db.collection('querys').insert({name: "c", number: 1})db.collection('querys').insert({name: "c", number: 2})db.collection('querys').insert({name: "c", number: 3, bool: false})

80

ExerciseWrite a Node.js application that connects to : mongodb://usr:[email protected]:45089/workshopwrite functions that return items matching the following conditions1. name=a2. name=a, number=33. name=a or number=34. number is smaller than 25. all items that have a field called bool6. name is not b , number is biger than 2

81

A possible solutionfunction get(query) { db.collection('querys').find(query).toArray(function (err, result) { if (err) throw err; console.log('Query = ' + JSON.stringify(query)); console.log(result); });}

82

Resultquery ={name:'a'};

[ { _id: 537623665e7e2f181746dbd3, name: 'a', number: 2 }, { _id: 537623665e7e2f181746dbd2, name: 'a', number: 1 }, { _id: 537623665e7e2f181746dbd4, name: 'a', number: 3 } ]

83

ResultQuery = {"$and":[{" name":"a"},{"number":3}]} [ { _id: 537623665e7e2f181746dbd4, name: 'a', number: 3 } ]

84

ResultQuery = {"$or":[{"name":"a"},{"number":3}]} [ { _id: 537623665e7e2f181746dbd3, name: 'a', number: 2 }, { _id: 537623665e7e2f181746dbda, name: 'c', number: 3, bool: false }, { _id: 537623665e7e2f181746dbd2, name: 'a', number: 1 }, { _id: 537623665e7e2f181746dbd4, name: 'a', number: 3 }, { _id: 537623665e7e2f181746dbd7, name: 'b', number: 3 } ]

85

ResultQuery = {"number":{"$lt":2}} [ { _id: 537623665e7e2f181746dbd5, name: 'b', number: 1 }, { _id: 537623665e7e2f181746dbd8, name: 'c', number: 1 }, { _id: 537623665e7e2f181746dbd2, name: 'a', number: 1 } ]

86

ResultQuery = {"bool":{"$exists":true}} [ { _id: 537623665e7e2f181746dbda, name: 'c', number: 3, bool: false } ]

87

ResultQuery = {"$and":[{"name":{"$ne":"b"}},{"number":{"$gt":2}}]}

[ { _id: 537623665e7e2f181746dbda, name: 'c', number: 3, bool: false }, { _id: 537623665e7e2f181746dbd4, name: 'a', number: 3 } ]

Software

node.js workshop- node.js databases