Sunnie Chung CIS 612
MongoDB Guide CIS 612 SS Chung
1. Installation
1) installing MongoDb
Sudo apt-get install mongoDb
after installing mongoDB, we have to make changes on it
Update apt-get update
2) Starting mogo services
MongoDB 2.4.9 (mongodb-osx-x86_64-2.4.9)
To start a MongoDB instance:
$ mongod mongod --help for help and startup options Tue Apr 1 15:19:17.445 [initandlisten] MongoDB starting : pid=616 port=27017
Sunnie Chung CIS 612
dbpath=/data/db/ 64-bit host=Thuats-MacBook-Pro.local Tue Apr 1 15:19:17.445 [initandlisten] Tue Apr 1 15:19:17.445 [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000 Tue Apr 1 15:19:17.445 [initandlisten] db version v2.4.9 Tue Apr 1 15:19:17.445 [initandlisten] git version:
…
2. MongoDB Shell
MongoDB comes with a JavaScript shell that allows interaction with a MongoDB
instance from the command line.
To start the shell: $ mongo MongoDB shell version: 2.4.9 connecting to: test Welcome to the MongoDB shell. For interactive help, type "help". For more comprehensive documentation, see http://docs.mongodb.org/ Questions? Try the support group http://groups.google.com/group/mongodb-user Server has startup warnings: Tue Apr 1 15:19:17.445 [initandlisten] Tue Apr 1 15:19:17.445 [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000 >
The shell is a full-featured JavaScript interpreter, capable of running JavaScript
programs.
Sunnie Chung CIS 612
3) get into mongo shell to check whether shell is running or not
4) From the mongo shell, confirm you are in the mydb database by issuing the
following:
Sunnie Chung CIS 612
5) If mongo does not return mydb for the previous operation, set the context to
the database, with the following operation:
6) Inserting into Database
Sunnie Chung CIS 612
Sunnie Chung CIS 612
• Creating a Documents j and k and insert into employee collection
Sunnie Chung CIS 612
• Finding particular data from the Collection
• Showing Collections in mydb
• Fetching distinct value
SQL Terms/Concepts MongoDB Terms/Concepts
Sunnie Chung CIS 612
SQL Terms/Concepts MongoDB Terms/Concepts
Database
Database
table Collection
row Document
column Field
index index
table joins embedded documents and linking
primary key
Specify any unique column or column combination as primary
key.
primary key
In MongoDB, the primary key is automatically set to
the_id field.
aggregation (e.g. group by) aggregation
SQL Terms, Functions, and Concepts MongoDB Aggregation Operators
Sunnie Chung CIS 612
SQL Terms, Functions, and Concepts MongoDB Aggregation Operators
WHERE $match
GROUP BY $group
HAVING $match
SELECT $project
ORDER BY $sort
LIMIT $limit
SUM() $sum
3. MongoDB Query/Syntax
The shell is standalone MongoDB client. On startup, the shell connects to the ‘test’
database and assigns this database connection to global variable ‘db’.
To show current databases > show dbs
local 0.078125GB
To create a new database: > use blog
If there is a database exists, then it switches to that one.
The CRUD operations used to manipulate and view data in the shell.
Sunnie Chung CIS 612
Create a new document: > post = {"title": "My Blog Post", "content" : "This is a blog post.", "data" : new Date()} { "title" : "My Blog Post", "content" : "This is a blog post.", "data" : ISODate("2014-04-01T19:39:36.521Z") }
‘post’ is a JavaScript object represents the documents, there are three keys ‘title’,
‘content’, and ‘date’
Insert into collection: > db.blog.insert(post)
To see the collection: > db.blog.find() { "_id" : ObjectId("533b16898bce20d2fd851cfc"), "title" : "My Blog Post", "content" : "This is a blog post.", "data" : ISODate("2014-04-01T19:39:36.521Z") }
> db.blog.findOne() { "_id" : ObjectId("533b16898bce20d2fd851cfc"), "title" : "My Blog Post", "content" : "This is a blog post.", "data" : ISODate("2014-04-01T19:39:36.521Z") }
To see how MongoDB created that document: > db.blog.find().explain() { "cursor" : "BasicCursor", "isMultiKey" : false, "n" : 1, "nscannedObjects" : 1, "nscanned" : 1, "nscannedObjectsAllPlans" : 1, "nscannedAllPlans" : 1, "scanAndOrder" : false, "indexOnly" : false, "nYields" : 0, "nChunkSkips" : 0, "millis" : 0, "indexBounds" : { }, "server" : "Thuats-MacBook-Pro.local:27017"
}
To update:
Sunnie Chung CIS 612
> post.comments = [] [ ] > db.blog.update({title: "My Blog Post"}, post) > db.blog.findOne() { "_id" : ObjectId("533b16898bce20d2fd851cfc"), "title" : "My Blog Post", "content" : "This is a blog post.", "data" : ISODate("2014-04-01T19:39:36.521Z"), "comments" : [ ] }
To delete: > db.blog.remove({title : "My Blog Post"}) > db.blog.findOne()
null
To build index: > db.blog.ensureIndex({title:1})
To show all existing indexes: > db.blog.getIndexes() [ { "v" : 1, "key" : { "_id" : 1 }, "ns" : "blog.blog", "name" : "_id_" }, { "v" : 1, "key" : { "title" : 1 }, "ns" : "blog.blog", "name" : "title_1" }
]
To remove index: > db.blog.dropIndex({title:1}) { "nIndexesWas" : 2, "ok" : 1 }
Sunnie Chung CIS 612
Insert with and without giving _Id
Sunnie Chung CIS 612
Store Function
Sunnie Chung CIS 612
Create Collection
Sunnie Chung CIS 612
Map and Reduce
4. MongoDB Application
MongoDB Drivers and Client Libraries: MongoDB supports variety of modern
programming languages including C, C++, C#, Java, Node.js, PHP, Python…
Example of MongoDB application with Java:
Sunnie Chung CIS 612
MongoDB-Java-Driver-2.12.0
try {
// connect to MongoDB server
MongoClient mongo = new MongoClient("localhost" , 27017);
// display all databases
List<String> dbs = mongo.getDatabaseNames();
System.out.println("Databases:");
for(String db : dbs) {
System.out.println(db);
}
System.out.println();
// display collections
DB db = mongo.getDB("blog");
Set<String> collections = db.getCollectionNames();
System.out.println("Collections:");
for (String string : collections) {
System.out.println(string);
}
System.out.println();
// save a document into collection
DBCollection collection = db.getCollection("blog");
BasicDBObject document = new BasicDBObject();
document.put("title", "My Blog Post 2");
document.put("content", "This is blog post 2.");
document.put("data", new Date());
collection.insert(document);
// update document
BasicDBObject oldObject = new BasicDBObject();
oldObject.put("title", "My Blog Post");
BasicDBObject newObject = new BasicDBObject();
newObject.put("title", "My Blog Post - updated");
BasicDBObject updateObject = new BasicDBObject();
updateObject.put("$set", newObject);
collection.update(oldObject, updateObject);
// find document
BasicDBObject searchObject = new BasicDBObject();
searchObject.put("title", "My Blog Post - updated");
DBCursor cursor = collection.find(searchObject);
while (cursor.hasNext()) {
System.out.println(cursor.next());
}
} catch (UnknownHostException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Output: Databases:
blog
local
Sunnie Chung CIS 612
Collections:
blog
system.indexes
{ "_id" : { "$oid" : "5346493bcf16e43d6ecc279d"} , "content" :
"This is a blog post." , "data" : { "$date" : "2014-04-
10T07:26:12.828Z"} , "title" : "My Blog Post - updated"}
5. MongoDB Import/Export
MongoDB can import input files of formats JSON, CSV or TSV and also can export
database to those format using mongoimport and mongoexport respectively.
Syntax: mongoimport --collection collection --file collection.json mongoexport --collection collection --out collection.json
Import a CSV file (NASDAQ_daily_prices_B.csv) into MongoDB collection stocks
$ cat NASDAQ_daily_prices_B.csv exchange,stock_symbol,date,stock_price_open,stock_price_high,stock_price_low,stock_price_close,stock_volume,stock_price_adj_close NASDAQ,BBND,2010-02-08,2.92,2.98,2.86,2.96,483800,2.96 NASDAQ,BBND,2010-02-05,2.85,2.94,2.79,2.93,884000,2.93 NASDAQ,BBND,2010-02-04,2.83,2.88,2.78,2.83,1333300,2.83 NASDAQ,BBND,2010-02-03,2.98,3.03,2.80,2.83,1015800,2.83 NASDAQ,BBND,2010-02-02,3.05,3.10,2.96,2.97,513100,2.97 NASDAQ,BBND,2010-02-01,3.11,3.13,3.00,3.04,997000,3.04 NASDAQ,BBND,2010-01-29,3.01,3.14,2.96,3.14,1132900,3.14 … $ mongoimport --db stocks --collection nasdaq_daily_prices --type csv --file /Users/nqt289/Desktop/NASDAQ_daily_prices_B.csv --headerline connected to: 127.0.0.1 Thu Apr 10 05:24:46.009 Progress: 780677/21998523 3% Thu Apr 10 05:24:46.009 14000 4666/second Thu Apr 10 05:24:49.004 Progress: 2011431/21998523 9% Thu Apr 10 05:24:49.004 36200 6033/second Thu Apr 10 05:24:52.004 Progress: 3300955/21998523 15% Thu Apr 10 05:24:52.004 58600 6511/second Thu Apr 10 05:24:55.005 Progress: 4575925/21998523 20% Thu Apr 10 05:24:55.006 81300 6775/second Thu Apr 10 05:24:58.009 Progress: 5845580/21998523 26% Thu Apr 10 05:24:58.009 104000 6933/second Thu Apr 10 05:25:34.005 374000 7333/second …
Sunnie Chung CIS 612
Thu Apr 10 05:25:35.956 check 9 388777 Thu Apr 10 05:25:35.956 imported 388776 objects
Check result collection in the shell: > show dbs blog 0.203125GB local 0.078125GB stocks 0.453125GB > use stocks switched to db stocks > show tables nasdaq_daily_prices system.indexes > db.nasdaq_daily_prices.find().limit(5) { "_id" : ObjectId("5346635c6857e587111a2466"), "exchange" : "NASDAQ", "stock_symbol" : "BBND", "date" : "2010-02-08", "stock_price_open" : 2.92, "stock_price_high" : 2.98, "stock_price_low" : 2.86, "stock_price_close" : 2.96, "stock_volume" : 483800, "stock_price_adj_close" : 2.96 } { "_id" : ObjectId("5346635c6857e587111a2467"), "exchange" : "NASDAQ", "stock_symbol" : "BBND", "date" : "2010-02-05", "stock_price_open" : 2.85, "stock_price_high" : 2.94, "stock_price_low" : 2.79, "stock_price_close" : 2.93, "stock_volume" : 884000, "stock_price_adj_close" : 2.93 } { "_id" : ObjectId("5346635c6857e587111a2468"), "exchange" : "NASDAQ", "stock_symbol" : "BBND", "date" : "2010-02-04", "stock_price_open" : 2.83, "stock_price_high" : 2.88, "stock_price_low" : 2.78, "stock_price_close" : 2.83, "stock_volume" : 1333300, "stock_price_adj_close" : 2.83 } { "_id" : ObjectId("5346635c6857e587111a2469"), "exchange" : "NASDAQ", "stock_symbol" : "BBND", "date" : "2010-02-03", "stock_price_open" : 2.98, "stock_price_high" : 3.03, "stock_price_low" : 2.8, "stock_price_close" : 2.83, "stock_volume" : 1015800, "stock_price_adj_close" : 2.83 } { "_id" : ObjectId("5346635c6857e587111a246a"), "exchange" : "NASDAQ", "stock_symbol" : "BBND", "date" : "2010-02-02", "stock_price_open" : 3.05, "stock_price_high" : 3.1, "stock_price_low" : 2.96, "stock_price_close" : 2.97, "stock_volume" : 513100, "stock_price_adj_close" : 2.97 }
Export that collection to JSON format: $ mongoexport -d stocks -c nasdaq_daily_prices -q "{stock_price_open: { \$gte: 50 }}" --out /Users/nqt289/Desktop/gte50.json connected to: 127.0.0.1 exported 9911 records
$ cat gte50.json { "_id" : { "$oid" : "5346635d6857e587111a4cda" }, "exchange" : "NASDAQ", "stock_symbol" : "BOLT", "date" : "2007-07-25", "stock_price_open" : 51, "stock_price_high" : 51.47, "stock_price_low" : 44.1, "stock_price_close" : 47.04, "stock_volume" : 1109600, "stock_price_adj_close" : 31.36 }
Sunnie Chung CIS 612
{ "_id" : { "$oid" : "5346635d6857e587111a4cdb" }, "exchange" : "NASDAQ", "stock_symbol" : "BOLT", "date" : "2007-07-24", "stock_price_open" : 52.4, "stock_price_high" : 52.4, "stock_price_low" : 48.55, "stock_price_close" : 49.43, "stock_volume" : 650600, "stock_price_adj_close" : 32.95 } …