Rapid Development with Schema-less Data models

The MyEdu Profile Project

Table of Contents

• Technology Stack• What is the MyEdu Profile Project?• MongoDb vs MySQL decision• Version 1 using MongoDB• Mistakes we made along the way• What we have learned along the way• What we are doing next

MyEdu Technology Stack

The MyEdu Profile Project

MyEdu is the leading academic and career network for college students and college recruiters. Millions of students have used MyEdu to graduate in less time, and reduce the cost of earning

their degree, and get jobs and internships. Employers use MyEdu to build their brands with highly targeted groups of

college students and hire the best talent for their companies.

We understood that change to the product was

inevitable and continuous.

MongoDb vs MySQL decision

• Technical goals for the user profile product– Move fast and iterate often.– Minimize the downtime required to launch new

features– Make sure the new product would scale

MongoDb vs MySQL decision

• Our decision to use MongoDB did not come without a lot of debate internally. In the end, the two main arguments were:– “We can do all that in MySQL.”– Schema-less design is going to be hard to

maintain.

MongoDb vs MySQL decision

• Argument 1: “We can do all this in MySQL.”– We could have built it in MySQL, but was going to

get complicated fast.– Sure, it would start simple.

MongoDb vs MySQL decision

• Well what if a user has more than one

MongoDb vs MySQL decision

• Add more meta data about a user

MongoDb vs MySQL decision

• All the sudden you have this schema and growing:

MongoDb vs MySQL decision

• The result of the web service call to get the user profile would look something like:

MongoDb vs MySQL decision

• The result of the web service call to get the user profile would look something like:

Looks like a Mongo document to me

MongoDb vs MySQL decision

• Argument 2: Schema-less design is going to be hard to maintain.• Lucky for us, the web services team had already been

employing a Service / Mapper / Model pattern. • With this pattern we are able to control the structure of

each mongo document with a defined data model.

Why we chose MongoDB?

• It goes back to our product goals:– Move fast and iterate often.– Minimize the downtime required to launch new

features– Make sure the new product would scale.

The MyEdu profile over the last 3 months

• MyEdu Profile Project iteration 1

The MyEdu profile over the last 3 months

• Iteration 2 – Adding more tiles

The MyEdu profile over the last 3 months

• Iteration 3 – Tile Management w/ Duplicate Tile types

Why we chose MongoDB?

• It goes back to our product goals:– Move fast and iterate often.– Minimize the downtime required to launch new

features– Make sure the new product would scale.

A product that can Scale

• With proper indexing alone we have over 600k mongo profile documents on just 3 member replica set.

• We actually lowered our overall site speed time by 0.3 seconds and lowered our profile product speed 300% seconds when we introduced the MongoDB version.

Why we chose MongoDB?

• It goes back to our product goals:– We want to move fast and iterate often.– We wanted to minimize the downtime required to

launch new features– We wanted to make sure the new product would

scale.

Mistakes we have made

• Proper Indexing is super important.

• You have to stop thinking like MySQL.

Bad Index

What we have learned along the way

• A Schema-less Data model has many benefits but the one we have found most useful so far is the ability to rapidly change our data to meet the needs of new product features and requirements.

• The document object is a familiar structure for a web developer.

What we are doing next

• Event tracking with MongoDB• Using Json Patch for doing updates