68
BIG application design Saturday, November 10, 12

Big app design for Node.js

Embed Size (px)

DESCRIPTION

Designing the code of a large application in production and scaling up to thousands of users while doing it all in Node.js is a challenge that no developer should face alone. I will discuss how we have developed a highly decoupled, plugin-based architecture and a decentralized infrastructure for Cloud9 IDE, along with the technologies we've developed and the difficulties we faced in order to build the largest Node.js application that exists in production.

Citation preview

Page 1: Big app design for Node.js

BIGapplication

designSaturday, November 10, 12

Page 2: Big app design for Node.js

@sergimansilla

Saturday, November 10, 12

Page 3: Big app design for Node.js

http://c9.io

Saturday, November 10, 12

Page 4: Big app design for Node.js

Smart code completion

Saturday, November 10, 12

Page 5: Big app design for Node.js

Static analysis

Saturday, November 10, 12

Page 6: Big app design for Node.js

Free linux VM

Saturday, November 10, 12

Page 7: Big app design for Node.js

Bring your own machineSaturday, November 10, 12

Page 8: Big app design for Node.js

Real terminalSaturday, November 10, 12

Page 9: Big app design for Node.js

Hundreds of thousands of lines of JavaScript

Saturday, November 10, 12

Page 10: Big app design for Node.js

dynamic,weakly-typed

language

Saturday, November 10, 12

Page 11: Big app design for Node.js

one single thread

Saturday, November 10, 12

Page 12: Big app design for Node.js

Remote VM

Cloud9 datacenter

IDE serverRemote

VM

Remote VM

Openshift

Saturday, November 10, 12

Page 13: Big app design for Node.js

Saturday, November 10, 12

Page 14: Big app design for Node.js

Saturday, November 10, 12

Page 15: Big app design for Node.js

Modularization

Saturday, November 10, 12

Page 16: Big app design for Node.js

Black box coding

Saturday, November 10, 12

Page 17: Big app design for Node.js

Saturday, November 10, 12

Page 18: Big app design for Node.js

Good at abstracting away

require

Saturday, November 10, 12

Page 19: Big app design for Node.js

Not that good at app-level abstraction

require

Saturday, November 10, 12

Page 20: Big app design for Node.js

duplicated modules

maps to folder names

Relies on FSrequ

ire

hard to configure module

Saturday, November 10, 12

Page 21: Big app design for Node.js

Dependency error handling

coding time compile time run time

KA-BO

OM!

- Server crash- Unhappy customers- Developer gets fired

Saturday, November 10, 12

Page 22: Big app design for Node.js

How to fix it?

Static dependency list

Named services

Easy configuration

Resolve at startup

No FS required

Pass an object

Saturday, November 10, 12

Page 23: Big app design for Node.js

Architect

github.com/c9/architect

Saturday, November 10, 12

Page 24: Big app design for Node.js

Everything is a plugin

Plugins can consume plugins

An application is just a set of plugins

Architect

Saturday, November 10, 12

Page 25: Big app design for Node.js

Dependency model

require

Architect

runtime

compile time

Saturday, November 10, 12

Page 26: Big app design for Node.js

Dependency error handlingArchitect

coding time compile time run time

- Happy customers- Developer keeps job- Fails before release

Saturday, November 10, 12

Page 27: Big app design for Node.js

github.com/sergi/nodejsconfit2012

Saturday, November 10, 12

Page 28: Big app design for Node.js

Saturday, November 10, 12

Page 29: Big app design for Node.js

Declare entity ‘cook’ with behavior

Use ‘cook’ to do some cooking

Saturday, November 10, 12

Page 30: Big app design for Node.js

Cook Cooking action

Dependency model

Saturday, November 10, 12

Page 31: Big app design for Node.js

package.json

Builds dependency tree

executing codewithout

Saturday, November 10, 12

Page 32: Big app design for Node.js

{{

cook/package.json

do-­‐cooking/package.json

Saturday, November 10, 12

Page 33: Big app design for Node.js

Next

Extract the code

Wrap in Architect plugin code

Make two plugins

Saturday, November 10, 12

Page 34: Big app design for Node.js

Function signature

Call when done

Saturday, November 10, 12

Page 35: Big app design for Node.js

Architect plugin code

Module.exports

Options - we’ll talk about it later

Imports - everything you ‘consume’

Register - call when done

Saturday, November 10, 12

Page 36: Big app design for Node.js

Saturday, November 10, 12

Page 37: Big app design for Node.js

Saturday, November 10, 12

Page 38: Big app design for Node.js

Easy to testMock dependencies

Saturday, November 10, 12

Page 39: Big app design for Node.js

‘cook’ should be called 8 times

Saturday, November 10, 12

Page 40: Big app design for Node.js

No black magic

Specify dependency model

Feed architect a config file

Call ‘createApp’

Saturday, November 10, 12

Page 41: Big app design for Node.js

Saturday, November 10, 12

Page 42: Big app design for Node.js

Saturday, November 10, 12

Page 43: Big app design for Node.js

Configuration

Per-plugin options

No global options object

Specify in config file

Saturday, November 10, 12

Page 44: Big app design for Node.js

Saturday, November 10, 12

Page 45: Big app design for Node.js

Saturday, November 10, 12

Page 46: Big app design for Node.js

Options

Passed in at startup

Options can be dependencies

Fail if options aren’t present

Saturday, November 10, 12

Page 47: Big app design for Node.js

Saturday, November 10, 12

Page 48: Big app design for Node.js

Saturday, November 10, 12

Page 49: Big app design for Node.js

Your app as chunks of functionality

rather than sets of classes

Saturday, November 10, 12

Page 50: Big app design for Node.js

chunks of functionality

Implicit type constraints

Keep implementation private

Swap features instead of interfaces

Saturday, November 10, 12

Page 51: Big app design for Node.js

How does Cloud9 use it?

Open source version

Local version (OS + sync)

Hosted version

Normal

FTP

SSH

Saturday, November 10, 12

Page 52: Big app design for Node.js

Saturday, November 10, 12

Page 53: Big app design for Node.js

Swap feature per implementation

On OSS: talk local filesystem

On FTP: talk FTP library

On SSH: talk via a SSH bridge

Saturday, November 10, 12

Page 54: Big app design for Node.js

Architect is a DI framework

Saturday, November 10, 12

Page 55: Big app design for Node.js

But a quite special one

Saturday, November 10, 12

Page 56: Big app design for Node.js

IDE instance(FTP)

IDE instance(SSH)

IDE instance(Normal)

IDE instance(Normal)

Single node.js process

Other code (dashboard etc.)

Saturday, November 10, 12

Page 57: Big app design for Node.js

Architect can do

Multiple instances of same pluginRun independently

But in the same process

Saturday, November 10, 12

Page 58: Big app design for Node.js

Centralized event bus

Loose coupling between plugins

No hard dependencies!

Can also do inter-context communication

Saturday, November 10, 12

Page 59: Big app design for Node.js

Eventbus

Plugin

Other plugin

Emit event

React on event

Saturday, November 10, 12

Page 60: Big app design for Node.js

Saturday, November 10, 12

Page 61: Big app design for Node.js

Saturday, November 10, 12

Page 62: Big app design for Node.js

And now scale

Swap it with i.e. Redis PubSub

Plugins will never notice

Modular awesomeness!

Saturday, November 10, 12

Page 63: Big app design for Node.js

Modularize in feature blocks

Don’t over engineer

Don’t create too small blocks

They aren’t interfaces!

Saturday, November 10, 12

Page 64: Big app design for Node.js

Avoid context switching

Less code!

Less errors!

Less boilerplate!

Saturday, November 10, 12

Page 65: Big app design for Node.js

Loose coupling

Use (don’t abuse) an event bus

Smaller dependency graph

Saturday, November 10, 12

Page 66: Big app design for Node.js

github.com/c9/architect

Saturday, November 10, 12

Page 67: Big app design for Node.js

Happy coding!

Saturday, November 10, 12

Page 68: Big app design for Node.js

@sergimansillagithub.com/sergi

http://c9.io

Saturday, November 10, 12