+

JavaScript Dependency Management Breaking Down the Barriers to Modular JavaScript Systems

Sean M. Duncan

J S

D M

0 1 0 0 1 0 1 0

0 1 0 1 0 0 1 1

0 1 0 0 0 1 0 0

0 1 0 0 1 1 0 1

+

JS

MD

Sean M. Duncan

  Senior Software Consultant with Solution Design Group

  9 years of experience working with Web applications

  Primarily Enterprise Java space with a recent focus on dynamic languages   Carol.com (Groovy/Grails)

  Giftag.com (Python/Django/Google App Engine)

  Giftag Firefox Addon (JavaScript)

+

JS

MD

My Reintroduction to JavaScript

  I’m a fairly new believer in the potential for elegant and maintainable large-scale JavaScript systems

  While working on the Giftag Firefox addon I started to see a side of JavaScript that had escaped me before

  Douglas Crockford would call it “The Good Parts”. I call it “Professional-Grade JavaScript”

+

JS

MD

JavaScript’s Sordid Past Small tasks in limited contexts

  Writing systems that behaved consistently across popular browsers was difficult

  JavaScript’s scoping confused developers due to inconsistency with other languages that share a C-like syntax

  Debugging support for JavaScript was severely lacking

  Testing and verification tools were either not available or still in an early state of evolution

+

JS

MD

JavaScript’s Sorted Present Rich Internet Applications Demand More

  Cross-browser consistency is now much easier to achieve

  Quality of JavaScript literature has drastically improved   JavaScript: The Good Parts by Douglas Crockford

  Pro JavaScript Techniques by John Resig

  High-quality, general-purpose frameworks are now widely available

  Browser vendors and extension developers have made and continue to make strides in the area of debugging tools

  Quality testing and verification tools are now available

+

JS

MD

Professional-Grade JavaScript Modern JavaScript requires the same discipline and organization we employ with other platforms

  Focus on the parts of the language that exhibit consistency and steer clear of those that don’t (see JSLint)

  Practice Unobtrusive JavaScript by separating server-side data, client-side logic and content markup

  Understand functional scoping and closure

  Strive to have minimal impact on the global namespace

  Verify the proper function of your code (see jqUnit/jqMock)

  Decompose your system into loosely coupled, purpose-focused modules

  Organize your project source in a manner that caters to intuition

+

JS

MD

The Case for Modular JavaScript

  Proven approach for other platforms that scale well to large bodies of code   Java, Python, Ruby and Groovy

  Loosely coupled systems are easier to enhance and maintain

  Purpose-focused modules are easier to verify and reuse

  Lends itself well to both object-oriented and functional solutions (see Python)

  Enables us to divide and conquer larger problems through layers of abstraction

+

JS

MD

The Elements of Modularity

  Logical Modularity:   The problem domain concepts are captured in discrete software

artifacts

  These software artifacts must have minimal knowledge of other components within the system

  Components work together at different layers of abstraction to accomplish the software’s goal

  Physical Modularity:   File system resources are organized parallel to the concepts

within the problem domain

  Easiest to accomplish when file system structures mirror logical structures

+

JS

MD

JavaScript Makes Modularity Difficult JavaScript as a language facilitates modularity; JavaScript as a platform stands in the way

  Developers must manually enter script tags to load modules

  Direct and transitive dependencies must be explicitly declared

  Tags must be arranged in transitive dependency order

  Pages become tightly coupled to modules they should need no knowledge of

  Manual dependency management is initially tedious, and maintenance is worse

+

JS

MD

Transitive Dependency Management

  Modules declare their direct dependencies and the module loader/manager takes it from there

  We couldn’t imagine Java, Python, Groovy or Ruby without it

  Why not JavaScript?   See JavaScript’s Past (small tasks)

  An important tool that should be in our Professional-Grade JavaScript toolbox

+

JS

MD

First-Generation Approaches

  Script Tag Append 1.  A dependent module declares a dependency with something

like module.require(‘foo’);

2.  The dependency manager adds a <script src=“foo.js”/> element to the Document Object Model (DOM)

  Ajax and Eval 1.  A dependent module declares a dependency with something

like module.require(‘foo’);

2.  The dependency manager makes an Ajax request for the module with xhr.open(‘GET’, ‘foo.js’, false); send(null); …

3.  The dependency manager loads the response text by calling window.eval(xhr.responseText);

+

JS

MD

Script Tag Append

  Pros:   Simple to implement

  Standard caching behavior

  Source code is properly attributed to its file resource

  Cons:   The timing of the response to the script element introduction is

inconsistent from one browser to the next

  Dependent modules may be interpreted before their dependencies

+

JS

MD

Ajax and Eval

  Pros:   Still relatively simple to implement

  Standard caching behavior

  Source can be loaded synchronously, making transitive dependency order easy to guarantee

  Cons:   The use of window.eval(xhr.responseText) causes source to be

attributed to the module manager rather than the module’s file resource

  Some browsers have been known to ignore the asynch flag in the xhr.open(method, url, asynch) call

+

JS

MD

Dojo Toolkit’s Module Loader

  Ajax and Eval with Global Callback

  Translates module names into file resources in a manner similar to Java packages

  Registered ’onLoad’ callbacks are invoked when all pending modules are loaded

  Originally designed to streamline the distribution of the Dojo Toolkit rather than to be a general solution

+

JS

MD

Dojo Toolkit Code Examples

 Publish a module:dojo.provide(‘my.module’);

 Create a module:dojo.declare(‘my.module’, null, { // module body});

  Declare a dependency: dojo.require(‘your.module’);

  Run some dependent code: dojo.addOnLoad(function() { // code that depends on ‘your.module’});

+

JS

MD

YUI Loader

  Script Tag Append with Availability Polling and Callbacks

  Feature-rich but biased toward visual components   Can be used for CSS as well as JavaScript

  Rollup feature loads bundles of related modules in one file

  Availability determination is complex and inconsistent across browsers   “Nothing can be done … except to pause and hope for the best.”

  Availability Polling:   YUI loader polls for a variable that non-YUI scripts publish

when they are interpreted

+

JS

MD

YUI Loader Code Examples

  Publish a module to the loader: loader.addModule({ name: ‘colorpicker’, type: ‘js’, varName: ‘colorPickerLoaded’, // availability polling fullPath: ‘http://www.example.com/js/module.js’, requires: [‘module.a’, ‘module.b’]});

  Configure the loader and pull in some dependencies: var loader = new YAHOO.util.YUILoader({ require: ["colorpicker", "treeview"], onSuccess: function() { // code that depends on colorpicker and treeview }, combine: true // allow “rollup” of required modules});loader.insert();

+

JS

MD

jQuery & Prototype

  Ajax and Eval / Basic Script GET

  Neither really trying to be a player in the dependency management space

  jQuery:$.getScript(‘scripts/foo/bar/baz.js’, function() { // code that requires ‘foo.bar.baz’});

  Prototype:var getScript = function(url, callback) { new Ajax.Request(url, { method: ‘get’, onSuccess: function(transport) { window.eval(transport); callback(); } });};

+

Jingo JavaScript Dependency Manager jingo.googlecode.com

Breaking Down the Barriers to Modular JavaScript Systems

John Bailey Sean M. Duncan

I N

G OJ

+

JS

MD

Jingo

  Created to solve the problem of JavaScript dependency management in a lightweight, framework-agnostic package

  Avoids technical limitations and framework bias of common approaches to JavaScript dependency management

  Streamlines both the development and maintenance of modular JavaScript systems

+

JS

MD

How Jingo Works

  Works within the constraints of browser processes by extending the script lifecycle

  Combines Script Tag Append and Module Body Callbacks to make modules available in transitive dependency order

  Module Resolution advances modules through a progression of three distinct states

+

JS

MD

The Module Lifecycle: Loading

  Translates module names into file resources in a manner similar to Java packages

  Script tag with its ‘src’ attribute set to the translated resource path is appended to the DOM

+

JS

MD

The Module Lifecycle: Declared

  Browser loads the source for a given module, interprets the file, and executes the jingo.declare call

  Jingo registers the module’s body callback and ensures all direct dependencies are transitioned into loading state

+

JS

MD

The Module Lifecycle: Resolved

  Modules with no dependencies are immediately resolved

  Modules are resolved by calling the registered body callback

  Jingo continues to scan for declared modules ready for resolution

  Process repeats until all declared modules and their dependencies have been resolved

  Module Body Callbacks are guaranteed to be invoked in transitive dependency order

+

JS

MD

Visualizing the Module Lifecycle

<script> jingo.anonymous({ require: [‘B’, ‘C’], exec: function() { // module A } }); </script>

<script src=“jingo.js”/>

- Dependency Tree - - Page Source -

+

JS

MD

Visualizing the Module Lifecycle

<script>...</script> <script src=“jingo.js”/>

<script src=“B.js”/> <script src=“C.js”/>

- Dependency Tree - - Page Source -

+

JS

MD

Visualizing the Module Lifecycle

<script>...</script> <script src=“jingo.js”/>

<script src=“B.js”/> <script src=“C.js”/> <script src=“D.js”/> <script src=“E.js”/>

- Dependency Tree - - Page Source -

+

JS

MD

Visualizing the Module Lifecycle

<script>...</script> <script src=“jingo.js”/>

<script src=“B.js”/> <script src=“C.js”/> <script src=“D.js”/> <script src=“E.js”/> <script src=“F.js”/>

- Dependency Tree - - Page Source -

+

JS

MD

Visualizing the Module Lifecycle

<script>...</script> <script src=“jingo.js”/>

<script src=“B.js”/> <script src=“C.js”/> <script src=“D.js”/> <script src=“E.js”/> <script src=“F.js”/> <script src=“G.js”/> <script src=“H.js”/>

- Dependency Tree - - Page Source -

+

JS

MD

Getting Started with Jingo Initializing Jingo

  Jingo’s main repository defaults to ‘scripts’, its logging verbosity defaults to ‘warn’ and its script loading timeout defaults to 30 seconds

  If the defaults work for your project, initialization is unnecessary

  Override the default main repository and logging verbosity: jingo.init({ repos: { main: ‘../scripts’ }, verbosity: ’debug’, timeout: 10000});

+

JS

MD

Getting Started with Jingo A Simple Reusable Module

  Declare a single module that doesn’t require any other modules to do its work: jingo.declare({ name: 'Greeter’, as: function() { Greeter = function() { this.welcome = function(name) { alert('Hello ' + name + ', how are you?'); }; }; }});

+

JS

MD

Getting Started with Jingo A Namespaced Reusable Module

  Unfortunately, the previous example pollutes the global namespace

  Introduce a namespace called hallmart: jingo.declare({ name: ’hallmart.Greeter’, as: function() { hallmart.Greeter = function() { this.welcome = function(name) { alert('Hello ' + name + ', how are you?'); }; }; }});

  Jingo ensures the existence of the enclosing namespace for us

+

JS

MD

Getting Started with Jingo A Reusable Module with Nested Namespacing

  This pattern works for arbitrarily deep namespaces as well

  Create an iteration module nested within a utilities package: jingo.declare({ name: ‘hallmart.util.iterators’, as: function() { hallmart.util.iterators = { each: function(data, closure) { for(key in data) { if(data.hasOwnProperty(key) { closure(key, data[key]); } } }; }; }});

+

JS

MD

Getting Started with Jingo A Reusable Module with Dependencies

  Modules only have to declare their direct dependencies

  Declare a dependent module: jingo.declare({ require: [ ‘hallmart.Greeter’ ], name: ‘hallmart.Store’, as: function() { hallmart.Store = function() { var greeter = new hallmart.Greeter(); this.admit = function(customer) { greeter.welcome(customer.name); }; }; }});

+

JS

MD

Getting Started with Jingo Declaring an Anonymous Module

<html>...<script type=“text/javascript” src=“scripts/jingo.js”></script><script type=“text/javascript”> jingo.anonymous({ require: [‘hallmart.util.iterators’, ‘hallmart.Store’], exec: function() { var each = hallmart.util.iterators.each; var customers = [{name: ‘Bil’}, {name: ‘Ted’}]; var store = new hallmart.Store(); each(customers, function(index, customer) { store.admit(customer); }; } });</script>...</html>

+

JS

MD

Getting Started with Jingo Rollup Module for Unobtrusive JavaScript

jingo.declare({ require: [‘hallmart.util.iterators’, ‘hallmart.Store’], name: ‘hallmart.controller.foo’, as: function() { var each = hallmart.util.iterators.each; var customers = [{name: ‘Bil’}, {name: ‘Ted’}]; var store = new hallmart.Store(); each(customers, function(index, customer) { store.admit(customer); }; }});

...<script type=“text/javascript” src=“scripts/jingo.js”/><script type=“text/javascript”> jingo.anonymous({ require: [‘hallmart.controller.foo’] });</script>...

+

JS

MD

Getting Started with Jingo Cross-Project Reuse via Multiple Module Repositories

  Modularity creates opportunities for reuse beyond the confines of a single project: jingo.init({ repos: { mib: ‘http://mib.security.com/scripts’ }});

  Require modules from multiple repositories: jingo.declare({ require: [ ‘mib:security.guard’, ‘hallmart.Greeter’ ], name: ‘hallmart.Store’, as: function() { // module body }});

+

JS

MD

Distinguishing Features

  Cross-browser consistency   Firefox, Internet Explorer, Safari, Chrome, Opera

  Small footprint   Download size < 6KB

  Increased module load efficiency may lead to net decrease in overall network traffic

  Standard caching   Jingo module source is cached in the same manner as source

loaded via manual script tag entry

  Debugging support   Source code is attributed to the appropriate file resource

+

JS

MD

Distinguishing Features, continued

  Framework-agnostic   Focused, unbiased, standalone JavaScript dependency solution

  Scope assurance   Module Body Callbacks guarantee the presence of a non-global

enclosing scope in the body of every Jingo module

  Namespace assurance   Jingo automatically ensures the existence of enclosing

namespaces for namespaced modules

  Intuitive project structure   File system organization mirrors the logical module

structure

+

JS

MD

Ideas for the Future

  Central script repository   jingo.init({ repos: { jingo: ‘http://repos.jingo-js.com/scripts’ }});

jingo.declare({ require: [ ‘jingo:foo.bar.baz’ ], ...});

  Jingo Server-Side   Same syntax as Jingo but server-side process builds complete

dependency graph rollup and returns it as one file   Server-side caching of assembled script rollups

  Any suggestions?

+

JS

MD

Thanks Everyone

  Particularly:   Refactr (hosting)

  Solution Design Group (pizza!)

+

JS

MD

Live Code Demo with John

  jingo.googlecode.com

  Sean Duncan sean@artisanlogic.com

  John Bailey john@artisanlogic.com

+

JS

MD

Questions?

  jingo.googlecode.com

  Sean Duncan sean@artisanlogic.com

  John Bailey john@artisanlogic.com

+

JS

MD

General-Purpose Frameworks

  jQuery (www.jquery.com)

  Prototype (www.prototypejs.com)

  Dojo Toolkit (www.dojotoolkit.com)

  ExtJS (www.extjs.com)

  YUI (developer.yahoo.com/yui/)

  Archetype (www.archetypejs.org)

  …

+

JS

MD

Testing and Verification Tools

  Unit Testing   jqUnit (jqunit.googlecode.com)

  JsUnit (www.jsunit.net)

  Mock Objects   jqMock (jqmock.googlecode.com)

  JSMock (jsmock.sourceforge.net)

  JSCoverage (siliconforks.com/jscoverage/)

  JSLint (www.jslint.com)

  Selenium (seleniumhq.org)

+

JS

MD

Other Valuable Tools

  Compaction   JSMin (www.crockford.com/javascript/jsmin.html)

  Packer (dean.edwards.name/packer/)

  Dojo ShrinkSafe (dojotoolkit.org/docs/shrinksafe)

  YUI Compressor (developer.yahoo.com/yui/compressor/)

  Embedded Documentation   JSDoc Toolkit (code.google.com/p/jsdoc-toolkit/)