Write code you can depend on!THE HIDDEN COST OF DEPENDENCIES AND AVOIDING THE LEFT-PAD PROBLEM

Who are you?

About Me

Logan Spears

Software Consultant since 2011

Director of Development ShaleApps2015

Worked on

iOS

Android

Server (mostly go)

Some web

github.com/loganjspears

What is the left-pad problem?

The left-pad Problem

npm package left-pad was unpublished

left-pad was a dependency of React, Babel, and other major packages.

Left-pad and its dependents were unavailable for 2.5 hours during the outage.

npm Strikes Back

npm restored left-pad and other affected packages

They severely restricted un-publishing packages

The day is saved! Total down time was only 2.5 hours. Glad it won’t happen again!

Can we review dependencies for a sec?

Dependency Pros and Cons

Pros

You get to stand on the shoulders of programming giants

Developer productivity is increased

DRY principals are upheld

Cons

Cognitive load increased

Abandoned projects are common

Reproducible builds can be more difficult

Surface area is larger for bugs and security vulnerabilities

Is there a larger problem here?

The Larger Problem

Dependency graphs are extensive

express dependency graph includes:

41 nodes

31 maintainers

“ee-first” contains two functions

“range-parser” contains one function

Not all dependency versions are pinned

Review of Semantic Versioning

MAJOR version when you make incompatible API changes,

MINOR version when you add functionality in a backwards-compatible manner, and

PATCH version when you make backwards-compatible bug fixes.

Express dependencies shown to the right. The “~” indicates packages that will automatically absorb patch numbers.

Breaking the Build

Can you trust 31 maintainers and countless contributors to push bug free code?

Semantic versioning is not a hard guarantee

You can’t guarantee your dependencies will pin their dependencies’ version numbers.

Security Concerns

rimrafall malicious package executing rm -rf /* /.*

npm accounts of dependency maintainers can be compromised

Commonly confused packages e.g. “express” vs “expressjs”

I deal with it now. What’s the big deal?

npm install != npm install

npm install can produce different results with the same package.json

Heroku, for example, uses npm install for deployments

Best practices advise to exclude dependencies from git

Your dev, CI, and production environment can all act differently

Everything can break when you push to production!

npm.org might be down!

Ok fine I care. What should I do?

Make your builds reproducible!

Check your dependencies into git

Architecture problems can arise

Have to remove build artifacts

npm shrinkwrap

Build Docker image (shown later)

Reduce your dependencies!

Okay you hate npm are we done?

Nope. Lets take a look at Go!

Out of the box

Typical workflow

Write code

import “github.com/org/dep”

go get

Write more code

Shared $GOPATH

Go is a new awesome language. There can’t be any problems here!

go get… Problems

$GOPATH dependency versions can conflict

Dependencies point to master branch which can change

go get can produce different results from the same source

Can I have semantic versioning back?

How do I avoid getting shot by that gopher?

Hacks prior to Go 1.6

gopkg.in

URL based versioning

Ex. “gopkg.in/yaml.v1”

GB

Alternative go tool

Separate $GOPATH for each project

godep

Embed dependencies in source

Rewrite import paths

>= Go 1.6

Vendor support added in Go 1.6

Tools now copy source into vendor directory

godep uses /vendor

Builds now reproducible from just the source

Your build can’t fail if github is down!

I am cool and use Docker. Am I covered?

Docker w/ Node & npm

Bad

CI and Production using source and Dockerfile instead of image

Good

Saving your image and running tests on it

Docker w/ Go

Bad

Good

“”

A little copying is better than a little dependency.

ROB PIKE