tus.io – Resumable file uploads for web and mobile apps by Felix Geisendörfer

felixge

tus.ioResumable file uploads for web and mobile apps

Felix Geisendörfer

?

felixge

About Me

felixge

I Open Source

felixge

felixge

@felixgefelixge

@felixgefelixge

Nodecopter.com

felixge

Implemented file upload?

felixge

The Case For Resumable File

Uploads

felixge

Content is king

@felixgefelixge

User Generated Content

felixge

If content is king, you should care about

acquiring it

felixge

Getting harder

@felixgefelixge

HD cameras everywhere

HD Cameras

Hugefile sizes

Example: 45 second video on iPhone 5 = ~100 MB

@felixgefelixge

Slow uplinks

• Wifi: ~2.5min (~5000kbsp)

• LTE: ~10 min (~1300 kbsp)

• 3G: ~40min (at ~330 kbps)

• Edge: ~68 min (at 200 kbps)

Average upload speeds for 100 MB** sources: Apple Network Link Conditioner and http://www.lte-anbieter.info/presse/12/analyse-speed-lte.html

@felixgefelixge

Unreliable Networks

felixge

99.9 / 100 MB upload, 2 seconds remaining

error: connection lost, please upload again!

felixge

resumable file uploading

felixge

possible today, but way too hard

felixge

HTML Uploads

@felixgefelixge

Form-based File Upload in HTML

1 <form2 action="/"3 method="post"4 enctype="multipart/form-data"5 >6 <input name="upload" type="file" accept="image/*">7 <input type="submit" value="Upload File">8 </form>

RFC1867 - Nov 25, 1995

@felixgefelixge

multipart/form-data

POST / HTTP/1.1Host: localhost:8080Content-Type: multipart/form-data; boundary=---------------------------1735847702826983019589436949Content-Length: 2936447

-----------------------------1735847702826983019589436949Content-Disposition: form-data; name="upload"; filename="gopher.png"Content-Type: image/png

PNG

IHDR�7/iCCPICC Profilec``2ptqre``�+)rwR�R`��� ``�\\�yy |��<more data>-----------------------------32473823211195181971638105612--

@felixgefelixge

Missing Features

• Progress indication

• Background Uploads

• Selecting multiple files

• File size limits

• Drag & Drop

• etc.

felixge

@felixgefelixge

@felixgefelixge

HTML5

• File API + XMLHttpRequest2

• Background Uploads, Drag & Drop, Reading file data in JS, Selecting Multiple Files, Upload Progress, etc.

• Available in Chrome, Firefox, Safari, Opera, IE 10

felixge

How to resume a http upload*?

* ideally in a RESTful way

felixge

not defined : /

felixge

<rant>

rfc2616 is a bad place to look for wisdom

@felixgefelixge

GET / HTTP/1.1Host: foo.example.comAccept: text/plain;q=0.1,text/*;q=1,*/*;q=0Range: bytes=8-

@felixgefelixge

GET / HTTP / 1 . 1Accept: text/plain ; q = 0 .1 ,,,, ,, ,,, ,,, ,,, ,,, ,,,,,,,, ,,,, ,, text/* ; q = 1 . 00 Range: bytes = 8 -Host: foo.example.com Accept: , */* ; q = 0 ,

felixge

</rant>

felixge

Prior Art

@felixgefelixge

YouTube API v2.0 – Resumable Uploads

• Based on Google Gears protocol

• Invents new http code 308 Resume Incomplete

• Uses PUT to “query” upload status

• Abuses “Content-Range”, “Range” headers

• Not meant for interoperability

@felixgefelixge

Amazon S3

• Multipart API

• Complicated, hard to implement for clients, hard for servers

• 5 MB chunk size limit

• REST? What’s REST?

• Not meant for interoperability

@felixgefelixge

resumable.js

• Uses multipart/form-data, not RESTful

• Requires lots of small chunk requests (ovhead)

• Poorly specified protocol

• Still: Probably best open source solution out there right now

felixge

the tus resumable upload protocol

felixge

simple, open, freehttps://github.com/tus/

tus-resumable-upload-protocol

v0.2.1 released 3 days go

felixge

POST /files HTTP/1.1Host: tus.example.orgContent-Length: 0Final-Length: 100

HTTP/1.1 201 CreatedLocation: http://tus.example.org/files/1

@felixgefelixge

PATCH /files/1 HTTP/1.1Host: tus.example.orgContent-Length: 100Offset: 0

[file data]

HTTP/1.1 200 Ok

@felixgefelixge

HEAD /files/1 HTTP/1.1Host: tus.example.org

HTTP/1.1 200 OkOffset: 70

@felixgefelixge

PATCH /files/1 HTTP/1.1Host: tus.example.orgContent-Length: 30Offset: 70

[remaining file data]

HTTP/1.1 200 Ok

felixge

clients and servers for all platforms

@felixgefelixge

clients and servers

• tusd - reference server, implemented in Go

• brewtus - node.js server implemented by Naren Venkataraman

• tus-jquery-client, JS client using jQuery

• tus-ios-client, native objective C client library

• tuspy, python client by Naren Venkataraman

http://www.tus.io/implementations.html

felixge

upload acceleration

felixge

checksums, meta data, streams, etc.

felixge

demo

@felixgefelixge

Future

• Release v1.0 of protocol

• Submit as RFC

• Adoption by all majors frameworks, libraries and products

@felixgefelixge

felixge

@felixgefelixge

tus.io

felixge

Questions?