OSAF, Chandler, WebDAV, CAP, CalDAV…

Lisa Dusseault

Open Source Application Foundation

Topics

• My background• OSAF in brief• Chandler

– Product Vision– Protocol requirements

• WebDAV introduction– Properties– Distributed Authoring– Advanced features

• CalDAV architecture

My background

• IETF:– Chair of WebDAV, XMPP, IMAPEXT WGs– Participant in CalSch, HTTP, others

• Microsoft Exchange PM– Calendaring standard architect– Content indexing– IRC, instant messaging– WebDAV

• Xythos Director of Development– Massively scalable WebDAV server

• OSAF

OSAF

– Founded in 2001 by Mitch Kapor as non-profit– Currently 20 people (mostly developers)– Volunteer assistance encouraged

Create and gain wide adoption of Open Source application software

of uncompromising quality.

Chandler Vision

• New model for personal information organization– Combine email, tasks, contacts and events– Easier to ‘clear the table’ and focus

• Extensible, customizable– User defines and bookmarks views very easily– User adds important metadata fields– 3rd-party plug-ins or ‘parcels

• Share!

Traditional PIM

Mixed Collections

Chandler Protocol Requirements

ChandlerChandlerserverserver

CalendarCalendar

CAP?Synch?Publish?

EmailEmail

Other Chandler

Sharing

IMAP, POP, iMIP

PDA?

HTTP?Web/fileWeb/fileserverserver

Sharing: Functional Requirements

• Synchronize Calendars– View somebody else’s calendar offline– Add events to shared calendars– Discuss availability

• Share Views– Collections of mixed items– Sorting, columns, other details of view also

• Share/synchronize Contacts, tasks, email• Sharing “circle”

Sharing: Abstract Requirements

• Mutual authentication• Access control / multiple authors• Data browsing• Data synchronization• Search• Notifications?• IM?• Locate peers dynamically?

Existing Standards

Requirement Standards Others

Data browse/synch HTTP/WebDAV, NFS CIFS

Search WebDAV

Access Control WebDAV, NFS CIFS

Locate peers Zeroconf JXTA

Notifications XMPP (Jabber), SIP

Mutual P2P Authentication

XMPP PKI

IM XMPP, SIMPLE

Modular Protocol Composition

Datastore

WebDAVXMPP

SASL

URLs

Directory

LDAPSSL

MIME

Event server

PKI

“Silo” Protocol Design

TCP

event

CAP

calendar

post

newsgrp

NNTP

msg

IMAP

folder

•authenticate•sessions•browse•synch•acl

•authenticate•sessions•browse•synch

•authenticate•sessions•browse•synch•acl

resource

HTTP

directory

•authenticate•sessions•browse•synch•acl

Chandler 1.0 Sharing

• IMAP/POP3/SMTP support for doing mail• Propose HTTP/WebDAV for sharing

– Between Chandler peers and publish to server– Browsing, search and synchronization– To share contacts, tasks, email, calendar data– To share Chandler views (heterogeneous)

• Use other standards for other pieces – Possibly PKI for sharing circle trust relationships– Possibly XMPP for chat, peer relationship, event

notifications

Why OSAF Likes WebDAV

• Solves repository access requirements– Browse, search, synchronize – Multiple authors (ACL, locking, versioning)

• Provides additional benefits– HTTP base provides URLs, libraries, fast development, simple

clients, and extensible protocol

• Clear data model for any application semantics – Properties– Collections and resources– Synchronization

• Proven, deployed, open technology

WebDAV Topics

• Introduction: goals, interoperability, model• HTTP

– Synchronization via ETags

• WebDAV– Properties– Distributed Authoring

• Access Control• DeltaV

WebDAV Goals

• Extend HTTP, preserving semantics– Edit Web pages– Use any authoring tool – Use native URLs for in-place authoring

• Enable applications– Any document-plus-metadata application– Clear extensible data model

• Web File System

Interoperability

WebDAV

Web servers

Document Management

Email

Archives

Publishing• MS Office• Adobe tools

Web Browsers

File Explorers

Site and Content Management

Productivity Applications

File system view

Workflow model

Usage examples

• MS Office/Adobe: authoring• Apache, IIS 5: Web site authoring• Exchange 2000: email/calendaring• Apple iCal• Mac OS/X: publish to Web servers• Xythos WFS: file sharing• Chubb: underwriter repository• Pacific National Labs: chemistry repository• Subversion: source control repository• Excosoft, Interwoven, Vignette, Documentum: Content

management

Data model

http://example.com/hr/

http://example.com/hr/ergnomics/posture.doc

http://example.com/hr/index.html

A collection

http://example.com/hr/ergonomics/ A sub-collection

A resource

HTTP Model

• Data Model: Resources, entities• Communication model: request/response• Address model

– A URL points to a resource

HTTP Features

• Methods– GET, PUT, DELETE– POST– HEAD, TRACE

• ETags• Intermediary support

ETags

• Resources have ETags– Chosen/published by server– Remembered by caches

etag1284467

/folder

doc1.txt

doc1.txt etag1284467etag1284458doc2.txt

Cache

etag8255591

doc2.txt

Need to GET

Synchronization

• HTTP only:

GET <resource-url> HTTP 1.1

If-Match: etag1088ae32• Must remember every resource• Still -- it works

Performance

• HTTP designed for low latency– Size of requests just not an issue

• Pipelining may be important in some applications

WebDAV Focus

• URLs, MOVE, COPY, MKCOL• Properties• Locks• Access Control• DeltaV

How WebDAV affects URLs

• MOVE, COPY• BIND, UNBIND --> see bindings• Collection membership

– Can list children of /specs– Address is determined by parent(s)

/specs

doc1.txt doc2.txt

Properties

• Expose HTTP data in listings– For each resource in collection… etag, last-modified,

etc

• Expose WebDAV information– Is this a collection

• Expose custom data– Invoice #, customer ID…– Picture size, date of photo…– To addresses, from address, date received…

Properties

• PROPFIND– Get properties for a single resource– Get properties for all resources inside a collection

(direct children or recursive)– Specific list of properties or names of all properties

• PROPPATCH– Operates only on single resource– Set property values– Create and delete dead properties

Live vs. Dead properties

• Live properties: server generates, enforces, watches or …– getETag, resourceType

• Dead properties: client has complete control– invoiceID, customerID, description

• Live properties are frequently protected; dead properties all fall under same access control

Examples: Directory listing<multistatus> <response> <href>URL for /specs</href> <propstat>Details of /specs</propstat> </response> <response> <href>URL for doc1</href> <propstat>Details of doc1</propstat> </response> <response> <href>URL for doc2</href> <propstat>Details of doc2</propstat> </response></multistatus>

/specs

doc1.txt doc2.txt

Example: Text Properties in XML

<response xmlns=“DAV:”>

<href>http://example.com/folder/doc.txt</href>

<propstat>

<prop>

<creationdate>1997-12-01T17:42:21-08:00

</creationdate>

<displayname>Proposal for Reorg</displayname>

<getetag>e8829107</getetag>

</prop>

</propstat>

</response>

Example: XML properties in XML

<prop xmlns=“DAV:”>

<m:author xmlns:m=“ietf:iana:schemes:wg:meta”>

<m:firstname>Lisa</m:firstname>

<m:lastname>Dusseault</m:lastname>

</m:author>

</prop>

Synchronization, take 2

• WebDAV:PROPFIND <collection-url> HTTP 1.1

Depth: infinity

<?xml version=“1.0”?>

<propfind xmlns=“DAV:”>

<prop><getetag></prop>

</propfind>

• Easier to get list of changed and new resources

Locks

• Avoid “Lost Update” problem– HTTP ETags help but aren’t sufficient

• Tells who is editing a document• Widely used

– Adobe clients require support

• Optional feature of base WebDAV spec

Lost update problem

PUT file

GET fileHello Allison

Hello Alice

Dear Allison

Change salutation

PUT file

A’s fix is lost!

GET file

Client A Server State Client B

Change name

Lost update with Etags

GET fileHello Allison

Hello Alice

Change salutation

PUT file, check ETag

GET file

B must start over!

Client A Server State Client B

PUT file

GET file

Change name

New ETag assigned

Concurrent edit with locks

LOCK file failsHello Allison

Change salutation

PUT file UNLOCK file

Client A Server State Client B

PUT file

LOCK file

Change name

GET file

UNLOCK fileLOCK file worksGET fileHello

Alice

Dear Alice

Read properties

Write properties

Modify entityRead entity

What locks aren’t

• Locks don’t prevent read operations• Locks don’t create a sandbox• Locks don’t involve a transaction

ACLs

• RFC3744• Three working implementations • Interoperability work in progress

ACLs

• Model– Each resource has an Access Control List (ACL)– Each ACL has n Access Control Entries (ACE)

granting specific permission to specific principal

• Permissions similar to NFS: read, write…• Principals includes users, groups• Compatible with popular file systems

Versioning

• RFC3253, March 2002• DeltaV Working Group

– Lots of IBM input

• Implementations of ‘core’ features proven– Version history– Checkin, checkout

Version history, URLs

File.doc

v1

root-version

http://example.com/file.doc

http://example.com/?v=file.doc&n=1

successor-set

v2 http://example.com/?v=file.doc&n=2

Versioning

• Advanced support for:– Forks, merges– Server Workspaces– Activities: atomic multi-file checkins, multi-file

branches– Baselines– Versioned collections

Calendaring

Calendaring Standards Status

• IETF Working group:– been through 5 chairs since 1996– 8 years of debate over CAP model and design

• Design by committee• Changing editors• Changing names CIP, CTP, CAP…

– Compare to iCalendar and iMIP: focus success

• Inventing many things from scratch– Session control, feature negotiation– Addressing, hierarchical object access, and queries– Access control, other security design

Current CAP problems

• 136 pages and still underspecified• Complex -- both clients and servers• Maintains connection between server and client• No Web addresses defined for calendar items• Data model poorly defined• Offline operation undefined• Not supported by Microsoft, Apple, IBM• Not yet a standard

Data model problems

• Are recurrances first-class items?• Does a hierarchy imply inheritance?• Is free-busy information a roll-up?• Do free-busy ACLs differ from ordinary ACLs?• What class of thing are VCALSTOREs,

VAGENDAs, VQUERYs, VTIMEZONEs, VEVENTs, etc.– Which are the same kinds of things?– Which are different?

Chandler problems with CAP

• Server centric– What functions can the client perform? Chandler

wants control– Hard to make peer-to-peer operation consistent

• Connection-oriented– Makes servers less scalable– Client must keep open yet another connection– Search instead of Synch

• Poor performance– Many roundtrips, e.g. GENERATE-UID

Chandler problems with CAP, cont…

• Data access and scheduling mixed – Alexander Taler’s mail of 24 May 1999

• But scheduling model doesn’t help multiple clients work well together

• What if clients want to use XMPP to instantly schedule

Chandler problems with CAP, cont…

• Poor extensibility (IMAP model)– Chandler wants to annotate and share much more– How to put a picture like a map along with an event?

• No leverage– Compare to leverage of WebDAV– BEEP…

• More work– No CAP client libraries exist today (python, other)– IMAP-like model harder for clients: more required

features

Too many features

• Don’t need searching if using synch• Don’t want scheduling but server MUST support

– Don’t want to accept alarms in scheduling requests

• Don’t want attendees to set own status – At least not directly

Requirements for Calendar Server Access Protocol

• Authenticate to server• Access control for multiple authors• Data browsing

– Support for vCalendar data format, including free-busy

• Data synchronization• Search• Invitations, fan-out?

– Or is this a client function

• Notifications– Reminders, incoming invitations

Where to layer

BEEP

CAP orWebDAV

CalDAV

WebDAV -- Application neutral

text img vCard vCal

WebDAV

SSL/TLS

TCP

Data formats

Data access

Data privacy

Transport

Extend classic protocol layering

CalDAV overview

• Data model– Calendars are WebDAV collectionshttp://example.com/users/lisa/calendars/karate

– Events are HTTP (WebDAV) resources

http:.../calendars/karate/seminar.ics

• HTTP URLs extremely portable• Full HTTP backward compatibility

– ETags, last modified date, caching

• Full WebDAV backward compatibility

File formats

• HTTP GET should return something useful• GET <event-url>

– Returns iCalendar document?– Returns xCalendar?– Negotiation is a possibility

• GET <calendar-url>– Possible roll-up into large iCalendar file– Possible entry-point into WebUI– Note WebDAV doesn’t define GET of collections

CalDAV and WebDAV Properties

• Any implementor can add custom properties without conflict

• XML property format – conducive to WebUI implementations

• Backward compatibility easier– Clients only ask for properties they know how to

display– IMAP model: very difficult to achieve backward

compatibility

Specific properties

<response xmlns:c=“ietf:wg:calsch”> <href>http://example.com/lisa/event1.ics</href> <propstat> <prop> <c:dtstart>2004-05-01T10:00-08:00</c:dtstart> <c:dtend>2004-05-01T11:00-08:00</c:dtstart> <c:transp><c:OPAQUE-NOCONFLICT/></c:transp> <c:summary>Important Meeting</c:summary> </prop> <status>HTTP/1.1 200 OK</status> </propstat></response>

CalDAV and Locks

• Optional feature• Definitely useful for shared calendars• Also useful for multiple clients

– PDA and laptop

CalDAV and WebDAV ACLs

• Access control could be optional– A simple system can make calendars readable only

by owners– A slightly more complex system can make events

either private (only owner) or public)– A high value-add server can have powerful access

control, e.g. shared calendars

CalDAV and DeltaV

• Versioning?– Definitely optional– High value-add feature– Useful to see who changed an event

Support for CalDAV

• Apple ‘iCal’ – Publish vCalendar document via WebDAV

• Mozilla– Uses Apple model, supports CalDAV

• MS Exchange - more structured, granular – Browse calendars via WebDAV– Use WebDAV properties for appointment information– Not interoperable with Apple, Mozilla

• Cyrusoft reviewing CalDAV draft

Summary

• Chandler is ambitious, open, sharing-oriented– Combines multiple data types in shared views

• WebDAV is common data access protocol– Supports versioning, search, access control, locking,

addressing -- all independent of data formats

• WebDAV + iCalendar = CalDAV– Internet-Draft (work in progress)– Working on next version

Bonus Slides

iMIP, vCalendar and Calendar Servers

AttendeeSchedulingClient

Email1Email1

CalendarAccess

IMAPiMIP

vCalvCal

Cal1Cal1

1

Email2Email2

SMTP

2

POP

3

Cal2Cal2

CalendarAccess

vCal 4

Modular Protocol Composition

DataStore MIME

WebDAV

XMPP

URLs

SASL

URLs

SSL

events

data

Direc-tory LDAP

Permissions

• All– Read– Write

• Bind, unbind• Write-content, write-properties

– Administer• Read-acl, write-acl• Unlock (destroy lock)• Read-current-user-privilege-set

Inheritance

• Each resource has ‘inherited-acl-set’ property– Can declare which other resource(s) it gets ACLs

from– On some systems, a user with write-acl privilege

could set this property to change inheritance

Principals

• Browsable collections of principals• Each principal is a resource

– Has URL for identification in ACLshttp://example.com/principals/users/AliceW

/principals

/users /groups

AliceW BobR sales dev

Principals and Directories

WebDAV LDAP?

transformResult: simpler client

Implementation-specificStandard