OSAF, Chandler, WebDAV, CAP, CalDAV… Lisa Dusseault Open Source Application Foundation

Topics My background OSAF in brief Chandler WebDAV introduction Product Vision Protocol requirements WebDAV introduction Properties Distributed Authoring Advanced features CalDAV architecture

My background IETF: Microsoft Exchange PM Chair of WebDAV, XMPP, IMAPEXT WGs Participant in CalSch, HTTP, others Microsoft Exchange PM Calendaring standard architect Content indexing IRC, instant messaging WebDAV Xythos Director of Development Massively scalable WebDAV server OSAF

OSAF Create and gain wide adoption of Open Source application software of uncompromising quality. Founded in 2001 by Mitch Kapor as non-profit Currently 20 people (mostly developers) Volunteer assistance encouraged

Chandler Vision New model for personal information organization Combine email, tasks, contacts and events Easier to ‘clear the table’ and focus Extensible, customizable User defines and bookmarks views very easily User adds important metadata fields 3rd-party plug-ins or ‘parcels Share!

Traditional PIM

Mixed Collections

Chandler Protocol Requirements Chandler server Calendar CAP? Synch? Publish? Email Other Chandler Sharing IMAP, POP, iMIP PDA ? HTTP? Web/file server Explain that the Chandler--Chandler connection involves several things: peer to peer location, authorization, repository access, invitations and notifications

Sharing: Functional Requirements Synchronize Calendars View somebody else’s calendar offline Add events to shared calendars Discuss availability Share Views Collections of mixed items Sorting, columns, other details of view also Share/synchronize Contacts, tasks, email Sharing “circle”

Sharing: Abstract Requirements Mutual authentication Access control / multiple authors Data browsing Data synchronization Search Notifications? IM? Locate peers dynamically?

Existing Standards Requirement Standards Others Data browse/synch HTTP/WebDAV, NFS CIFS Search WebDAV Access Control WebDAV, NFS Locate peers Zeroconf JXTA Notifications XMPP (Jabber), SIP Mutual P2P Authentication XMPP PKI IM XMPP, SIMPLE We work hard to identify existing standards, because solutions may already exist *and* help us plug into existing environments more easily. There can be existing code, libraries, administrative support, etc. I’ll get back to why CAP isn’t actually a standard yet.

Modular Protocol Composition Event server URLs MIME Data store XMPP WebDAV SSL PKI SASL LDAP A similar example is how SIMPLE is using SSL and URLs and MIME explicitly; LDAP can implicitly be used, and WebDAV is showing up for accessing stored items such as buddy lists and personal information. Directory

“Silo” Protocol Design IMAP NNTP CAP HTTP authenticate sessions browse synch acl authenticate sessions browse synch authenticate sessions browse synch acl authenticate sessions browse synch acl folder newsgrp calendar directory msg post event resource With each new application, we go through very time-consuming process to identify the threat model, the session model, the data model… there are internet-drafts out there today explaining how to do SASL authentication for NNTP, because it wasn’t designed when these modules were available. TCP

Chandler 1.0 Sharing IMAP/POP3/SMTP support for doing mail Propose HTTP/WebDAV for sharing Between Chandler peers and publish to server Browsing, search and synchronization To share contacts, tasks, email, calendar data To share Chandler views (heterogeneous) Use other standards for other pieces Possibly PKI for sharing circle trust relationships Possibly XMPP for chat, peer relationship, event notifications HTTP/WebDAV requirements: we expect people to eventually plug Chandler into their authoring or content management environments. But even before that, if Westwood provides calendar server functionality it will need to have a WebUI for kiosk access.

Why OSAF Likes WebDAV Solves repository access requirements Browse, search, synchronize Multiple authors (ACL, locking, versioning) Provides additional benefits HTTP base provides URLs, libraries, fast development, simple clients, and extensible protocol Clear data model for any application semantics Properties Collections and resources Synchronization Proven, deployed, open technology A cell phone is an example of an HTTP client that can browse URLs and download events so a user on the road can see their day. There’s multi-vendor interoperable support -

WebDAV Topics Introduction: goals, interoperability, model HTTP WebDAV Synchronization via ETags WebDAV Properties Distributed Authoring Access Control DeltaV

WebDAV Goals Extend HTTP, preserving semantics Enable applications Edit Web pages Use any authoring tool Use native URLs for in-place authoring Enable applications Any document-plus-metadata application Clear extensible data model Web File System

Site and Content Management Interoperability Web Browsers Web servers Productivity Applications Archives MS Office Adobe tools WebDAV Publishing Document Management File Explorers Site and Content Management Email

File system view

Workflow model

Usage examples MS Office/Adobe: authoring Apache, IIS 5: Web site authoring Exchange 2000: email/calendaring Apple iCal Mac OS/X: publish to Web servers Xythos WFS: file sharing Chubb: underwriter repository Pacific National Labs: chemistry repository Subversion: source control repository Excosoft, Interwoven, Vignette, Documentum: Content management

Data model A collection http://example.com/hr/ A resource http://example.com/hr/index.html A sub-collection http://example.com/hr/ergonomics/ http://example.com/hr/ergnomics/posture.doc

HTTP Model Data Model: Resources, entities Communication model: request/response Address model A URL points to a resource The purpose or function of a URN is to provide a globally unique, persistent identifier used for recognition, for access to characteristics of the resource or for access to the resource itself. A URN is a name with global scope which does not imply a location. It is intended that the lifetime of a URN be permanent.

HTTP Features Methods ETags Intermediary support GET, PUT, DELETE POST HEAD, TRACE ETags Intermediary support

ETags Resources have ETags Chosen/published by server Remembered by caches /folder Cache doc1.txt etag1284467 doc2.txt etag1284458 doc1.txt doc2.txt etag1284467 etag8255591 Need to GET

Synchronization HTTP only: GET <resource-url> HTTP 1.1 If-Match: etag1088ae32 Must remember every resource Still -- it works

Performance HTTP designed for low latency Size of requests just not an issue Pipelining may be important in some applications

WebDAV Focus URLs, MOVE, COPY, MKCOL Properties Locks Access Control DeltaV

How WebDAV affects URLs MOVE, COPY BIND, UNBIND --> see bindings Collection membership Can list children of /specs Address is determined by parent(s) /specs doc1.txt doc2.txt

Properties Expose HTTP data in listings Expose WebDAV information For each resource in collection… etag, last-modified, etc Expose WebDAV information Is this a collection Expose custom data Invoice #, customer ID… Picture size, date of photo… To addresses, from address, date received…

Properties PROPFIND PROPPATCH Get properties for a single resource Get properties for all resources inside a collection (direct children or recursive) Specific list of properties or names of all properties PROPPATCH Operates only on single resource Set property values Create and delete dead properties

Live vs. Dead properties Live properties: server generates, enforces, watches or … getETag, resourceType Dead properties: client has complete control invoiceID, customerID, description Live properties are frequently protected; dead properties all fall under same access control

Examples: Directory listing <multistatus> <response> <href>URL for /specs</href> <propstat>Details of /specs</propstat> </response> <href>URL for doc1</href> <propstat>Details of doc1</propstat> <href>URL for doc2</href> <propstat>Details of doc2</propstat> </multistatus> /specs doc1.txt doc2.txt

Example: Text Properties in XML <response xmlns=“DAV:”> <href>http://example.com/folder/doc.txt</href> <propstat> <prop> <creationdate>1997-12-01T17:42:21-08:00 </creationdate> <displayname>Proposal for Reorg</displayname> <getetag>e8829107</getetag> </prop> </propstat> </response>

Example: XML properties in XML <prop xmlns=“DAV:”> <m:author xmlns:m=“ietf:iana:schemes:wg:meta”> <m:firstname>Lisa</m:firstname> <m:lastname>Dusseault</m:lastname> </m:author> </prop>

Synchronization, take 2 WebDAV: PROPFIND <collection-url> HTTP 1.1 Depth: infinity <?xml version=“1.0”?> <propfind xmlns=“DAV:”> <prop><getetag></prop> </propfind> Easier to get list of changed and new resources

Locks Avoid “Lost Update” problem Tells who is editing a document HTTP ETags help but aren’t sufficient Tells who is editing a document Widely used Adobe clients require support Optional feature of base WebDAV spec

Lost update problem PUT file GET file Hello Allison Hello Alice Dear Allison Change salutation A’s fix is lost! Client A Server State Client B Change name

Lost update with Etags Client A Server State Client B GET file Change name Hello Allison Change salutation PUT file New ETag assigned PUT file, check ETag Hello Alice B must start over! GET file

Concurrent edit with locks Client A Server State Client B LOCK file GET file Hello Allison LOCK file fails Change name PUT file UNLOCK file LOCK file works Hello Alice GET file Change salutation PUT file UNLOCK file Dear Alice

What locks aren’t Locks don’t prevent read operations Locks don’t create a sandbox Locks don’t involve a transaction Read properties Write properties Read entity Modify entity

ACLs RFC3744 Three working implementations Interoperability work in progress

ACLs Model Permissions similar to NFS: read, write… Each resource has an Access Control List (ACL) Each ACL has n Access Control Entries (ACE) granting specific permission to specific principal Permissions similar to NFS: read, write… Principals includes users, groups Compatible with popular file systems

Versioning RFC3253, March 2002 DeltaV Working Group Lots of IBM input Implementations of ‘core’ features proven Version history Checkin, checkout

Version history, URLs File.doc http://example.com/file.doc v1 v2 root-version v1 http://example.com/?v=file.doc&n=1 successor-set v2 http://example.com/?v=file.doc&n=2

Versioning Advanced support for: Forks, merges Server Workspaces Activities: atomic multi-file checkins, multi-file branches Baselines Versioned collections

Calendaring

Calendaring Standards Status IETF Working group: been through 5 chairs since 1996 8 years of debate over CAP model and design Design by committee Changing editors Changing names CIP, CTP, CAP… Compare to iCalendar and iMIP: focus  success Inventing many things from scratch Session control, feature negotiation Addressing, hierarchical object access, and queries Access control, other security design vCalendar was a success because it had a narrow focus, a specific vision, a small set of contributors, a readable specification, and a clear interoperability need. Not to mention support by major vendors. None of these is true for CAP. Also note the personnel problems in CalSch -- tiny group now has lost most of its original members, fails to attract new, terrible mailing list participation including vicious arguments. Finally, it’s quite probably that CAP will become a RFC anyway -- most human groups don’t like to admit failure. That doesn’t mean it will become the interoperability standard.

Current CAP problems 136 pages and still underspecified Complex -- both clients and servers Maintains connection between server and client No Web addresses defined for calendar items Data model poorly defined Offline operation undefined Not supported by Microsoft, Apple, IBM Not yet a standard -- Complex clients and servers: It’s bad enough to define a standard that requires a lot of programming to make a server, but it’s even worse when you force the client to be complex and difficult. Ideally, clients should be able to pick and choose what functions to use to accomplish goals in the simplest way possible. That’s not the CAP model. -- Maintining a connection: The connection problem is one that will really hurt CAP adoption by low-cost service providers. Maintaining a connection hurts scalability. E.g. Hotmail provides HTTP and WebDAV access to email but not IMAP because it can’t handle enough clients. -- Data model poorly defined: this may not seem to be a problem, but it is. It makes it very hard for readers to wrap their heads around the specification and to implement it correctly. It makes it even harder to extend CAP.

Data model problems Are recurrances first-class items? Does a hierarchy imply inheritance? Is free-busy information a roll-up? Do free-busy ACLs differ from ordinary ACLs? What class of thing are VCALSTOREs, VAGENDAs, VQUERYs, VTIMEZONEs, VEVENTs, etc. Which are the same kinds of things? Which are different?

Chandler problems with CAP Server centric What functions can the client perform? Chandler wants control Hard to make peer-to-peer operation consistent Connection-oriented Makes servers less scalable Client must keep open yet another connection Search instead of Synch Poor performance Many roundtrips, e.g. GENERATE-UID

Chandler problems with CAP, cont… Data access and scheduling mixed Alexander Taler’s mail of 24 May 1999 But scheduling model doesn’t help multiple clients work well together What if clients want to use XMPP to instantly schedule

Chandler problems with CAP, cont… Poor extensibility (IMAP model) Chandler wants to annotate and share much more How to put a picture like a map along with an event? No leverage Compare to leverage of WebDAV BEEP… More work No CAP client libraries exist today (python, other) IMAP-like model harder for clients: more required features

Too many features Don’t need searching if using synch Don’t want scheduling but server MUST support Don’t want to accept alarms in scheduling requests Don’t want attendees to set own status At least not directly

Requirements for Calendar Server Access Protocol Authenticate to server Access control for multiple authors Data browsing Support for vCalendar data format, including free-busy Data synchronization Search Invitations, fan-out? Or is this a client function Notifications Reminders, incoming invitations

Where to layer CalDAV CAP or WebDAV BEEP If you know me, I’m obsessed with layering…

WebDAV -- Application neutral text img vCard vCal Data formats WebDAV Data access SSL/TLS Data privacy TCP Transport Extend classic protocol layering

CalDAV overview Data model HTTP URLs extremely portable Calendars are WebDAV collections http://example.com/users/lisa/calendars/karate Events are HTTP (WebDAV) resources http:.../calendars/karate/seminar.ics HTTP URLs extremely portable Full HTTP backward compatibility ETags, last modified date, caching Full WebDAV backward compatibility

File formats HTTP GET should return something useful GET <event-url> Returns iCalendar document? Returns xCalendar? Negotiation is a possibility GET <calendar-url> Possible roll-up into large iCalendar file Possible entry-point into WebUI Note WebDAV doesn’t define GET of collections

CalDAV and WebDAV Properties Any implementor can add custom properties without conflict XML property format conducive to WebUI implementations Backward compatibility easier Clients only ask for properties they know how to display IMAP model: very difficult to achieve backward compatibility

Specific properties <response xmlns:c=“ietf:wg:calsch”> <href>http://example.com/lisa/event1.ics</href> <propstat> <prop> <c:dtstart>2004-05-01T10:00-08:00</c:dtstart> <c:dtend>2004-05-01T11:00-08:00</c:dtstart> <c:transp><c:OPAQUE-NOCONFLICT/></c:transp> <c:summary>Important Meeting</c:summary> </prop> <status>HTTP/1.1 200 OK</status> </propstat> </response>

CalDAV and Locks Optional feature Definitely useful for shared calendars Also useful for multiple clients PDA and laptop

CalDAV and WebDAV ACLs Access control could be optional A simple system can make calendars readable only by owners A slightly more complex system can make events either private (only owner) or public) A high value-add server can have powerful access control, e.g. shared calendars

CalDAV and DeltaV Versioning? Definitely optional High value-add feature Useful to see who changed an event

Support for CalDAV Apple ‘iCal’ Mozilla Publish vCalendar document via WebDAV Mozilla Uses Apple model, supports CalDAV MS Exchange - more structured, granular Browse calendars via WebDAV Use WebDAV properties for appointment information Not interoperable with Apple, Mozilla Cyrusoft reviewing CalDAV draft

Summary Chandler is ambitious, open, sharing-oriented Combines multiple data types in shared views WebDAV is common data access protocol Supports versioning, search, access control, locking, addressing -- all independent of data formats WebDAV + iCalendar = CalDAV Internet-Draft (work in progress) Working on next version

Bonus Slides

iMIP, vCalendar and Calendar Servers Email1 Calendar Access IMAP iMIP vCal Cal1 1 Email2 SMTP 2 Cal2 Calendar Access vCal 4 POP 3 iMIP is RFC2447 -- iMIP is a specific instance of iTIP Scheduling Client Attendee

Modular Protocol Composition SSL URLs WebDAV data Data Store MIME events XMPP URLs Direc -tory LDAP SASL

Permissions All Read Write Administer Bind, unbind Write-content, write-properties Administer Read-acl, write-acl Unlock (destroy lock) Read-current-user-privilege-set

Inheritance Each resource has ‘inherited-acl-set’ property Can declare which other resource(s) it gets ACLs from On some systems, a user with write-acl privilege could set this property to change inheritance

Principals Browsable collections of principals Each principal is a resource Has URL for identification in ACLs http://example.com/principals/users/AliceW /principals /groups /users sales dev AliceW BobR

Principals and Directories LDAP? WebDAV transform Result: simpler client Standard Implementation-specific