ORTC Walk Through IETF 88 Robin Raymond, Hookflash Bernard Aboba, Microsoft Adalberto Foresti, Microsoft Open Technologies, Inc. ORCA – Object-RTC API Community Group

Meeting Purpose The purpose of this meeting is explain: – Why a community has come together to work on an alternative API – High level features of the ORTC API – How to use the API to implement a simple video chat demo – How you can get involved with developing the spec and code – Introduce open source effort / call to action There will be time for refreshments and Q&A the last 30 minutes Session is recorded and streamed to the web (sponsored by Hookflash)

Introduction to ORTC ORTC – Object Realtime Communications – A next-generation JavaScript API for browser, mobile, and cloud realtime audio / video scenarios – ORTC is an alternative API to the W3C WebRTC 1.0 draft API – It is being developed in the ORCA W3C Community Group

The ORCA W3C Community Group W3C Community Groups are forums open to all without fee – Simple IPR policy – royalty-free commitments to one’s own contributions – Not on standards track – Mature CG specs can get broader IPR commitments, transition to standards track – Don’t have to be W3C member!

The ORCA W3C Community Group Object RTC API Community Group, – Formed in August 2013 Chair and editor: Robin Raymond – Goal to develop WebRTC API that exposes an object API rather than blobs of SDP – Community group founded by JS developers coming from both SIP/other protocol signaling backgrounds, node.js and WebRTC enthusiasts.

This is not a community group meeting Goal here is to explain mission and progress, recruit new members To suggest API changes, join the Community Group – we can’t accept suggestions from those who have not signed the Contributor License Agreement

Special Thanks to… ORCA Community Group asked Microsoft Open Technologies if they would implement ORTC as demo / validation of concept. Special thanks for Microsoft Open Technologies, Inc. for implementing demo / validation of concept. (and the refreshments!)

WebRTC Needs Mobile: SDKs enabling development of mobile apps protocol- compatible with WebRTC. Analytics. Diagnostics and monitoring for carrier scale operation. Cloud. Easy setup and management of WebRTC systems. – Tools. Enabling connection of WebRTC clients to legacy infrastructure (e.g. node.js) – APIs. Enabling management of STUN/TURN credentials, audio/video mixers, SIP proxies, XMPP connection managers, etc. Extensibility: Enabling addition of functionality beyond the core (e.g. mobile-friendly video codecs such as H.264/SVC)). A next generation JavaScript API compatible with the above.

What are ORTC potential uses? Browser code (simplified RTC control) Mobile development via “ortclib” SDK under development Server code via Node.js new services or legacy inter-op solutions – call to contribute under development Open Source projects are under way!

Write ORTC apps using existing WebRTC 1.0 Browsers An WebRTC 1.0 “shim” will be made available to maintain backwards compatibility If you like the WebRTC 1.0 API, you can still use the 1.0 API!

WebRTC 1.0 (SDP) SDP / SIP (shim) 1.0 Strategy Node.JS Browsers Chrome / Firefox / etc ORTC (shim) Node.JS Web App Browsers (see demo) Browsers (see demo) ORTC 2.0 Future ? ORTC Migration Strategy Web App Mobile (ortclib) Mobile (ortclib) Mobile

Isn’t ORTC just CU-RTCWeb? No Both are object models but at different levels While CU-RTCWeb inspired ORTC, they are are different. Issues with CU-RTCWeb: -too powerful/difficult to use -exposing too much socket level control (i.e. security risks) -couldn’t shim CU-RTCWeb

Where will ORTC be supported by vendors? Support is via the shim over WebRTC 1.0 for now for browsers. Direct ORTC API source code is being developed. Microsoft Open Tech developed a shim plugin for IE (but not a product announcement).

openpeer/ortc GitHub Specification Repo Original home of the ORTC API discussion Spec discussion spun off to CG, OSS code for ORTC stays in GitHub Why do we need the GitHub repo? – Pull requests – discussed on public-orca mailing list

openpeer/ortclib GitHub Mobility Repo ORTC mobile implementation Maintained by Hookflash Exposes C++ WebRTC library as an ORTC API To be wrapped in iOS, Java (for Android) and JavaScript hybrid

openpeer/ortc-node GitHub Node.JS Repo ORTC Node.JS implementation Call to action – seeking contributors – What should implementation look like? – What are native code integration points?

Will contributing to ortc break protocol compatibility with web browser? No Wire level compatibility is maintained Maintaining wire compatibility is a tenet of ORTC implementations with WebRTC enabled browsers* We encourage expansion of features, not removal of basic wire support. * as much as they are compatible with each other

Motivation for ORTC Design

Traditional Browser Rendering Engine Description Format (HTML/CSS) Description Format (HTML/CSS) Resource Render Engine Output JS

JS asks browser “Give me webpage that meets this criteria” 1 1 Modify browser provided HTML 3 3 Set HTML 4 4 Browser generates some HTML (gives to JavaScript) 2 2 This is not how we program HTML on the web…

WebRTC 1.0 Capture / Render Engine Description Format (SDP) Description Format (SDP) Resource Capture/Render Engine Input/Output JS Description Format (SDP) Description Format (SDP) Resource Capture/Render Engine Input/Output JS { Cloud }

ORTC Capture / Render Engine Capture/Render Engine Input/Output Capture/Render Engine Input/Output { Cloud } JS

ORTC Simple Use Case RTCConnection { cloud } Application Browser RTCConnection Application Browser RTP / DATA / DTLS / ICE signaling

Questions?

ORTC API

ORTC Simple Use Case RTCConnection { cloud } Application Browser RTCConnection Application Browser RTP / DATA / DTLS / ICE signaling

MediaStream MediaStreamTrack RTCTrack RTCStream ORTC - Object Relationships RTCConnection RTCConnectionOptions RTCConnectionInfo RTCIceConnectionInfo RTCDtlsConnectionInfo RTCIceCandidate RTCSocket RTCStream MediaStream RTCTrack MediaStreamTrack RTCIceCandidate local remote RTCConnectionInfo RTCIceConnectionInfo RTCDtlsConnectionInfo remote local

RTCConnection [Constructor(RTCConnectionOptions options)] interface RTCConnection : EventTarget { readonly attribute RTCSocket socket; readonly attribute RTCConnectionState state; attribute RTCConnectionRole role; readonly attribute RTCConnectionInfo local; attribute RTCConnectionInfo remote; void addRemoteCandidate(RTCCandidateDescription candidate); void connect(); void gather(); (RTCSTream? or RTCTrack?) send((MediaStream or MediaStreamTrack or RTCTrack or RTCStream) streamOrTrack); (RTCStream? or RTCTrack?) receive((RTCStream or RTCTrack) streamOrTrack); sequence sendStreams(); sequence sendTracks(); sequence receiveStreams(); sequence receiveTracks(); void close (); attribute EventHandler oncandidate; attribute EventHandler onendofcandidates; attribute EventHandler onactivecandidate; attribute EventHandler onstatechanged; attribute EventHandler onnetworkchange; attribute EventHandler onunknowntrack; };

RTCConnection (p1) -RTCSocket (attached to one network socket) -States -“new” – connection not started -“searching” – searching for connectivity -“halted” – no connectivity found -“connected” – found connectivity (ready) -“closed” – connection is finished (one-time-use)

RTCConnection (p2) -role (“controlling” vs “controlled”) -Local and remote information -ICE credentials (usernameFrag / password) -DTLS fingerprint (generated or discovered) -ICE Candidates – local and remote (for connectivity checks)

RTCConnection (p3) -Gathers local ICE candidates -Performs ICE connectivity checks -Causes sending / receiving of media tracks RTP data over the “nominated” ICE connection

RTCConnection (p4) RTCStreams and RTCTracks (sending or receiving) -Contains mapping of on-the-wire RTC/RTP/RTCP/etc information to media streams / media stream tracks) -Contains respective “MediaStream” or “MediaStreamTrack” objects.

RTCConnection “connect” Set remote “RTCIceConnectionInfo” ICE “usernameFrag” and “password” Call “connect” to cause candidate search Will causes state changes from “new”, “searching”, “haulted”, “connected”

RTCConnection “gather” Find new candidates - Restarts network topology ICE candidate gathering Does not affect connection state by itself Start gathering candidates before “connect”

RTCConnection “send” Send “MediaStream”, “MediaStreamTrack”, “RTCStream”, “RTCTrack” Convenience to auto-map “MediaStream” and “MediaStreamTrack” into RTP layer constructs Manipulate RTP mapping by first creating “RTPStream” / “RTCTrack” from “MediaStream” and “MediaStreamTrack”

RTCConnection “receive” Send “RTCStream”, “RTCTrack” Setup mapping for “MediaStream” and “MediaStreamTrack” for RTP layer constructs Manipulate mapping by constructing “RTPStream” / “RTCTrack” objects

RTCConnection “close” Shuts down RTCConnection object Object goes to “closed” state

RTCConnection events oncandidateFires for local candidate (can be cancelled) onendofcandidatesFires when all local candidates are discovered onactivecandidateFires when local:remote candidate connection changes onstatechangedFires as connection state changes onnetworkchangeFires when network topology change has been detected (auto “gather” called but can be cancelled) onunknowntrackFires when track discovered which has no mapping

RTCSocket interface RTCSocket { };

RTCSocket -No exposed attributes, members or events -Represents one physical local socket (as well as virtualized paired firewall/relay socket) -Re-use the same RTCSocket in another RTCConnection to send over same physical local socket

RTCStream [Constructor, Constructor(MediaStream)] interface RTCTrack { attribute MediaStream source; attribute DOMString msid; sequence tracks(); void start(); void stop(); void remove(); };

RTCStream -References single MediaStream -Collection of RTCTracks -Change collection then call RTCConnection “send”/”receive” again -Simplifies MediaStream usage in RTCConnection -Convenience “start” / “stop” methods -“remove” method to remove from RTCConnection

RTCTrack [Constructor, Constructor(MediaStreamTrack)] interface RTCTrack { attribute MediaStreamTrack source; attribute DOMString id; attribute DOMString kind; attribute unsigned long ssrc; attribute sequence codecs; attribute sequence mediaAttributes; attribute Object rtpExtHeaders; void start(); void stop(); void remove(); };

RTCTrack -References single MediaStreamTrack -Mapping of RTP/RTCP/etc wire information such as: -ssrc -codecs -media attributes (e.g. max width / height) -Change properties then call RTCConnection “send” / “receive” again -Convenience “start” / “stop” methods -“remove” method to remove from RTCConnection

RTCConnectionInfo dictionary RTCConnectionInfo { attribute RTCIceConnectionInfo ice; readonly attribute RTCDtlsConnectionInfo dtls; }; RTCIceConnectionInfo RTCDtlsConnectionInfo / CertificateFingerprint dictionary RTCIceConnectionInfo { DOMString usernameFrag; DOMString password; }; dictionary RTCDtlsConnectionInfo { attribute CertificateFingerprint fingerprint; }; dictionary CertificateFingerprint { getter ArrayBuffer (DOMString hashFunction); };

RTCIceCandidateInfo dictionary RTCIceCandidateInfo { DOMString foundation; int component; DOMString transport; int priority; DOMString connectionAddress; int connectionPort; DOMString type; DOMString relAddress; int relPort; };

Object Realtime Communications API (ORTC) Next generation WebRTC API, under development in the W3C ORCA community group: – – Rationale: webrtc-js-obj-api-rationalehttp://tools.ietf.org/html/draft-raymond-rtcweb- webrtc-js-obj-api-rationale Specification: – Github repository: – ORTC API drafts and shim: – ortclib: – Demos:

To Learn More ORCA Community Group: Demo and discussion tomorrow to follow…