11 Halloween, 2011 Cullen Jennings

2 PeerConnection emits messages like this: “SDP\n v=0\n o= IN IP \n s= \n c=IN IP \n t= \n m=audio RTP/AVP 0”

3 What if we want to add something else? For instance, a message ID Maybe some structure would be nice….

4 { “sdp”:”v=0\n o= IN IP \n s= \n c=IN IP \n t= \n m=audio RTP/AVP 0” } Now at least we can extend things }

5 Easy to get confused if we have glare We both send SDP at the same time Is the SDP I just got a new call or an answer to my offer?

6 { “messageType”: “OFFER”, “sdp”:”v=0\n o= IN IP \n s= \n c=IN IP \n t= \n m=audio RTP/AVP 0” }

7 We might have multiple sessions going on Either simultaneously or in sequence Message delays can make these look like each other

8 { “messageType”: “ANSWER”, "callerSessionId":" ABCDEF", "calleeSessionId":"abc ", “sdp”:”v=0\n o= IN IP \n s= \n c=IN IP \n t= \n m=audio RTP/AVP 0” } Session ID must be globally unique

9 { “messageType”: “OFFER”, "callerSessionId":" ABCDEF", … } { “messageType”: “ANSWER”, "callerSessionId":" ABCDEF", "calleeSessionId":"abc ", … } Each side contributes a session ID Session defined by combination of each side

10 A session has a sequence of offer/answer pairs Example: upgrade to video We have an audio call You decide to add video This requires multiple offer/answer pairs How do we distinguish them?

11 { “messageType”: “OFFER”, "callerSessionId":" ABCDEF", “seq”:1, “sdp”:”v=0\n o= IN IP \n s= \n c=IN IP \n t= \n m=audio RTP/AVP 0” } Sequence indicates the current offer/answer exchange OFFER and ANSWER have same sequence number

12 It’s not safe to have multiple OFFERs outstanding What happens if I do two changes in succession? Example: Other side adds video I accept but then user changes camera to one with different capabilities I need to re-OFFER but when? After he’s gotten my ANSWER

13 { “messageType”: “OK”, “seq”:”2”, "callerSessionId":" ABCDEF", "calleeSessionId":"abc ”, } OK message indicates ANSWER received and accepted New session parameters are active Safe to do a new OFFER/ANSWER pair

14 ICE is slow (can take 3-5 seconds to converge) We want to start ICE as soon as possible Best experience is to start ICE when Bob receives the OFFER even before he accepts the call But we don’t know the media parameters till he answers the call For instance, he might accept audio but not video

15 { “messageType”: “ANSWER”, "callerSessionId":" ABCDEF", "calleeSessionId":"abc ", “moreComing”:true, “sdp”:… // ICE candidates but recvonly media } moreComing flag means that another ANSWER will follow to this OFFER OFFER/ANSWER transaction isn’t complete till moreComing=false No OK for moreComing=true

16 Need to pass in TURN and STUN servers and credentials Often optimal to use more than one Proposal, array of URL Note Current text broken for IPv6 IP addresses Draft has: new PeerConnection( “TURNS :3478”, … Proposal: New PeerConnection( [“turns:// :3478”,”stun://example.com”], …

17 JS Objects JSON strings Other Use ROAP? Move def of application/html-peer-connection-deta to ROAP?

18 Help ! Draft has All SDP media descriptions for streams represented by MediaStream objects must include a label attribute ("a=label:") whose value is the value of the MediaStream object's label attribute. [SDP] [SDPLABEL] Is this label really the same as the SDP “a=label” ? Can’t require that we have this with legacy things? Why do we have this?

19 Proposal Have a stats method on PeerConnection Have it return a dictionary of stats Define an IANA registry of well known stat names The dictionary value for a stat can be primitive type or an array or object

20 Tell CODECs something about the application that they may need to understand to make good encoding choices Audio: is music or is spoken voice Video: prefer spatial or temporal fidelity Proposal IANA registry of well known hints Have some sort of setHints method on a media streams

21 It happens (lets add video … ) Deal with this in IETF ROAP takes care of it

22 Proposal – Focus on the API Send and receive a string/blob Unreliable delivery ? Reliable delivery ? Out of order delivery? Stream of data ?

23 Need to callback when received DTMF and way to tell audio stream to send DTMF Proposal: Add callback and method to send to audio streams Add a way to disable it when creating stream

24 Draft has The relative order of all tracks in a user agent must be stable. All audio tracks must precede all video tracks. Why do we need audio before video? Complicates mapping to SDP which is not this way