CLUE Framework Issues CLUE virtual interim meeting Jan 27, 2014 Mark Duckworth draft-ietf-clue-framework-13 1

Spatial info within composed MCC The MCC addition to the framework added some text about describing layout within a composition. Ticket #5 already closed, saying we don’t want to address describing layout within a composition. Design team discussion on Jan 14 agreed. Is there consensus that the framework should say the consumer should not assume anything about video layout within a composed MCC, and an advertisement does not provide this information? 2

MCC Referencing Other Captures Framework already says MCC can refer to other captures to be included in the MCC, for both advertisement and configure messages Proposal – update framework to clarify that framework examples are conceptual, and not necessarily using the syntax from the data model Continue the discussion about “MCC grouping labels” and regular expressions for captureIDs in the context of the data model Does the group agree? 3

MCC Example clean up discussed on list, example section part of broader multipoint switching discussion Example needs some cleanup – best way could depend on results of broader multipoint switching discussion and issues. 4

Switched Capture Example Topo-Mixer – media switching variety – see draft-ietf-clue-rtp-mapping and draft-ietf-avtcore-rtp-topologies-update Mixer (middle box) provides conceptual sources (Media Captures), selecting one source at a time from the original sources 5

Example Video Layout This endpoint is receiving 9 video captures. MCC1, MCC2, MCC3 has the current talker MCC4 – MCC9 has other recent talkers MCC1MCC2MCC3 MCC4 MCC5 MCC6 MCC7 MCC8 MCC9 Blue person starts talking, mixer switches video London Paris 6 MCC1MCC2MCC3 MCC4 MCC5 MCC6 MCC7 MCC8 MCC9

Switching Mixer Diagram Mixer sends 9 Video Captures to A Mixer selects which sources to send – based on its own policy – or based on explicit requests from the consumer Mixer A B D C E RTP Rewrite F G

Different Approaches 1.Provider (mixer) makes switching decisions, mixer's advertisement has one big capture scene with many MCCs, plus other information about the original source scenes and captures. 2.Same as above, but Consumer (terminal) makes switching decisions (group seems not interested in this, so won’t discuss today). 3.Provider (mixer) makes switching decisions, mixer's advertisement has several smaller scenes with spatial information. 8

Approach 1 – Advertisement to A Scene1: – CSE1(video): MCC1(VC1,VC4,VC6,VC9,VC10,VC11) – MCC2(VC2,VC5,VC7,VC9,VC10,VC11) – MCC3(VC3,VC8,VC9,VC10,VC11) – MCC4, … MCC9 – CSE2(audio):MCC20(), MCC21(), MCC22() (three dominant talkers, each spatially covering whole scene?) – if Adv has these VCs for each MCC, it might as well use Approach 3 with separate scenes – MCCs can have spatial attributes – Can this really work? Scene2 (B) CSE1: VC1, VC2, VC3 Scene3 (C)CSE1: VC4, VC5 Scene4 (D)CSE1: VC6, VC7, VC8 Scene5 (E)CSE1: VC9 Scene6 (F)CSE1: VC10 Scene7 (G)CSE1: VC11 Consumer picks number of captures it wants to receive 9

Approach 2 – Advertisement to A Advertisement could be the same as Approach 1, or it could be more flexible and allow any VC in each MCC Scene1: – CSE1: MCC1( ) – MCC2( ) – MCC3( ) – MCC4,MCC5,MCC6,MCC7,MCC8,MCC9 – MCCs don’t have spatial attributes Consumer sends Configure messages to select which VC it wants in each MCC, and sends a new message every time it wants a change 10

Approach 3 – Advertisement to A Each scene represents another endpoint, or small group of endpoints Capture Scene 1 (current talker, higher priority): – CSE1: MCC1(VC1,VC4,VC6,VC9,VC10,VC11) – left, soundlevel:0 – MCC2(VC2,VC5,VC7,VC9,VC10,VC11) – center, soundlevel:0 – MCC3(VC3,VC8,VC9,VC10,VC11) right, soundlevel:0 – CSE2(audio): MCC20(), MCC21(), MCC22() (three dominant, spatially each covers whole scene) Capture Scene 2: (lower priority, less recent dominant speakers) – CSE1: MCC4(VC1,VC4,VC6,VC9,VC10,VC11) – left, soundlevel:1 – MCC5(VC2,VC5,VC7,VC9,VC10,VC11) – center, soundlevel:1 – MCC6(VC3,VC8,VC9,VC10,VC11) – right, soundlevel:1 MCCs do have spatial attributes More scenes like the first two, with lower priority, higher soundlevel# Could also be done without advertising VC1 – VC11 at all. In that case, MCCs could just be normal VCs I think. 11

1 Provider choose, one large scene 2 Consumer choose 3 Provider choose, multiple scenes Need to use MCC and Adv all VC Yes No, but might want to for other reasons (spatial audio/video relations?) Need to relate incoming packets to original source MC Yes, to get original spatial info Yes, if using audio streams as basis for switching No, but might want to for other reasons Consumer needs recent speaker info NoYesNo DisadvantagesEncodings can remap to different rendering area with each switch. Provider and Consumer make more assumptions about what the other is doing AdvantagesConsumer knows best how to choose, based on it’s desired UX Simpler in terms of protocol messages 12

Associate Rx streams with captures Need ability for consumer to associate received encodings with CLUE captures Has support on the list Issue for signaling and RTP mapping, but should be summarized in framework Associate encoding in RTP with provider’s encoded capture (MCC or individual capture) Associate encoding in RTP with the original capture, which is a constituent of provider’s MCC 13

Consumer needs “recent talker” info (not discussing today) Related to “approach 2” of switched capture discussion, for case when consumer chooses. Can the consumer get the info it needs from just the audio encodings it receives? Or should there be a more explicit signaling mechanism for the provider to give this to consumer? Note this general mechanism is already deployed in Microsoft Lync, and seems to work well 14

Need “active capture” info Signaling the “active video capture”, not using just audio energy in the audio stream. Old issue discussed a couple years ago in the working group Example: endpoint provider sends 3 video captures, and 1 mono audio – Consumer (MCU topo-mixer) wants to know which video has the loudest talker, so it can send that one to a simple endpoint that just receives one video stream – How does this MCU consumer know which one it is? Provider can signal this somehow, if it knows locally which video is associated with the talker – in RTP? – in CLUE channel? Relevant for the MCC example in framework

“overloading” media channels Editor note in section 5, and section 10 Propose clarifying this, to be consistent with signaling document m-line is either non-CLUE or CLUE, not both CLUE can cause changes in encoded streams, but must be compliant with most recent SDP O/A 16

RTP multiplexing Editor note in section 5, about multiplexing and RTP mapping. Propose remove the sentences about multiplexing, as the note suggests 17

Open Tickets #10 sufficient info for receiver – ready to close this? #19, #26 site switching – close after MCC example cleanup? #32 Roles – Christian proposed new terms, will propose text for framework based on design team Dec 17. #33 Security – Mary has proposal, need to add it to framework #35 consistency with data model - ongoing 18