H323 MCU Functionality

Overview: Current version CDE allows single H323 End Point to join the audio/video conference using H323CDEAdapter Release 1.7.4.The new version of CDE will allow creation/termination of an audio/video conference room in which multiple H323 End Points are able to join using H323CDEAdapter. In both the cases H323CDEAdapter will be considered as a single logical Web Client by the CDE MCU.

Operational Concepts and Scenarios: The AV Client is responsible for handling audio and video for a web client. Communication between CDEMCU and H323CDEAdapter is based on a protocol provided by NECST. The purpose of this protocol is to manage communication between CDEMCU and H323CDEAdapter. CDEMCU will send multiple audio buffers in a single stream; H323CDEAdapter will detect the speaker among the web clients. H323CDEAdapter will also detect the speaker among the H323 End Points The video display will depend upon the type of the video mode. There are basically two video modes: Mixed Video Mode Single Video Mode

In the mixed video mode the video display will be as : a) Mixed Video Mode In the mixed video mode the video display will be as : H323CDEAdapter will display quadrant image at the H323 End Point side. . First Quadrant Second Quadrant Third Quadrant Fourth Quadrant The image of the Web Client speaker will always be shown in the first quadrant. The image of the speaker among the H323 End Points will be shown at the second quadrant. The image of any two H323 end points other than the speaker will be switched between third and fourth quadrants. If H323 End Point, who was speaker and whose image was being displayed in the second quadrant logs out, then any of the H323 End Point image will be displayed in the second quadrant. In case there is no Web client present in the conference, the first quadrant will display gray screen. Same 4 quadrant image will be sent to CDE MCU server.

b) Single video mode Display the video of the speaker among the H323 EP at the WC side. Display the video of the speaker among all the participants at the H323 EP side. If the H323 EP is itself the speaker among all the participants then display the video of the previous speaker among all the participants at that H323 EP.

The audio of the H323 End points will be sent to all other H323 End points except himself. H323CDEAdapter will mix the audio from the CDEMCU with the audio from all the H323 End Points and will send it back to CDE MCU. H323CDEAdapter will send the mixed audio of CDE MCU and the audio of all the H323 End Points except itself to each H323 End Point.

Following diagram describes the functional blocks for H323CDEAdapter and its interaction with CDEMCU: H323CDEAdapter CDEMCU Audio (Send/Receive) Video (Send/Receive) H323 MCU AV Client Protocol Entities 1 2 3 4 H323endpoint

In the implementation each of the H323 EP will have a audio buffer dictionary in which it will have audio buffer for each of the H323 EP except himself and an audio buffer for the CDE audio. We will also have a audio buffer dictionary for CDE MCU which will have audio buffer for every H323 EP. EP2 EP3 EP4 EP5 EP6 CDE EP1 EP2 EP1 EP3 EP4 EP5 EP6 CDE CDEMCU EP1 EP2 EP3 EP4 EP5 EP6

Implementation The classes that incorporate changes for the implementation of this requirement are: H323CAConferenceManager.cpp H323CAConnection.cpp H323CACDEConnection.cpp H323CAVideoBuffer.cpp

H323Connection* H323CAConferenceManager::CreateConnection (unsigned callRefValue) { //To check whether the no. of H323 EP logged in is less than six or not. If yes then // create a new connection for this H323 EP if( GetNoOfH323EP() < MAX_H323_EP ) iNoOfH323Login++; ssrc = dwH323SSRC & 0xff00ffff; // To generate the SSRC for the ssrc |= DWORD(iNoOfH323Login<<16); // H323 EP pH323Connection = new H323CAConnection(*this, callRefValue,ssrc); IncrNoOfH323EP(); // Increment the no. of H323 EP //logged in SetAnswerSend(false); } -----------

BOOL H323CAConferenceManager::WriteH323Audio (const void BOOL H323CAConferenceManager::WriteH323Audio (const void* buffer , PINDEX amount,const PString & thisToken,PString roomID, DWORD ssrc) { if ( DetectAudioLevel(buffer,amount) ) //If the audio packets coming are above audio // level Check the SSRC for H323 EP speaker and Overall speaker if(CheckH323EPSpeaker(ssrc)); if(CheckOverallSpeaker(ssrc)); //Write the audio packet in the audio buffer of every connection(H323EP) except //this connection //Write the audio packet in the audio buffer in the audio buffer dictionary for CDE }

BOOL H323CAConferenceManager::ReadH323Audio (const void BOOL H323CAConferenceManager::ReadH323Audio (const void* buffer , PINDEX amount,const PString & hisToken,PString roomID) { // Read the CDE audio and write in the audio buffer for CDE in every // connection(H323EP) // Call the ReadH323Audio method for this connection to read the mixed //audio conn->ReadH323Audio(thisToken, buffer, amount); }

BOOL H323CAConferenceManager::ReadCDEAudio (const void. buffer , BOOL H323CAConferenceManager::ReadCDEAudio (const void* buffer , PINDEX amount) { //scan through the audio buffers in the audio buffer dictionary for CDE //and mix the signals PINDEX i; for (i = 0; i < H323AudioBuffers.GetSize(); i++) PString key = H323AudioBuffers.GetKeyAt(i); H323AudioBuffers[key].ReadAndMix((BYTE *)buffer, amount, H323AudioBuffers.GetSize()); }

BOOL H323CAConferenceManager::WriteCDEVideo ( const void. buffer, BOOL H323CAConferenceManager::WriteCDEVideo ( const void * buffer, PINDEX amount, BOOL) { // If mixed video mode then write the video buffer in the CDE video //buffer oCDEVideoBuffer.Write((BYTE *)buffer, amount); // If the video mode is set single then set the OverallSpeakerSSRC //and OverallPreviousSpeakerSSRC according to the video display //rules and write the video buffer in the corresponding buffer // i.e. in oOverallSpeakerVideoBuffer or in //OverallPrevSpeakerVideoBuffer }

void H323CAConferenceManager::UpdateUserList(DWORD SSRC, PString userName, PString userType ,BOOL type) { if ( type ) // If user logs in // If it is the first participant in the conference then set the //dOverallSpeakerSSRC and // dOverallPrevSpeakerSSRC as zero for proper video display. // Add the audio count object for the user in the overall audio count list as well. } else // user logs out // remove the audio count object for the user from the overall audio count list // check if the user that logs out is Overall speaker and if yes then set the //dOverallSpeakerSSRC=0 // check if the user that logs out is Overall Previous speaker and if yes then set // dOverallPrevSpeakerSSRC=0

int H323CAConferenceManager::AdapterLoginRequest(PString CallerName) { // Check for pCDEConnection ptr and perform login functions only if it I //s Null }

void H323CAConferenceManager::AdapterLogoutRequest() { //Clear H323 EP list //Clear the H323 Audio count list and also the overall audio count list. // Clear the overall speaker video buffer and overall Previous Speaker // video buffer // Set the dOverallSpeakerSSRC and dOverallPrevSpeakerSSRC to zero. } void H323CAConferenceManager::SendReleaseComplete() { //Clear H323 EP list //Clear the H323 Audio count list and also the overall audio count list. }

void H323CAConferenceManager::AddH323Member(H323CAConnection void H323CAConferenceManager::AddH323Member(H323CAConnection * newMember, DWORD ssrc) { // If this is the first entry in the room then create the room and video buffer and video position list // Add this H323 EP to the H323 EP list and add the audio count object for this end point in // H323AudioCountList and OverallAudioCountList. // Add the new member to every other H323 member // and every other H323 member to the new H323 member for (PINDEX i = 0; i < H323EPList.GetSize(); i++) PString token = H323EPList[i]; if (token != newToken) { H323CAConnection * conn = (H323CAConnection *) FindConnectionWithLock(token); if (conn != NULL) { conn->AddH323Member(newToken); newMember->AddH323Member(token); conn->Unlock(); } newMember->AddH323Member(CDEToken); // Add this H323 EP to the CDE

void H323CAConferenceManager::RemoveH323Member(H323CAConnection void H323CAConferenceManager::RemoveH323Member(H323CAConnection * oldConn, DWORD ssrc) { //Clear the corner of the video window that the connection just vacated. // Remove the member from the other members // Remove the member from the CDE //Remove the audio count for this H323 EP from OverallAudioCountList //and H323AudioCountList // Check if the H323 EP that logs out is Overall Previous speaker and if yes // then set the // dOverallPrevSpeakerSSRC=0 // Remove the room if it becomes empty and call adapter logout request }

PINDEX H323CAConferenceManager::FindTokensVideoPosn(const PString & PINDEX H323CAConferenceManager::FindTokensVideoPosn(const PString & thisToken, PString roomID) { // To return the video Position of the H323 EP } BOOL H323CAConferenceManager::CheckH323EPSpeaker(DWORD SSRC) // To check the H323 Audio Packet for the speaker detection among the //H323 EP BOOL H323CAConferenceManager::CheckOverallSpeaker(DWORD SSRC) // to check the Audio Packet of either H323 EP or WC for the speaker //detection among the participants

BOOL H323CAConferenceManager::WriteVideo(const PString & thisToken, BOOL H323CAConferenceManager::WriteVideo(const PString & thisToken, const void * buffer, PINDEX amount, PString roomID, DWORD ssrc) { // This is called when video mode is set as mixed // get the video position of the H323 EP in the quadrant display // Check for H323 Speaker SSRC and set its value in accordance with //video display scenario and // write the video buffer in the corresponding video buffer }

BOOL H323CAConferenceManager::ReadVideo(const void. buffer, PINDEX BOOL H323CAConferenceManager::ReadVideo(const void * buffer, PINDEX amount, PString roomID ) { // read the CDE video oCDEVideoBuffer.Read((BYTE*)buffer, amount); // Write in the video buffer for the H323 EP at position 0 i.e. in the first //quadrant videoBuffer.Write((BYTE *)buffer, amount, 0); // Read the complete video buffer videoBuffer.Read((BYTE *)buffer,amount); // Write the same video buffer in the H323VideoBuffer to be send to CDE //MCU oH323VideoBuffer.Write((BYTE *)buffer , amount); }

BOOL H323CAConferenceManager::WriteVideoSingleMode ( const PString & BOOL H323CAConferenceManager::WriteVideoSingleMode ( const PString & thisToken, const void * buffer, PINDEX amount ,PString roomID, DWORD ssrc ) { //Check for H323 EP speaker and if 0 set the value to this SSRC // Now Check the value of H323 EP speaker and if equals this SSRC //then write the video in H323VideoBuffer // Check the value of OverallSpeakerSSRC and //OverallPrevSpeakerSSRC and set their values in accordance with the //video display scenario and write the video in corres. Video buffer }

BOOL H323CAConferenceManager::ReadVideoSingleMode( const PString & thisToken, const void * buffer, PINDEX amount, PString roomID ,DWORD ssrc) { // read OverAllSpeaker video if the SSRC is not of the H323EP Speaker if( ssrc != GetOverallSpeakerSSRC() ) oOverallSpeakerVideoBuffer.Read((BYTE*)buffer, amount); } else // read the OverallPreviousSpeaker video oOverallPrevSpeakerVideoBuffer.Read((BYTE *)buffer , amount);

void H323CAConnection::CleanUpOnCallEnd() { // close all the audio video channels for incoming and outgoing audio/ //video transmission // Remove the member from the list refCAConfManager.RemoveH323Member(this,SSRC); } H323Connection::AnswerCallResponse H323CAConnection::OnAnswerCall(const PString & caller, const H323SignalPDU & setupPDU, H323SignalPDU & ) ……………….. // Add the member in the list refCAConfManager.AddH323Member(this,SSRC);

void H323CAConnection::OnIncomingVideo(const void void H323CAConnection::OnIncomingVideo(const void * buffer, PINDEX amount) { // If video mode is mixed video mode then call corresponding function refCAConfManager.WriteVideo(GetCallToken(), buffer, amount, roomID, SSRC); //if video mode is single video mode then call refCAConfManager.WriteVideoSingleMode(GetCallToken(), buffer, amount, roomID, SSRC); } void H323CAConnection::OnOutgoingVideo(const void * buffer, PINDEX amount) refCAConfManager.ReadVideo(buffer, amount, roomID); refCAConfManager.ReadVideoSingleMode(GetCallToken(), buffer, amount, roomID, SSRC);

void H323CAConnection::AddH323Member(const PString & token) { // Add the audio buffer for the specified token in this connection audioBuffer.SetAt(token, new H323CAAudioBuffer); } void H323CAConnection::RemoveH323Member(const PString & token) // Remove the audio buffer for the specified token from this connection audioBuffer.RemoveAt(token);

BOOL H323CAConnection::WriteH323Audio(const PString & token, const void * buffer, PINDEX amount) { // Get the audio buffer for the specified token in this connection H323CAAudioBuffer *audioBuff = audioBuffer.GetAt(token); // and write the audio in that audio buffer audioBuff->Write((BYTE *)buffer, amount); } BOOL H323CAConnection::ReadH323Audio(const PString & /*token*/, const void * buffer, PINDEX amount) // scan through the audio buffers and mix the signals PINDEX i; for (i = 0; i < numChannels; i++) PString key = audioBuffer.GetKeyAt(i); audioBuffer[key].ReadAndMix((BYTE *)buffer, amount, numChannels);

void H323CACDEConnection::AudioVideoTimer(PTimer& , INT) { //Start Read Audio timer by giving the notifier function and delay time timerReadAudio.SetNotifier(PCREATE_NOTIFIER(OnReadAudioTime)); timerReadAudio.RunContinuous(READ_AUDIO_TIMEOUT); } void H323CACDEConnection::OnWriteAudioTimeForH323() ------------- // Check for overall speaker also void H323CACDEConnection::OnReadAudioTime(PTimer &, INT) // Call this function at regular intervals to read audio for CDE ReadAudioForCDE();

void H323CAVideoBuffer::Clear(PINDEX posn) { // Clear the video buffer at the specified position for mixed video //mode } void H323CAVideoBuffer::Write(BYTE * data, PINDEX amount, PINDEX posn) // Write the video buffer at the specified position for mixed video //mode