Changes between Version 1 and Version 2 of 3rd_Party_Media


Ignore:
Timestamp:
Mar 15, 2008 10:57:24 AM (16 years ago)
Author:
bennylp
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • 3rd_Party_Media

    v1 v2  
    55PJSIP consists of several libraries which mostly are independent from one another. The library architecture diagram below shows how libraries are interacting with each other. 
    66 
     7[[Image(diagram.jpg)]] 
    78 
    89 
    910== SIP and Media Interaction == 
    1011 
    11 Pseudo-code when creating a call (same flow for caller and callee): 
     12Before we discuss how to integrate third party media library with PJSIP, we need to understand how PJSIP interacts with the media stack (PJMEDIA, in this case). This interaction/integration is done by either PJSUA-LIB or application, depending on whether the application uses PJSUA-LIB or PJSIP directly. 
    1213 
    13 '''create_call()'''[[BR]] 
    14 {[[BR]] 
    15     ''// Create the dialog''[[BR]] 
    16     '''dlg = pjsip_dlg_create_uac/uas();''' 
     14There are few places where PJSIP and PJMEDIA interact with each other: 
     15 * '''During call/invite session creation''': we need to specify local SDP when creating invite session. 
     16 * '''After SDP negotiation''': once SDP has been negotiated, we can start our media streaming. 
     17 * Anytime thereafter when the invite session requires us to update our offer or answer. 
    1718 
    18     ''// Create media transport (pair of RTP/RTCP socket) for''[[BR]] 
    19     ''// transmitting/receiving the media from remote endpoint''[[BR]] 
    20     '''media_transport = pjmedia_transport_udp_create();''' 
     19These will be discussed in more detail in the following sections. 
    2120 
    22     ''// Get the socket address info of the media transport''[[BR]] 
    23     ''// These are needed to create a complete SDP descriptor''[[BR]] 
    24     '''media_sock_info = pjmedia_transport_get_info(media_transport);''' 
     21=== Creating a Call/Invite Session === 
     22 
     23An invite session is created by application (or PJSUA-LIB) when we make outgoing call or when we receive incoming call. We need to specify our local SDP when creating the invite session, because SDP negotiation will be done by the invite session [*], and to do this it needs SDP from both local and remote endpoints. Normally application just need to ask PJMEDIA to create local SDP for us, based on capabilities that have been registered to it. 
     24 
     25  ''[*] To be precise, the negotiation actually is done by the SDP negotiator (part of PJMEDIA/PJSDP library) which will be invoked by the invite session. The decision to make SDP negotiation as integral part of invite session is to shield application from the complexity of SDP offer/answer session (for example, with late offer/answer, offer/answer quirks related to the use of PRACK, and ability to update session early with UPDATE), while maintaining the flexibility for the application to control when and what offer and answer to be given to remote.'' 
     26 
     27Below is the pseudo-code for creating a call/invite session. The procedure is relatively similar for both caller and callee. 
     28 
     29{{{ 
     30#!c 
     31create_call() 
     32{ 
     33    // Create the dialog 
     34    dlg = pjsip_dlg_create_uac/uas(); 
     35 
     36    // Create media transport (pair of RTP/RTCP socket) for 
     37    // transmitting/receiving the media from remote endpoint 
     38    media_transport = pjmedia_transport_udp_create(); 
     39 
     40    // Get the socket address info of the media transport 
     41    // These are needed to create a complete SDP descriptor 
     42    media_sock_info = pjmedia_transport_get_info(media_transport); 
    2543    
    26     ''// Create local SDP to be given to invite session''[[BR]] 
    27     '''sdp = pjmedia_endpt_create_sdp(media_sock_info);''' 
     44    // Create local SDP to be given to invite session 
     45    sdp = pjmedia_endpt_create_sdp(media_sock_info); 
    2846 
    29     ''// Create the invite session, giving it both the dialog''[[BR]] 
    30     ''// and local SDP to be negotiated''[[BR]] 
    31     '''inv = pjsip_inv_create_uac/uas();'''[[BR]] 
     47    // Create the invite session, giving it both the dialog 
     48    // and local SDP to be negotiated 
     49    inv = pjsip_inv_create_uac/uas(dlg, sdp); 
    3250} 
     51}}} 
    3352 
    34 '''pjsip_endpt_create_sdp(media_sock_info)'''[[BR]] 
    35 {[[BR]] 
    36   ''// Query list of codecs that we support''[[BR]] 
    37   '''codec_list[] = pjmedia_codec_mgr_enum_codecs();''' 
     53To look what ''pjmedia_endpt_create_sdp()'' function is doing in detail, below is the pseudo-code of the function: 
    3854 
    39   ''// Put each codec information in the SDP''[[BR]] 
    40   ... 
     55{{{ 
     56#!c 
     57pjsip_endpt_create_sdp(media_sock_info) 
     58{ 
     59  // Query list of codecs that we support from the codec manager 
     60  codec_list[] = pjmedia_codec_mgr_enum_codecs(); 
    4161 
    42   ''// Put media socket address info in SDP c= and m= line''[[BR]] 
    43   ... 
     62  // Put each codec information in the SDP 
     63  for each codec in codec_list { 
     64    put codec info in SDP 
     65  } 
    4466 
    45   '''return sdp;''' 
     67  // Put media socket address info in SDP c= and m= line  
     68  sdp->connection_line = media_sock_info->rtp.ip_address; 
     69  sdp->m_line.port = media_sock_info->rtp.port_number; 
     70 
     71  return sdp; 
    4672} 
     73}}} 
     74 
     75 
     76=== Starting the Media === 
     77 
     78As discussed above, SDP negotiation is done by the SDP negotiator object that is invoked internally by the invite session. The invite session user (application or PJSUA-LIB) will get notification whenever the SDP negotiation has been done, via invite session's '''on_media_update()''' callback. 
     79 
     80The ''on_media_update()'' callback contains ''status'' parameter which tells application whether SDP negotiation has been successful (or not). We are normally only interested with the successful status to start our media. 
     81 
     82Below is the pseudo-code to implement ''on_media_update()'' callback. You can see a working implementation of this function in ''pjsua_call_on_media_update()'' function of {{{pjsua_call.c}}}. 
     83 
     84{{{ 
     85!#c 
     86void on_media_update(inv, status) 
     87{ 
     88   if (status != PJ_SUCCESS) { 
     89      // Handle failed negotiation scenario 
     90      ... 
     91      return; 
     92   } 
     93 
     94   // Retrieve both local and remote active SDP. 
     95   // Active SDP is the SDP that has been negotiated 
     96   // by SDP negotiator. 
     97   local_sdp = pjmedia_sdp_neg_get_active_local(inv->neg); 
     98   remote_sdp = pjmedia_sdp_neg_get_active_remote(inv->neg); 
     99 
     100   // Update media based on local and remote SDP 
     101   update_media_channel(local_sdp, remote_sdp); 
     102} 
     103}}} 
     104 
     105And below is the pseudo-code of ''update_media_channel()'' function. You can see a working implementation of this function in ''pjsua_media_channel_update()'' of pjsua_media.c. 
     106 
     107The ''update_media_channel'' needs to handle two cases: 
     108 * when media is activated 
     109 * when media is deactivated (for example, on call hold) 
     110 
     111{{{ 
     112!#c 
     113update_media_channel(local_sdp, remote_sdp) 
     114{ 
     115   // Create media session info from both local and remote SDP. 
     116   // The media session contains everything that's needed to create our 
     117   // media stream (codec settings, transport settings (local and remote 
     118   // socket addresses), jitter buffer settings, etc.) 
     119   media_sess_info = pjmedia_session_info_from_sdp(local_sdp, remote_sdp); 
     120 
     121   // Handle inactive media 
     122   if (media_sess_info->dir = NONE) { 
     123      // Handle inactive media 
     124      .. 
     125       
     126      return; 
     127   } 
     128 
     129   // Customize media session info if wanted (for example, turn VAD/PLC  
     130   // on/off, change jitter buffer setting, set SSRC, etc). 
     131   .. 
     132 
     133   // Create the media session, giving it both the media session info 
     134   // and media transport. We created the media transport earlier when 
     135   // we create the invite session. 
     136   m_session = pjmedia_session_create(media_sess_info, media_transport); 
     137 
     138   // Extract audio stream from the media session 
     139   stream_port = pjmedia_session_get_port(m_session, 0); 
     140 
     141   // "Connect" the audio stream to sound device (or conference bridge, 
     142   // or anything else). 
     143   pjmedia_snd_port_connect(sound_device, stream_port); 
     144} 
     145}}} 
     146