Changes between Version 41 and Version 42 of Video_Users_Guide


Ignore:
Timestamp:
Mar 20, 2012 10:43:03 AM (12 years ago)
Author:
nanang
Comment:

Updated video codec parameter guide

Legend:

Unmodified
Added
Removed
Modified
  • Video_Users_Guide

    v41 v42  
    292292Application retrieves {{{pjsua_vid_win_info}}} with {{{pjsua_vid_win_get_info()}}}. The one window property that most applications will be interested with is the native window handle of the video. The native video handle is contained by {{{pjmedia_vid_dev_hwnd}}} structure inside {{{pjsua_vid_win_info}}}. Application can use the native handle to embed the video window into application's GUI structure. Alternatively, the library also provides few simple and most commonly used API to operate the window, such as {{{pjsua_vid_win_set_show()}}}, {{{pjsua_vid_win_set_size()}}}, etc., however the availability of these APIs are not guaranteed since it depends on the underlying backend device. 
    293293 
    294 === Modifying video codec parameters (video size, fps, bps) === 
    295  
    296 The default video size setting for outgoing video is specified in the encoded format field in the codec param, i.e: {{{pjmedia_vid_codec_param.enc_fmt.vid.size}}}. For H.264, its profile level limits the video size, so the remote H.264 SDP fmtp "profile-level-id" should allow this size. If the remote endpoint is also PJSIP, the SDP fmtp, e.g: "profile-level-id", can be set in {{{pjmedia_vid_codec_param.dec_fmtp}}}. 
    297  
    298 Here is a sample code to configure profile-level-id: 
    299  {{{ 
    300  ... 
    301  param.setting.dec_fmtp.param[N].name = pj_str("profile-level-id"); 
    302  param.setting.dec_fmtp.param[N].val  = pj_str("...."); 
    303  ... 
    304  pjsua_vid_codec_set_param(&codec_id, &param); 
    305  }}} 
    306  
    307 The outgoing frame-rate and bps setting can be set the same way as video size setting above, e.g: for 10 fps, just set 
    308 {{{pjmedia_vid_codec_param.enc_fmt.vid.fps}}} to {10, 1} then call {{{pjsua_vid_codec_set_param()}}}. 
     294=== Modifying video codec parameters for video call === 
     295 
     296Video codec parameters are specified in {{{pjmedia_vid_codec_param}}}. The codec parameters provides separate settings for each direction, encoding and decoding. Any modifications on video codec parameters can be applied using {{{pjsua_vid_codec_set_param()}}}. 
     297 
     298==== Size or resolution ==== 
     299 
     300Specify video picture dimension. 
     301 
     302 a. For encoding direction, configured via {{{pjmedia_vid_codec_param.enc_fmt.vid.size}}}, e.g: 
     303    {{{ 
     304/* Sending 1280 x 720 */ 
     305pjmedia_vid_codec_param.enc_fmt.vid.size.w = 1280; 
     306pjmedia_vid_codec_param.enc_fmt.vid.size.h = 720; 
     307    }}} 
     308    Note that there is possibility that the value will be adjusted to follow remote capability. For example, if remote signals that maximum resolution supported is 640 x 480 and locally the encoding direction size is set to 1280 x 720, then 640 x 480 will be used. 
     309 b. For decoding direction, two steps needed: 
     310    1. {{{pjmedia_vid_codec_param.dec_fmt.vid.size}}} should be set to the highest value expected for incoming video size. 
     311    2. signalling to remote, configured via codec specific SDP format parameter (fmtp): {{{pjmedia_vid_codec_param.dec_fmtp}}}. 
     312       - H263-1998, e.g: 
     313         {{{ 
     314/* 1st preference: 352 x 288 (CIF) */ 
     315pjmedia_vid_codec_param.dec_fmtp.param[n].name = pj_str("CIF"); 
     316pjmedia_vid_codec_param.dec_fmtp.param[n].value = pj_str("1"); /* the value actually specifies framerate, see framerate section below */ 
     317/* 2nd preference: 172 x 144 (QCIF) */ 
     318pjmedia_vid_codec_param.dec_fmtp.param[n+1].name = pj_str("QCIF"); 
     319pjmedia_vid_codec_param.dec_fmtp.param[n+1].value = pj_str("1"); /* the value actually specifies framerate, see framerate section below */ 
     320         }}} 
     321       - H264, the size is implicitly specified in H264 level (check the standard specification or [http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels this]) and on SDP, the H264 level is signalled via H264 SDP fmtp [http://tools.ietf.org/html/rfc6184#section-8.1 profile-level-id], e.g: 
     322         {{{ 
     323/* Can receive up to 1280×720 @30fps */ 
     324pjmedia_vid_codec_param.dec_fmtp.param[n].name = pj_str("profile-level-id"); 
     325pjmedia_vid_codec_param.dec_fmtp.param[n].value = pj_str("xxxx1e"); /* "1e" means level 3.1 */ 
     326         }}} 
     327 
     328==== Framerate ==== 
     329 
     330Specify number of frame processed per second. 
     331 
     332 a. For encoding direction, configured via {{{pjmedia_vid_codec_param.enc_fmt.vid.fps}}}, e.g: 
     333    {{{ 
     334/* Sending @30fps */ 
     335pjmedia_vid_codec_param.enc_fmt.vid.fps.num   = 30; 
     336pjmedia_vid_codec_param.enc_fmt.vid.fps.denum = 1; 
     337    }}} 
     338    Note that there is possibility that the value will be adjusted to follow remote capability. For example, if remote signals that maximum framerate supported is 10fps and locally the encoding direction framerate is set to 30fps, then 10fps will be used. 
     339 b. For decoding direction, two steps needed: 
     340    1. {{{pjmedia_vid_codec_param.dec_fmt.vid.fps}}} should be set to the highest value expected for incoming video framerate. Note that it is even better if this is set to 1.5x (or more) of the expected incoming framerate because the renderer will use this setting for its clock and there is always possibility that remote clock runs a bit faster (clock drift), so setting it higher will avoid high latency issue caused by clock drift. 
     341    2. signalling to remote, configured via codec specific SDP format parameter (fmtp): {{{pjmedia_vid_codec_param.dec_fmtp}}}. 
     342       - H263-1998, maximum framerate is specified per size/resolution basis, check [http://tools.ietf.org/html/rfc4629#section-8.1.1 here] for more info. 
     343         {{{ 
     344/* 3000/(1.001*2) fps for CIF */ 
     345pjmedia_vid_codec_param.dec_fmtp.param[m].name = pj_str("CIF"); 
     346pjmedia_vid_codec_param.dec_fmtp.param[m].value = pj_str("2"); 
     347/* 3000/(1.001*1) fps for QCIF */ 
     348pjmedia_vid_codec_param.dec_fmtp.param[n].name = pj_str("QCIF"); 
     349pjmedia_vid_codec_param.dec_fmtp.param[n].value = pj_str("1"); 
     350         }}} 
     351       - H264, similar to size/resolution, the framerate is implicitly specified in H264 level (check the standard specification, or http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels) and the H264 level is signalled via H264 SDP fmtp {{{profile-level-id}}}, e.g: 
     352         {{{ 
     353/* Can receive up to 1280×720 @30fps */ 
     354pjmedia_vid_codec_param.dec_fmtp.param[n].name = pj_str("profile-level-id"); 
     355pjmedia_vid_codec_param.dec_fmtp.param[n].value = pj_str("xxxx1e"); 
     356         }}} 
     357 
     358==== Bitrate ==== 
     359 
     360Specify bandwidth requirement for video payloads stream delivery. 
     361 
     362This is configurable via {{{pjmedia_vid_codec_param.enc_fmt.vid.avg_bps}}} and {{{pjmedia_vid_codec_param.enc_fmt.vid.max_bps}}}, e.g: 
     363{{{ 
     364/* Bitrate range preferred: 512-1024kbps */ 
     365pjmedia_vid_codec_param.enc_fmt.vid.avg_bps = 512000; 
     366pjmedia_vid_codec_param.enc_fmt.vid.max_bps = 1024000; 
     367}}} 
     368 
     369Notes: 
     370 - This setting is applicable for encoding and decoding direction, currently there is no way to set asymmetric bitrate. 
     371 - The bitrate setting of all codecs will be enumerated and the highest value will be signalled in bandwidth info in local SDP (see #1244). 
     372 - There is possibility that the encoding bitrate will be adjusted to follow remote bitrate setting, i.e: read from SDP bandwidth info (b=TIAS line) in remote SDP. For example, if remote signals that maximum bitrate is 128kbps and locally the bitrate is set to 512kbps, then 128kbps will be used. 
     373 - If codec specific bitrate setting signalling (via SDP fmtp) is desired, e.g: ''MaxBR'' for H263, application should put the SDP fmtp manually, for example: 
     374   {{{ 
     375/* H263 specific maximum bitrate 512kbps */ 
     376pjmedia_vid_codec_param.dec_fmtp.param[n].name = pj_str("MaxBR"); 
     377pjmedia_vid_codec_param.dec_fmtp.param[n].value = pj_str("5120"); /* = max_bps / 100 */ 
     378   }}} 
    309379 
    310380