Changes between Version 13 and Version 14 of Nokia_APS_VAS_Direct
- Timestamp:
- Feb 23, 2009 10:48:07 AM (14 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
Nokia_APS_VAS_Direct
v13 v14 31 31 === What APS-Direct is really === 32 32 33 To use APS-Direct means that you're opening the sound device in codec (i.e. non-PCM) mode. You '''still have the choice''', at run-time, to open the sound device in PCM mode, if you wish, for example to make use of the PCM features in PJMEDIA such as the tone generator, or if you want to use the software codecs such as Speex or G.722 in PJMEDIA. Note that if you use PJSUA-LIB then the management of closing/re-opening the sound device using the correct codec maybe done by PJSUA-LIB automatically.33 To use APS-Direct means that you're opening the sound device in codec (i.e. non-PCM) mode. You '''still have the choice''', at run-time, to open the sound device in PCM mode, if you wish, for example to make use of the PCM features in PJMEDIA such as the tone generator, or if you want to use the software codecs such as Speex or G.722 in PJMEDIA. Note that if you use PJSUA-LIB then the management of closing/re-opening the sound device using the correct codec will be done by PJSUA-LIB automatically. 34 34 35 To use APS-Direct also means that you are restricted to use the ''audio switchboard'' at compile time (audio switchboard will be explained later). This means that you loose the capability of mixing audio with PJSUA-LIB, as well as several other restrictions on your audio routing arrangements. Among other things, you can't have two calls active and connected to the audio device at the same time. You can have more than one calls, but one of them must be put on-hold.35 To use APS-Direct also means that you are restricted to use the [#switchboard audio switchboard] at compile time (audio switchboard will be explained later). This means that you loose the capability of mixing audio with PJSUA-LIB, as well as several other restrictions on your audio routing arrangements. Among other things, you can't have two calls active and connected to the audio device at the same time. You can have more than one calls, but only one can be active (we recommend to put the other call on hold in this case). 36 36 37 The sound device can only handle one format at a time. For example, if it is currently opened with G.729 format, you can't reconnect it to different media ports (such as stream or other pjmedia_port). You must first close it, then re-open it using the correct format. 37 The sound device can only handle one format at a time. For example, if it is currently opened with G.729 format, you can't reconnect it to different media ports (such as stream or other pjmedia_port). You must first close it, then re-open it using the correct format. Note that if you are using PJSUA-LIB, this will be handled automatically (i.e. PJSUA-LIB will close/reopen the sound device with the correct format depending on what is connected to port zero in the [#switchboard audio switchboard]). But still you are limited to only able to connect one media port to the sound device at the same time (e.g. you cannot hear the call and play ringback tone to the sound device simultaneously). 38 38 39 39 … … 44 44 - similarly when passthrough codec is selected in the stream, the stream will emit and take encoded audio frames (rather than PCM frames), hence it needs APS-Direct on the other side. 45 45 46 One important thing to note: '''you may still use software codecs such as Speex and G.722 even when your application is compiled with APS-Direct support'''. When one of these software codecs is selected to be used by the stream, the stream will work as usual (i.e. emitting and taking PCM audio frames), so the audio device '''must''' be opened in normal/PCM mode (i.e. non-APS-Direct mode). 46 One important thing to note: '''you may still use software codecs such as Speex and G.722 even when your application is compiled with APS-Direct support'''. When one of these software codecs is selected to be used by the stream, the stream will work as usual (i.e. emitting and taking PCM audio frames), so the audio device '''must''' be opened in normal/PCM mode (i.e. non-APS-Direct mode). If you are using PJSUA-LIB, then again this will be handled automatically. 47 48 49 50 [[BR]] 51 52 == Using APS-Direct or VAS-Direct == 53 54 Currently only APS-Direct is implemented, and here are the steps to build the application with APS- Direct feature. 55 56 1. Enable the appropriate sound device implementation. Currently only [http://trac.pjsip.org/repos/wiki/APS] and native WMME sound device backends support APS-Direct feature. 57 1. Enable audio switch board, i.e. in config_site.h: 58 {{{ 59 #define PJMEDIA_CONF_USE_SWITCH_BOARD 1 60 }}} 61 1. Selectively enable/disable which PCM codecs to be supported, for example to disable all PCM codecs: 62 {{{ 63 #define PJMEDIA_HAS_G711_CODEC 0 64 #define PJMEDIA_HAS_L16_CODEC 0 65 #define PJMEDIA_HAS_GSM_CODEC 0 66 #define PJMEDIA_HAS_SPEEX_CODEC 0 67 #define PJMEDIA_HAS_ILBC_CODEC 0 68 #define PJMEDIA_HAS_G722_CODEC 0 69 #define PJMEDIA_HAS_INTEL_IPP 0 70 }}} 71 1. Enable passthrough codecs, and selectively enable/disable which passthrough codecs to be supported. The passthrough codecs supported would depend on which codecs are supported by the sound device backends that you choose to use: 72 {{{ 73 #define PJMEDIA_HAS_PASSTHROUGH_CODECS 1 74 75 // Disable all passthrough codecs except PCMA and PCMU 76 #define PJMEDIA_HAS_PASSTHROUGH_CODEC_PCMU 1 77 #define PJMEDIA_HAS_PASSTHROUGH_CODEC_PCMA 1 78 #define PJMEDIA_HAS_PASSTHROUGH_CODEC_AMR 0 79 #define PJMEDIA_HAS_PASSTHROUGH_CODEC_G729 0 80 #define PJMEDIA_HAS_PASSTHROUGH_CODEC_ILBC 0 81 }}} 82 1. If you are using PJSUA-LIB, then relatively that's all needed to make use of APS-Direct. Please note that the application logic must take care that there can only be one source transmitting to any destination in the switchboard. 47 83 48 84 … … 51 87 == Changes == 52 88 53 The use of APS-Direct and VAS-Direct is very different than traditional PJMEDIA media processing, with the main difference being the audio frames returned by/given to the sound device are now in encoded format rather than in raw PCM format. The following changes will bedone in order to support this.89 The use of APS-Direct and VAS-Direct is very different than traditional PJMEDIA media processing, with the main difference being the audio frames returned by/given to the sound device are now in encoded format rather than in raw PCM format. The following changes have been done in order to support this. 54 90 55 91 … … 60 96 61 97 {{{ 62 typedef enum pjmedia_format {98 typedef enum pjmedia_format_id { 63 99 PJMEDIA_FORMAT_PCM, 64 100 PJMEDIA_FORMAT_ALAW, … … 67 103 PJMEDIA_FORMAT_AMR_NB, 68 104 .. 105 } pjmedia_format_id; 106 107 /** Media format information. */ 108 typedef struct pjmedia_format 109 { 110 pjmedia_format_id id; 111 pj_uint32_t bitrate; 112 pj_bool_t vad; 69 113 } pjmedia_format; 70 114 }}} 71 115 72 73 Support for new frame type (the {{{enum pjmedia_frame_type}}}): '''{{{PJMEDIA_FRAME_TYPE_EXTENDED}}}'''. When the frame's type is set to this type, the {{{pjmedia_frame}}} structure may be typecasted to '''{{{pjmedia_frame_ext}}}''' struct (new): 116 We also need to support passing around non-PCM frames in PJMEDIA. We added support for new frame type (the {{{enum pjmedia_frame_type}}}): '''{{{PJMEDIA_FRAME_TYPE_EXTENDED}}}'''. When the frame's type is set to this type, the {{{pjmedia_frame}}} structure may be typecasted to '''{{{pjmedia_frame_ext}}}''' struct (new): 74 117 75 118 {{{ … … 90 133 }}} 91 134 135 The format information has also been added to '''pjmedia_port_info''' structure. 92 136 93 The stream must also support non-PCM audio frames in its {{{get_frame()}}} and {{{put_frame()}}} port interface. 137 The stream must also support non-PCM audio frames in its {{{get_frame()}}} and {{{put_frame()}}} port interface. The stream will set the correct format in its '''pjmedia_port_info''' structure depending on the codecs being used (i.e. if passthrough codec is being used, the format will contain non-PCM format information). 94 138 95 139 === Passthrough Codecs === 96 140 97 While the actual codec encoding/decoding will be done by APS/VAS, "dummy" codec instances still need to be created in PJMEDIA:141 While the actual codec encoding/decoding will be done by the sound device, "dummy" codec instances still need to be created in PJMEDIA: 98 142 - PJMEDIA needs the list of codecs supported to be offered/negotiated in SDP, 99 143 - some codecs have special framing requirements which are not handled by the hardware codecs, for example the framing rules of AMR codecs ([http://tools.ietf.org/html/rfc3267 RFC 3267]). … … 110 154 111 155 112 === New Audio Switchboard (the non-mixing conference bridge) === 156 === New Audio Switchboard (the non-mixing conference bridge) === #switchboard 113 157 114 Since audio frames are forwarded back and forth in encoded format, obviously the traditional conference bridge would not be able to handle it. A new object will be added, we call this audio switchboard ({{{conf_switch.c}}}) and it's API will be compatible with the existing conference bridge API, so that it can replace the bridge in the application by compile time switch.158 Since audio frames are forwarded back and forth in encoded format, the traditional conference bridge would not be able to handle it. A new object will be added, we call this audio switchboard ({{{conf_switch.c}}}) and it's API will be compatible with the existing conference bridge API to maintain compatibility with existing applications/PJSUA-LIB. 115 159 116 160 Understandably some conference bridge features will not be available: … … 125 169 - much more lightweight (footprint and performance), 126 170 - supports routing audio from ports with different ''ptime'' settings. 127 128 129 130 131 132 [[BR]]133 134 == Using APS-Direct or VAS-Direct ==135 136 Currently only APS-Direct is implemented, and here are the steps to build the application with APS- Direct feature.137 138 1. Enable APS sound device implementation as described [http://trac.pjsip.org/repos/wiki/APS here].139 1. Enable audio switch board, i.e. in config_site.h:140 {{{141 #define PJMEDIA_CONF_USE_SWITCH_BOARD 1142 }}}143 1. Enable passthrough codecs, i.e. in config_site.h:144 {{{145 #define PJMEDIA_HAS_PASSTHROUGH_CODECS 1146 }}}147 148 For building sample application {{{symbian_ua}}}, those steps are enough since it's already prepared to use APS- Direct.149 150 For general application, there are few more things to be handled:151 - Reopening sound device when it needs to change the active format/codec, e.g: when a call is confirmed and stream has been created, the sound device format should be matched to the SDP negotiation result. Here is the sample code for application that using pjsua-lib, reopening sound device is done in {{{on_stream_created()}}} pjsua callback, this will replace the precreated pjsua-lib sound device instance:152 {{{153 /* Global sound port. */154 static pjmedia_snd_port *g_snd_port;155 156 /* Reopen sound device on on_stream_created() pjsua callback. */157 static void on_stream_created(pjsua_call_id,158 pjmedia_session *sess,159 unsigned stream_idx,160 pjmedia_port**)161 {162 pjmedia_port *conf;163 pjmedia_session_info sess_info;164 pjmedia_stream_info *strm_info;165 pjmedia_snd_setting setting;166 unsigned samples_per_frame;167 168 /* Get active format for this stream, based on SDP negotiation result. */169 pjmedia_session_get_info(sess, &sess_info);170 strm_info = &sess_info.stream_info[stream_idx];171 172 /* Init sound device setting based on stream info. */173 pj_bzero(&setting, sizeof(setting));174 setting.format = strm_info->param->info.format;175 setting.bitrate = strm_info->param->info.avg_bps;176 setting.cng = strm_info->param->setting.cng;177 setting.vad = strm_info->param->setting.vad;178 setting.plc = strm_info->param->setting.plc;179 180 /* Close sound device and get the conference port. */181 conf = pjsua_set_no_snd_dev();182 183 samples_per_frame = strm_info->param->info.clock_rate *184 strm_info->param->info.frm_ptime *185 strm_info->param->info.channel_cnt /186 1000;187 188 /* Reset conference port attributes. */189 conf->info.samples_per_frame = samples_per_frame;190 conf->info.clock_rate = strm_info->param->info.clock_rate;191 conf->info.channel_count = strm_info->param->info.channel_cnt;192 conf->info.bits_per_sample = 16;193 194 /* Reopen sound device. */195 pjmedia_snd_port_create2(app_pool,196 PJMEDIA_DIR_CAPTURE_PLAYBACK,197 0,198 0,199 strm_info->param->info.clock_rate,200 strm_info->param->info.channel_cnt,201 samples_per_frame,202 16,203 &setting,204 &g_snd_port);205 206 /* Connect sound to conference port. */207 pjmedia_snd_port_connect(g_snd_port, conf);208 }209 }}}210 - Note that sound device instance is now owned and managed by application, so211 212 {{{pjsua_media_config.snd_auto_close_time}}} will not work. Here is the sample code to utilize213 214 {{{on_stream_destroyed()}}} pjsua callback as the trigger of closing the sound device:215 {{{216 /* Close sound device on on_stream_destroyed() pjsua callback. */217 static void on_stream_destroyed(pjsua_call_id,218 pjmedia_session*,219 unsigned)220 {221 if (g_snd_port) {222 pjmedia_snd_port_destroy(g_snd_port);223 g_snd_port = NULL;224 }225 }226 227 }}}228 171 229 172