Changes between Initial Version and Version 1 of Nokia_APS_VAS_Direct

Timestamp:

Jan 14, 2009 10:45:04 AM (15 years ago)

Author:

bennylp

Comment:

--

Nokia_APS_VAS_Direct

@@ +1 @@
+= Using APS-Direct and VAS-Direct in PJMEDIA =
+
+The '''APS-Direct''' and '''VAS-Direct''' is our codenames for functionalities to use the hardware codecs that are supported by Nokia APS and/or VAS directly, bypassing media processing in PJMEDIA. This feature will be introduced in PJSIP version 1.1.
+
+APS stands for '''Audio Proxy Server''', and it is available as plug-in for Nokia S60 3rd Edition up to Feature Pack 2 version. This has been deprecated for FP2 devices and above, and it is being replaced by '''VoIP Audio Services (VAS)''', which is available as plug-in for S60 FP1/FP2 devices and will be available built-in in later S60 versions.
+
+'''Table of Contents'''
+[[PageOutline(2-3,,inline)]]
+
+[[BR]]
+
+== Introduction ==
+
+The Nokia APS and VAS support codecs such as G.711 (PCMA and PCMU), G.729, iLBC, and AMR-NB, though the availability of these codecs may vary according to the handset types. There are significant benefits of using these codecs instead of software codecs (in PJMEDIA-CODEC), with the main benefits are performance (hardware vs software codecs, latency) and the given codec licenses/royalties.
+
+Due to these benefits, the ability to use these codecs in PJSIP applications is very desirable.
+
+[[BR]]
+
+== Changes ==
+
+The use of APS-Direct and VAS-Direct is very different than traditional PJMEDIA media processing, with the main difference being the audio frames returned by/given to the sound device are now in encoded format rather than in raw PCM format. The following changes will be done in order to support this.
+
+
+=== Non-PCM Media Ports ===
+
+Media ports may now support non-PCM media, and this is signaled by adding a new "format" field in the {{{pjmedia_port_info}}}. The "format" field will use FOURCC identification. For now only stream that will have non-PCM capability.
+
+ {{{
+ typedef union pjmedia_fourcc { 
+   pj_uint32_t  u32;
+   char         c[4];
+ } pjmedia_fourcc;
+ }}}
+
+
+
+=== New Frame Type to Carry Encoded Frames ===
+
+Support for new frame type (the {{{enum pjmedia_frame_type}}}): '''{{{PJMEDIA_FRAME_TYPE_EXTENDED}}}'''. When the frame's type is set to this type, the {{{pjmedia_frame}}} structure may be typecasted to '''{{{pjmedia_frame_ext}}}''' struct (new):
+
+ {{{
+typedef struct pjmedia_frame_ext {
+    pjmedia_frame   base;          /**< Base frame info */
+    pj_uint16_t     samples_cnt;   /**< Number of samples in this frame */
+    pj_uint16_t     subframe_cnt;  /**< Number of (sub)frames in this frame */
+
+    /* Zero or more (sub)frames follows immediately after this,
+     * each will be represented by pjmedia_frame_ext_subframe
+     */
+} pjmedia_frame_ext;
+
+typedef struct pjmedia_frame_ext_subframe {
+    pj_uint16_t     bitlen;   /**< Number of bits in the data */
+    pj_uint8_t      data[1];  /**< The encoded data */
+} struct pjmedia_frame_ext_subframe;
+ }}}
+
+
+=== (Symbian) Sound Device API ===
+
+The APS/VAS based sound device backends will support additional APIs:
+ - to query the list of supported codecs/formats,
+ - to set which format to use when opening the sound device,
+ - audio routing to loudspeaker or earpiece (this API is already available)
+
+=== New Audio Switchboard (the non-mixing conference bridge) ===
+
+Since audio frames are forwarded back and forth in encoded format, obviously the traditional conference bridge would not be able to handle it. A new object will be added, we call this audio switchboard ({{{conf_switch.c}}}) and it's API will be compatible with the existing conference bridge API, so that it can replace the bridge in the application by compile time switch.
+
+Understandably some conference bridge features will not be available:
+ - audio mixing feature (no conferencing feature),
+ - audio level adjustment and query,
+ - passive ports.
+
+Some of the features of the switchboard:
+ - uses the conference bridge API to control it (so it is compile-time compatible),
+ - one source may transmit to more than one destinations (though obviously one destination cannot take more than one sources since the switchboard cannot mix audio). This is useful for example to implement call recording feature in the application.
+ - it is optimized for low latency to the sound device (no ''delaybuf'' buffering for the microphone),
+ - much more lightweight (footprint and performance),
+ - supports routing audio from ports with different ''ptime'' settings.
+
+
+=== Non-PCM Format for Stream ===
+
+The stream must also support non-PCM audio frames in its {{{get_frame()}}} and {{{put_frame()}}} port interface. A new "format" field will be added to {{{pjmedia_stream_info}}} to let the application controls the audio format. The format is in FOURCC.
+
+