There is G722 implementation that uses 10ms packet length. As currently the pjmedia G722 wrapper expects G722 packet length to be (multiple of) 20ms, the incoming 10ms G722 packets cannot be parsed properly, which leads to:

• no audio
• divide-by-zero exception

There is MIME subtype AAL2-G726-xx for the packetization variants

This ticket will enable PJSUA application to resolve target hostname to IPv6 address. As currently PJLIB-UTIL DNS resolver has not completely supported IPv6 yet, the resolution will only work when DNS server setting (i.e: pjsua_config.nameserver) is not configured, the library will resolve hostname using pj_getaddrinfo().

Update: Ticket #1927 has been completed, PJLIB-UTIL DNS resolver now supports IPv6 for DNS SRV and DNS A/AAAA resolutions. So the following specifications should be applicable to PJSUA-LIB with and without nameserver settings.

Account transport type

This new field is added to PJSUA account structure (in pjsua internal), it represents account transport preference, i.e: local account will always have specific transport type (as it is created with transport ID param), and normal account will have specific transport type only if it is bound to specific transport, i.e: via pjsua_acc_config.transport_id.

Contact generation

• If account is bound to a specific transport address family of the contact will use the transport address family.
• Otherwise:
  • if target is an IP address, generated contact will use the same IP version as target
  • if target is a hostname (this is rather difficult as we don't really resolve the target here, except for a special case below):
    • if account transport type is specified, generated contact will use IP version as account transport type
    • otherwise, it will simply use IPv4 (!) as there is no target resolution here, hence application is recommended to bind an IPv6-only account to a specific IPv6 transport.
    • special case when pjsua_acc_config.contact_use_src_port is set (by default it is set) and target param transport is TCP/TLS, it will resolve target hostname first, try to connect to that IP using transport with the same IP version as target, and update Contact address with the transport binding address, so if the hostname is resolved to IPv6, Contact address will use IPv6 address.

SIP transport

• If account is bound to a specific transport, it will use that transport
• Otherwise, it will use transport with same IP version as target (resolved address, if it is a hostname)

Media

Media transport is independent from SIP, currently it will simply follow the account setting, i.e: pjsua_acc_config.ipv6_media_use.

Caveat

• When making call using hostname 'localhost' (on Windows 10), it will now resolve to IPv6 address (was to IPv4 address). The solution is to make call using account that is bound to IPv4 transport. The OS seems to by default register 'localhost' as '::1' and '127.0.0.1' and '::1' has higher priority.

As described ​here, we are expecting source compatibility break in this new major version.

Unfortunately AES-GCM feature in libSRTP 1.5.4 (see ticket #1943) has compability issue with OpenSSL 1.1.0, so AES-GCM will be disabled when OpenSSL version being used is 1.1.0 or later. If application wants both (OpenSSL 1.1.0 and AES-GCM), it could use external libSRTP (development version seems to support OpenSSL 1.1.0).

Also just created #1993 for reminder to update libSRTP when a new version is released.

This ticket will enable TURN client to establish TCP connection with the client's peers:

client <--[tcp]--> TURN server <--[tcp]--> peer

The RFC 6062 describes two modes: open outgoing TCP connection to peer and accept incoming TCP connection from peer, this ticket implements only the latter mode, i.e: accept mode.

General flow for accept mode:

1. Client establishes TCP connection to TURN server, this is called control connection.
2. Client sends Allocate request with REQUESTED-TRANSPORT attribute with value set to TCP.
3. Client sends CreatePermission request to allow peer to initiate TCP connection to TURN server.
4. When peer initiates TCP connection to the allocated address on TURN server, client will receive ConnectionAttempt indication with CONNECTION-ID attribute.
5. If client decides to accept the connection, it should establish another TCP connection to TURN server, this connection is for data connection.
6. Client sends ConnectionBind request on that data connection with CONNECTION-ID attribute with value set to the same value in previous ConnectionAttempt indication.
7. Once client receives ConnectionBind success response, the data connection is fully established, the TURN server will simply relay data until any side closes the connection.

How to use

TURN socket application needs to do the following:

1. When calling pj_turn_sock_create(), conn_type must be set to PJ_TURN_TP_TCP.
2. When calling pj_turn_sock_alloc(), param.peer_conn_type must be set to PJ_TURN_TP_TCP.
3. Implement TURN socket callbacks on_connection_attempt and on_connection_status, please check the TURN socket docs for more info.
4. The rest is as usual, e.g: sending data can be done using pj_turn_sock_sendto(), incoming data will be notified via callback on_rx_data.

Thank you Guillaume Roguez and Sébastien Blin from Savoir-faire Linux for the patch.

This ticket will add support to bdSound's bdiMad v2.0.0. Thanks to Francesco Zocchi for the patch.

Currently the PJSUA-LIB-based application will fail if STUN resolution fails, and this seems brittle. We should have secondary STUN options and fail-over to the secondary if the resolution fails.

APS currently only supports 8Khz mono, due to the use of PCMU transcoding

Digest AKA is utilized by 3GPP/IMS, and is specified in RFC 3310: Hypertext Transfer Protocol (HTTP) Digest Authentication Using Authentication and Key Agreement (AKA).

Starting in Mac OS X 10.9, QTKit is deprecated. To disable QT video capture device, you can set PJMEDIA_VIDEO_DEV_HAS_QT to 0.

Add support for writing and reading WAV files encoded as 8-bit A-law/U-law.

Currently using android ndk r12 will fail when executing configure-android due to:

• removal of samples folder (since r11b)
• removal of ndk-build from ndk's root folder

CELT: ​http://www.celt-codec.org/

Backport of ticket #1200 to the 2.0-dev branch.

The IPP codecs to be supported are based on IPP samples speech codec implementations, e.g: G.729, G.723.1, G.726, G.728, G.722.1, AMR.

Support for libyuv library which provides YUV scaling and conversion functionality (​https://chromium.googlesource.com/libyuv/libyuv/).

Requirement

• Libyuv library. For building the library, we recommend using the steps described ​here, except for Android target as explained below.

This implementation will provide video scaling and conversion functionality.

Below are the steps to use the codec:

For GNU targets

1. Build Libyuv library. Make sure to execute the Release build step (ninja -j7 -C out/Release). The library will be built on ([libyuv_dir]/out/Release).
2. Run PJSIP's configure and specify the Libyuv folder using --with-libyuv option. E.g:

   ./configure --with-libyuv=/home/user/src/libyuv
   

For iOS targets

1. Build Libyuv library. Make sure to execute the Release build step (ninja -j7 -C out_ios/Release-iphoneos libyuv_unittest). The library will be built on ([libyuv_dir]/out_ios/Release-iphoneos).
2. Run PJSIP's configure-iphone and specify the Libyuv folder using --with-libyuv option. E.g:

   ./configure-iphone --with-libyuv=/home/user/src/libyuv
   

3. Add libyuv.a to application project in XCode. For armv7 architecture, also add libyuv_neon.a to the application project.

Notes for iOS targets:

• To build for armv7s, just follow the instructions for arm64, which will build libyuv for three architectures: armv7, armv7s, and arm64. armv7s it is still not supported because of the limitation from chromium build tools (documented in libyuv's issue ​here).

For Android targets

1. Get libyuv from ​here (tried to build using the official source, but failed).
2. From the libyuv-android directory, run ndk-build. The library will be built on ([libyuv_dir]/libs/armeabi).
3. Create dir out/Release under [libyuv_dir]/jni and copy the libyuv.so there. E.g:

   cd [libyuv_dir]
   mkdir -p jni/out/Release
   cp libs/armeabi/libyuv.so jni/out/Release
   

4. Run PJSIP's configure and specify the Libyuv folder using --with-libyuv option. E.g:

   ./configure --with-libyuv=[libyuv_dir]/jni
   

For Visual Studio

1. Build the Libyuv library.
2. Add Libyuv include and library location paths to Visual Studio search path settings.
3. Add this to your config_site.h:

   #define PJMEDIA_HAS_LIBYUV		1
   

The updated OpenCORE AMR instructions are in Using OpenCORE AMR NB and WB Codecs

The updated OpenCORE AMR instructions are in Using OpenCORE AMR NB and WB Codecs

IMPORTANT: Starting from release 2.6, OpenH264 1.0 is no longer supported as we move to OpenH264 version 1.6.0 instead. For updated instructions, please see ticket #1947.

Support for the open source implementation of H.264 codec from ​http://www.openh264.org.

Requirement:

• OpenH264 library version 1.0.0 release (download ​here)

Implementation features:

• Constrained Baseline, default level=30 (0x1e)
• packetization modes: 1, 0
• key frame request and indication
• obey remote's fmtp

Notes:

• currently transmission is hard coded to use only single NAL (packetization mode 0) regardless of remote's fmtp for interop reason. The same is also done in ffmpeg wrapper.
• whole frame for file encoding is implemented but currently doesn't work

Below are the steps to use the codec:

For GNU targets:

1. Build the OpenH264 library and install it to a directory (with "make install"). By default it installs to /usr/local. You need to modify the Makefile to install it to different location (=prefix).
2. Run PJSIP's configure. If you specify alternate location (prefix) to OpenH264 installation, specify the location with --with-openh264 option. E.g.:

   ./configure --with-openh264=/home/user/opt
   

3. Add this to your config_site.h:

   #define PJMEDIA_HAS_OPENH264_CODEC		1
   

For Visual Studio:

1. Build the OpenH264 library with MinGW make & gcc, and install it to a directory (with "make install"). By default it installs to /usr/local. You need to modify the Makefile to install it to different location (=prefix).
2. Create openh264.lib from openh264.def, note that the static lib libopenh264.a outputted by MinGW gcc cannot be used by VS, so we are going to use the dynamic lib version instead:
   1. open 'Visual Studio command prompt', e.g: Win Start Menu -> MS VS 2005 -> VS Tools -> VS 2005 Command Prompt
   2. change dir to OpenH264 source root directory
   3. run "lib /def:openh264.def", this will create openh264.lib, copy this file to the OpenH264 installation directory
   4. rename or copy libopenh264.dll to openh264.dll and put it in the system path
3. Add OpenH264 include and library installation paths to Visual Studio search path settings.
4. Add this to your config_site.h:

   #define PJMEDIA_HAS_OPENH264_CODEC		1
   

For iOS:

1. Follow the guidance for GNU targets above.
2. Include libopenh264.a and iOS-SDK's libstdc++.dylib to application project in XCode.
3. Better remove/rename libopenh264.dylib from the OpenH264 installation directory, as somehow it may cause crash on runtime (failed in loading dynamic lib).

For Android:

1. Build the OpenH264 library according to the instructions provided in README.md at the OpenH264 directory. Modify the Makefile to install it to different location (modify the "prefix") and run "make install".
2. Copy libopenh264.so to your Android application library directory (such as libs/armeabi).

Support for the open source implementation of H.264 codec from ​http://www.openh264.org.

Requirement:

• OpenH264 library version 1.6 release (download ​here)
• To avoid conflict, it is recommended to cleanly remove ALL previous/other versions of OpenH264 on your machine, especially if you previously used OpenH264 version 1.0 following the ticket #1758. (If you encounter runtime error openh264.cpp .......SVC encoder Initialize failed, rc=1, it is likely caused by this issue).

Implementation features:

• Constrained Baseline, default level=30 (0x1e)
• packetization modes: 1, 0
• key frame request and indication
• obey remote's fmtp

Notes:

• currently transmission is hard coded to use only single NAL (packetization mode 0) regardless of remote's fmtp for interop reason. The same is also done in ffmpeg wrapper.
• whole frame for file encoding is implemented but currently doesn't work

Below are the steps to use the codec:

For GNU targets:

1. Build the OpenH264 library and install it to a directory (with "make install"). By default it installs to /usr/local. You need to modify the Makefile to install it to different location (=prefix).
2. Run PJSIP's configure. If you specify alternate location (prefix) to OpenH264 installation, specify the location with --with-openh264 option. E.g.:

   ./configure --with-openh264=/home/user/opt
   

3. If you specify an alternate OpenH264 location, add the location path (the path which contains libopenh264*.dylib) to your OS dynamic library path. For example, on Mac:

   export DYLD_LIBRARY_PATH=/Users/teluu/openh264-1.6-mac/lib:$DYLD_LIBRARY_PATH
   

For Visual Studio:

1. Build the OpenH264 library with MinGW make & gcc, and install it to a directory (with "make install"). By default it installs to /usr/local. You need to modify the Makefile to install it to different location (=prefix).
2. Create openh264.lib from openh264.def, note that the static lib libopenh264.a outputted by MinGW gcc cannot be used by VS, so we are going to use the dynamic lib version instead:
   1. open 'Visual Studio command prompt', e.g: Win Start Menu -> MS VS 2005 -> VS Tools -> VS 2005 Command Prompt
   2. change dir to OpenH264 source root directory
   3. run "lib /def:openh264.def", this will create openh264.lib, copy this file to the OpenH264 installation directory
   4. rename or copy libopenh264.dll to openh264.dll and put it in the system path
3. Add OpenH264 include and library installation paths to Visual Studio search path settings.
4. Add this to your config_site.h:

   #define PJMEDIA_HAS_OPENH264_CODEC		1
   

For iOS:

1. Modify the Makefile to install it to a custom folder (find the string PREFIX and modify the value), for example: PREFIX=ios_local. Build the OpenH264 library (example: make OS="ios" ARCH=armv7). Install it to the specified directory (example: make OS="ios" ARCH=armv7 install).
2. Run PJSIP's configure with --with-openh264 option. E.g.:

   ./configure --with-openh264=/Users/teluu/openh264-iphone/ios_local
   

3. Copy libopenh264.a to your XCode project directory. Create a folder ios_local/lib inside your XCode project directory (example: for our ipjsua sample app, the directory will be: (PJPROJECT_DIR)/pjsip-apps/src/pjsua/ios/ios_local/lib). Then copy OpenH264 dynamic library files to the folder. Example:

   cp openh264-iphone/ios_local/lib/libopenh264.a pjproject-iphone/pjsip-apps/src/pjsua/ios/
   cp openh264-iphone/ios_local/lib/*.dylib pjproject-iphone/pjsip-apps/src/pjsua/ios/ios_local/lib
   

4. Add libopenh264.a and iOS-SDK's libstdc++.dylib to your application project in XCode.

For Android:

1. Build the OpenH264 library according to the instructions provided in README.md at the OpenH264 directory. Example for arm64: make OS=android NDKROOT=/home/android-ndk-r13b TARGET=android-15 ARCH=arm64 NDKLEVEL=21 (note that due to the different compiler used (gcc/clang) and system folder structure, using newer NDK may require some changes in the OpenH264 source base). There may be a failure when building the Open264 sample app, but it can be safely ignored since the library itself must have been successfully built by that point.
2. Modify the Makefile to install it to a different location (modify the "prefix") and run "make install". Manually copy libopenh264.so to "prefix"/lib, if necessary.
3. Copy libopenh264.so to your Android application library directory (such as jniLibs/armeabi, or jniLibs/arm64-v8a).

Opus is a totally open, royalty-free, highly versatile audio codec. Opus is unmatched for interactive speech and music transmission over the Internet, but is also intended for storage and streaming applications. It is standardized by the Internet Engineering Task Force (IETF) as RFC 6716 which incorporated technology from Skype's SILK codec and Xiph.Org's CELT codec. (From ​http://www.opus-codec.org)

How to use:

1. Download Opus source code from ​here (latest stable release 1.1.2 or above is recommended).
2. Build Opus:
   1. For iOS, get the build script ​here or ​here.
   2. For Android, get the Android makefile attached in this ticket (tested with Opus 1.1.2), or check ​here
      • create jni folder in Opus root directory and put Android.mk from this ticket attachment in there
      • run ndk-build from Opus root directory
      • make a new folder called opus-dev-lib and create subfolders: include/opus and lib in it
      • copy Opus header files from [OPUS_ROOT_DIR]/include to opus-dev-lib/include/opus and libopus.a file from [OPUS_ROOT_DIR]/obj/local/armeabi to opus-dev-lib/lib
   3. For GNU build system, e.g: *nix, Linux, MacOS, just invoke ./configure, make, and make install.
   4. For Windows:
      • use GNU build tools such as MinGW, or
      • use VS solution & projects provided by Opus (usually in win32 dir), then modify the following line in the end of pjmedia/src/pjmedia-codec/opus.c appropriately:

         #   pragma comment(lib, "libopus.a")
        

1. Run PJSIP's configure script with the option --with-opus=your_opus_installation_dir. For example on Android platform:

   ./configure-android --with-opus=[absolute-path-to]/opus-dev-lib
   

2. Verify if it's a sucess:

   checking for OPUS installations..
   Using OPUS prefix... [absolute-path-to]/opus-dev-lib/
   checking opus/opus.h usability... yes
   checking opus/opus.h presence... yes
   checking for opus/opus.h... yes
   checking for opus_repacketizer_get_size in -lopus... yes
   OPUS library found, OPUS support enabled
   

Opus Codec Settings

General codec settings for this codec such as VAD and PLC can be manipulated through the setting field in #pjmedia_codec_param (see the documentation of #pjmedia_codec_param for more info).

For Opus codec specific settings, such as sample rate, channel count, bit rate, complexity, and CBR, can be configured in #pjmedia_codec_opus_config. The default setting of sample rate is specified in #PJMEDIA_CODEC_OPUS_DEFAULT_SAMPLE_RATE. The default setting of bitrate is specified in #PJMEDIA_CODEC_OPUS_DEFAULT_BIT_RATE. And the default setting of complexity is specified in #PJMEDIA_CODEC_OPUS_DEFAULT_COMPLEXITY.

After modifying any of these settings, application needs to call #pjmedia_codec_opus_set_default_param(), which will generate the appropriate decoding fmtp attributes.

Here is an example of modifying the codec settings:

   pjmedia_codec_param param;
   pjmedia_codec_opus_config opus_cfg;

   pjmedia_codec_mgr_get_default_param(.., &param);
   pjmedia_codec_opus_get_config(&opus_cfg);
   ...
   // Set VAD
   param.setting.vad = 1;
   // Set PLC
   param.setting.vad = 1;
   // Set sample rate
   opus_cfg.sample_rate = 16000;
   // Set channel count
   opus_cfg.channel_cnt = 2;
   // Set bit rate
   opus_cfg.bit_rate = 20000;
   ...
   pjmedia_codec_opus_set_default_param(&opus_cfg, &param);

Please see QoS for the detailed description.

Support for SIP AKAv2 digest authentication as specified by RFC 4169, replacing the existing implementation which seems to be based on the older draft.

Thank you Alex Kolesnichenko for the patch!

Some specs:

1. The following RTCP XR block types shall be supported:
   • Receiver Reference Time Report Block
   • DLRR Report Block
   • Statistics Summary Report Block
   • VoIP Metrics Report Block
2. XR packet source and destination (source -> destination):
   • Receiver Reference Time Report Block: RTP receiver -> RTP transmitter
   • DLRR Report Block: RTP transmitter -> RTP receiver
   • Statistics Summary Report Block: RTP receiver -> RTP transmitter or network management server
   • VoIP Metrics Report Block: RTP receiver -> RTP transmitter or network management server
3. In order to reduce XR traffic, Statistics Summary Report Block and VoIP Metrics Report Block shall always be stacked together and sent out in one RTCP XR packet.
4. The period for sending Statistics Summary Report and VoIP Metrics Report shall be configurable.
5. In addition to periodical reports, at the end of a call or PTT, the RTP receiver shall send out one XR packet containing the final Statistics Summary Report and VoIP Metrics Report.
6. In order to allow enough time to measure round trip delay, the sending of Receiver Reference Time Report Block shall be scheduled to occur at least 500 msec before "each" periodical sending of Statistics Summary Report and VoIP Metrics Report.
7. The destination where Statistics Summary Report and VoIP Metrics Report are sent shall be configurable:
   • either the RTP transmitter or network management server.
8. It shall be configurable to use either a fixed assigned number or a randomly generated number for the 32-bit RTP Synchronization Source (SSRC) used to identify each RTP end point.
   • In a closed system (such as P25 and OpenSky), it is preferred to assign each device with a unique SSRC (a combination of device type and device ID). The advantages with assigned SSRC are that SSRC alone can be used to identify RTP end points and no two end points will use the same SSRC.
   • With randomly generated SSRC, network management server will not know the end points of XR reports unless it also receives other info such as RTCP SDES packets.
9. The following fields in the Statistics Summary Report Block shall be supported:
   • begin_seq
   • end_seq
   • lost_packets (in the above sequence number interval)
   • dup_packets (in the above sequence number interval)
   • min_jitter (in the above sequence number interval)
   • max_jitter (in the above sequence number interval)
   • mean_jitter (in the above sequence number interval)
   • dev_jitter (in the above sequence number interval)
10. The following fields in the VoIP Metrics Report Block shall be supported:
    • loss rate (accumulated since the beginning of a call)
    • discard rate (accumulated since the beginning of a call)
    • burst density (accumulated since the beginning of a call)
    • gap density (accumulated since the beginning of a call)
    • burst duration (accumulated since the beginning of a call)
    • gap duration (accumulated since the beginning of a call)
    • round trip delay (the most recently measured)
    • end system delay (= JB nominal delay + 2 * audio frame size per RTP packet)
    • Gmin (= 16)
    • RX config (= PLC + adaptive or fixed jitter buffer)
    • JB nominal (= initial jitter buffer delay if fixed jitter buffer is used)

In line with our roadmap for WebRTC interoperability, as mentioned in this draft of ​Web Real-Time Communication (WebRTC) document:

To reduce these costs and session set-up times,
   implementations are REQUIRED to support multiplexing RTP data packets
   and RTCP control packets on a single transport-layer flow.

this ticket will add support for RTP and RTCP multiplexing, in accordance to ​RFC 5761.

Add support for Message Summary/Message? Waiting Indication (MWI, RFC 3842). Two types of MWI events are supported: MWI with subscription and unsolicited MWI (unsolicited means NOTIFY is sent without subscription).

Specification:

• implement pjsip evsub package for MWI
• client and server support
• unsolicited MWI support
• support in PJSUA-LIB
• NOTIFY message body is unprocessed; they will be given to app unparsed.

New files in pjsip-simple:

• mwi.h
• mwi.c

Support in PJSUA-LIB:

• "mwi_enabled" field in pjsua account config (default is disabled)
• "on_mwi_info" callback
• "enable_unsolicited_mwi" in pjsua_config (default is enabled)
• if MWI subscription is terminated for some reason, PJSUA-LIB will renew periodically (PJSUA_PRES_TIMER timer)

PJSUA application:

• new "--mwi" option
• print MWI info on the callback

To implement MWI support in your application:

1. implement on_mwi_info() callback
2. at this point, any unsolicited incoming NOTIFY requests containing MWI will cause on_mwi_info() callback to be called, unless enable_unsolicited_mwi is explicitly disabled in pjsua_config.
3. if you'd like to use MWI subscriptions, set mwi_enabled to TRUE in pjsua_acc_config

Scope:

• Run-time configurable session interval (Session-Expires header) and minimum session interval (Min-SE header) per account basis.
• Refresher performs refreshes at half of session interval, and refreshee sends BYE slightly before session expiration. 'Slightly' == minimum of 32 seconds and one-third of session interval.
• Session interval negotiation, i.e: generate/handle 422 when SE less than Min-SE.
• Works even when only one UA in a dialog supports Session Timer (and it's not listed in Require header).
• Able to use Session Timer even when peer doesn't support Session Timer, perform refreshes by and for itself.
• Able to adapt when Session Timer is turned-off.

Limitations:

• Refreshing session using UPDATE hasn't been supported yet.
• Refreshing session will always restart the media session (even if the media session is not modified).
• Session Timer can always be listed in Supported header, but it's not necessarily always used (by omitting Session-Expires header), currently there is no mechanism to omit/turn-off Session-Expires header.

SIP UPDATE is specified in RFC 3311 for updating session parameters without affecting dialog's state.

Also to be fixed in this development is the offer/answer negotiation in pjsip's invite session, since with the support for UPDATE (and PRACK) there are more offer and answer scenarios to support.

This ticket implements SIP outbound (​RFC 5626), which is very useful in assisting NAT traversal, and Path extension (​RFC 3327) which is required for the outbound extension.

Please see the documentation in SRTP wiki page.

Please see ​third_party/build/samplerate/README.txt in pjsip distribution.

The Service-Route header is used for routing requests in 3GPP/IMS network. UAC would construct the initial route-set based on the URI listed in Service-Route headers.

This support will be done in PJSUA-LIB level.

Add support for VP8 and VP9 video codec using ​libvpx.

During our testing, for the same bitrate, VP8 quality seems to be better than H264, but the encoding speed is noticeably slower. For VP9 however, as documented by several other sources as well [1][2], the encoding speed is much much slower, even though we have followed the recommended setting for "Real-time CBR Encoding and Streaming" from ​their doc here. Because of this, VP9 is currently by default disabled. To enable it, you can set PJMEDIA_HAS_VPX_CODEC_VP9 compile-time macro.

[1] ​https://blogs.gnome.org/rbultje/2014/02/22/the-worlds-fastest-vp9-decoder-ffvp9/ [2] ​https://www.streamingmedia.com/Articles/ReadArticle.aspx?ArticleID=111334

Steps to use VP8/VP9

1. Download ​libvpx.
2. Build and install. General build steps can be found in libvpx README file.
   • For Android, these steps can be used:

     export NDK=/path/to/Android-NDK
     export HOST_TAG=linux-x86_64 # adjust to your building host
     
     export TOOLCHAIN=$NDK/toolchains/llvm/prebuilt/$HOST_TAG
     export AR=$TOOLCHAIN/bin/aarch64-linux-android-ar
     export AS=$TOOLCHAIN/bin/aarch64-linux-android-as
     export CC=$TOOLCHAIN/bin/aarch64-linux-android21-clang
     export CXX=$TOOLCHAIN/bin/aarch64-linux-android21-clang++
     export LD=$TOOLCHAIN/bin/aarch64-linux-android-ld
     export RANLIB=$TOOLCHAIN/bin/aarch64-linux-android-ranlib
     export STRIP=$TOOLCHAIN/bin/aarch64-linux-android-strip
     
     ./configure --target=arm64-android-gcc --enable-pic --disable-examples \
                 --disable-tools --disable-docs --enable-realtime-only \
                 --disable-install-bins --disable-unit-tests --disable-webm-io \
                 --disable-libyuv \
                 --prefix=/path/to/target/installation
     

3. Libvpx will then be autodetected by PJSIP configure script (alternatively, you can specify --with-vpx option).

Voice Processing IO Audio Unit is considered to provide nice features such as echo cancellation & AGC, it is supported since Mac OSX 10.7.

Our internal testing indicates that the EC performance is far more superior than any software EC. However, VPIO seems to work only with specific devices, thus requiring you to use the default/built-in audio I/O devices (i.e. it doesn't allow you to select your own devices). Application wishing to disable the hardware EC and use software EC instead can specify PJMEDIA_ECHO_USE_SW_ECHO in pjsua_media_config.ec_options (or MediaConfig.ecOptions in pjsua2).

*Limitation: only works in mono. *

How to use WebRTC AEC

To use WebRTC AEC, specify the value PJMEDIA_ECHO_WEBRTC in pjsua_media_config.ec_options (MediaConfig.ecOptions for PJSUA2), and specify the delay estimate for sound card and system buffers in pjsua_media_config.ec_tail_len (MediaConfig.ecTailLen for PJSUA2). WebRTC AEC's performance is very dependent on this delay calculation. A poor estimate, even by as little as 40ms, may affect the echo cancellation results greatly. Our WebRTC wrapper has enabled the delay agnostic feature, which will adjust the delay accordingly, however it may take some time (5-10s or more) for the AEC module to learn the optimal delay, thus a good initial estimate is necessary for good EC quality in the beginning of a call.

The delay should be at least as much as frame time (by default, ptime is 20ms) + PJMEDIA_WSOLA_DELAY_MSEC (the default is 5ms).

In PJSUA sample application, run it with the options --ec-opt=3 and specify the delay estimate in --ec-tail options. For example:

./pjsua --ec-opt=3 --ec-tail=30

Poor WebRTC EC quality

Disable PJMEDIA_WEBRTC_AEC_USE_MOBILE (set it to 0), then change the definition of SHOW_DELAY_METRICS in pjmedia/src/pjmedia/echo_webrtc.c to a non-zero value. For example, if you set to 250, with ptime 20ms, it will print the delay metrics stats every 5 seconds or so.

#define SHOW_DELAY_METRICS	250

What you need to see in the log is the line:

median=%d, std=%d, fraction of poor delays=%f

Fraction of poor delays is a value between 0 and 1. The closer the value to 1, the poorer the EC quality.

To improve the EC quality, here are a few things you can try:

• Adjust the EC tail value to the median value.
• Activate noise suppression (pass the options when creating EC with flag PJMEDIA_ECHO_USE_NOISE_SUPPRESSOR).
• Increase EC aggressiveness (pass the options when creating EC with flag PJMEDIA_ECHO_AGGRESSIVENESS_AGGRESSIVE).
• Disable the mobile version by setting PJMEDIA_WEBRTC_AEC_USE_MOBILE to 0 (by doing this, AEC will operate in floating-point mode (instead of integer), which may decrease performance).

Problem with EC quality on Mac

The problem with echo on Mac and negative delay has been reported by several of our users and seems to be a known issue, as documented in ​Chrome M45 WebRTC Release Notes and ​Issue 3929: AEC on Mac can be put in non-causal state. Unfortunately, there's no known workaround as of now.

*Starting from release 2.6, the build instructions below is no longer required since WebRTC has been integrated into third_party. For more details, please refer to ticket #1954 instead.*

How to build WebRTC

Mac

Refer to WebRTC development guide ​here.

Then configure PJSIP with:

./configure --with-webrtc=/Users/me/webrtc-mac

iOS (ARM)

Refer to WebRTC development guide ​here.

PJSIP's auto configuration will look for the library in out_ios/Release folder, so make sure you set the output dir properly when building !WebRTC

export GYP_GENERATOR_FLAGS="output_dir=out_ios"
ninja -C out_ios/Release

Then configure PJSIP with:

./configure --with-webrtc=/Users/me/webrtc-iphone

Android

Android development is only supported on Linux (reference ​here), so if you use other platforms, you need to set up a Linux virtual machine.

Alternatively, you can build the stripped down version of WebRTC instead, which will only build the required AEC module and its required dependencies. These steps below are tested on a Mac machine:

1. Create a working directory, for example: webrtc-android.
2. Go to the work dir and unzip webrtc-android-jni.zip (provided in the ticket attachment below).
3. Go to jni folder and run ndk-build.
4. Copy the resulting .so files from ../libs/[target_architecture] into your Android application project directory, for example:

   cp ../libs/armeabi/*.so /Users/me/pjproject/pjsip-apps/src/swig/java/android/libs/armeabi
   

5. Add this in PJSIP's user.mak, located inside PJSIP's work dir:

   export CFLAGS += -I"/Users/me/webrtc-android/src" -DPJMEDIA_HAS_WEBRTC_AEC=1 -DPJMEDIA_WEBRTC_AEC_USE_MOBILE=1
   export LDFLAGS += -L"/Users/me/webrtc-android/libs/armeabi" -lwebrtc
   

   Rebuild PJSIP.

For other architectures

(or custom WebRTC installation undetected by configure script)

You can customize PJSIP's user.mak file, located inside PJSIP's work dir:

export CFLAGS += -I"/Users/me/webrtc/src" -DPJMEDIA_HAS_WEBRTC_AEC=1
# Uncomment below if you want to use the lighter, mobile-optimized version
# export CFLAGS += -DPJMEDIA_WEBRTC_AEC_USE_MOBILE=1
export LDFLAGS += -L/Users/me/webrtc/[WebRTC_lib_path] -laudio_processing -lcommon_audio -lsystem_wrappers
# If you build for ARM architecture, uncomment below
# export LDFLAGS += -laudio_processing_neon -lcommon_audio_neon
# Or else, if you build for MIPS architecture, uncomment below
# export LDFLAGS += -laudio_processing_neon -lcommon_audio_neon
# Or else, if your machine supports SSE2, uncomment below
# export LDFLAGS += -laudio_processing_sse2

Then rebuild PJSIP.

Compatibility issue

There is a known compatibility issue with recent WebRTC version, so we'd recommend to use an older WebRTC version, e.g: about October 2015 version.

Support for Win64, including Visual Studio projects.

Currently some SDP optional lines are ignored by parser and there is no API to add these lines, while some users may need it, e.g: ​Peter Cai needs 'b=' line

Add support in PJSIP and PJSUA-LIB to allow applications to add custom headers in presence subscription requests.

In PJSUA-LIB, this will be implemented with this new field in pjsua_acc_config:

pjsip_hdr       sub_hdr_list; 

When remote has mode=20 and local has mode=30, local encounters error:

 08:27:23.648   strm00F496F4 Codec encode() error: Invalid PCM frame length 
(PJMEDIA_CODEC_EPCMFRMINLEN) [err:220085]

Initial hints for the implementation can be found ​here for OpenSL and ​here for Android JNI.

Support for G729 backend implementation from ​bcg729.

Requirement:

• Source code to bcg729, (download ​here).

Notes:

• The source code of bcg729 is licensed under GPLv2.

Below are the steps to use the codec:

For GNU targets:

1. Build the bcg729 library as specified by the their doc.
2. Run PJSIP's configure by specifying the location to bcg729 library using --with-bcg729 option. E.g.:

   ./configure --with-bcg=/home/user/opt
   

3. Make sure that the library is usable and correctly configured by looking at the configure output. The check is applicable for iOS and android build as well.

   Using bcg729 prefix... /Users/riza/src/bcg729/ios
   checking bcg729 usability... ok
   

For Visual Studio:

1. Build the bcg729 library using Visual Studio, by generating the project using cmake.
2. Add bcg729 include and library installation paths (folder that contains libbcg729.dll.a) to Visual Studio search path settings.
3. Add this to your config_site.h:

   #define PJMEDIA_HAS_BCG729		1
   

For iOS:

1. Build the bcg729 library as specified by the their doc.
2. Run PJSIP's configure with --with-bcg729 option. E.g.:

   ./configure-iphone --with-bcg729=/Users/teluu/bcg729/iphone
   

3. Copy libbcg729.a to your XCode project directory. Create a folder ios_local/lib inside your XCode project directory (example: for our ipjsua sample app, the directory will be: (PJPROJECT_DIR)/pjsip-apps/src/pjsua/ios/ios_local/lib). Then copy bcg729 dynamic library files to the folder. E.g:

   cp bcg729/lib/libbcg729.a pjproject-iphone/pjsip-apps/src/pjsua/ios/
   cp bcg729/lib/*.dylib pjproject-iphone/pjsip-apps/src/pjsua/ios/ios_local/lib
   

4. Add libbcg729.a to your application project in XCode.

For Android:

1. Patch Android.mk in the bcg729 source root directory, just append src/cng.c, src/dtx.c, src/vad.c to LOCAL_SRC_FILES.
2. Build the bcg729 library, e.g:

   cd [bcg729-source-path]
   $ANDROID_NDK_ROOT/build/ndk-build APP_ABI=arm64-v8a \
           NDK_PROJECT_PATH=. APP_BUILD_SCRIPT=Android.mk
   

3. Create directory lib under bcg729 library source, and copy the libbcg729.a from [bcg729-source-path]/obj/local/[ABI] to it.
4. Run PJSIP's configure with --with-bcg729 option. E.g.:

   ./configure-android --with-bcg729=/Users/teluu/bcg729/android
   

Some platforms have an optimized method to switch video devices while the session is running without causing long delay or stuttering. For example, iPhone 4 allows switching between front and rear cameras while the capture session is running.

This ticket implements a capability to allow video stream to switch it's underlying device while it is running. The capability should support both capture and render device.

Reference:

• ​http://developer.apple.com/library/ios/documentation/AudioVideo/Conceptual/AVFoundationPG/Articles/03_MediaCapture.html#//apple_ref/doc/uid/TP40010188-CH5-SW24

On iOS, application can now set the audio I/O latency by using PJMEDIA_AUD_DEV_CAP_INPUT_LATENCY / PJMEDIA_AUD_DEV_CAP_OUTPUT_LATENCY, which can set the latency (via Audio Session's property kAudioSessionProperty_PreferredHardwareIOBufferDuration) to as low as 5 ms.

This ticket implements support for multipart message bodies, with the following specification.

Multipart API

New APIs will be added to create, parse, and manipulate multipart message bodies. These APIs will reside in a new header file, <pjsip/sip_multipart.h>.

Parser Specification

Multipart message bodies in the incoming message will be parsed automatically. This is controlled by a new setting, PJSIP_AUTO_PARSE_MULTIPART, which default value is TRUE. If this setting is set to FALSE, the parser will revert to the old behavior, i.e. it will not automatically parse the multipart bodies in the incoming messages.

Note that the parser still will not parse the content of the multipart bodies. For example, consider the following example:

Content-Type: multipart/mixed;boundary=xxxxx

--xxxxx
Content-Type: application/sdp

v=0
o=- 3483093653 3483093653 IN IPv4 10.0.0.1
s=pjmedia

c=IN IP4 10.0.0.1
t=0 0

m=audio 4000 RTP/AVP 0
--xxxxx
Content-Type: text/plain

Hello please answer my call
--xxxxx--

With the above message, the parser recognizes that the message contains multipart bodies, so it will parse it into a multipart instance. The multipart instance will contain two body parts, application/sdp and text/plain. The parser however, will not parse the parts further (the parsing of the application/sdp content will be done by the invite session).

Multipart Handling in PJSIP Layer

These currently only applies to the invite session (pjsip_inv_session).

Incoming messages

The session will recognize multipart message bodies in incoming messages, and be able to look for the appropriate part in the bodies.

Outgoing messages

There will be no change in outgoing message API. If higher layer components (e.g. PJSUA-LIB) want to send multipart body, it may alter the transmit buffer before sending the message.

Multipart Handling in PJSUA-LIB

The following will describe the following pattern that will be used in PJSUA-LIB to support multipart message bodies.

Incoming messages

There will be no change in the handling of incoming messages containing multipart bodies, as they will be processed by the underlying session (e.g. invite session).

If application wants to extract the multipart content from an incoming message, it can inspect the message from the rdata or event parameter of the appropriate callback, recognize the multipart content from the message body's content type, and use the multipart API to extract the parts.

Outgoing messages

New fields will be added to the pjsua_msg_data:

struct pjsua_msg_data
{
    ...

    /** Content type of the multipart body */
    pj_str_t  multipart_ctype;

    /** List of multipart parts */
    pjsip_multipart_part parts;
};

If application wants to send multipart contents, it constructs the parts it intends to send, put them in the parts list above, and set the multipart_ctype appropriately (e.g. to "multipart/mixed").

Add support for multipart message bodies in incoming NOTIFY requests. With this ticket, PJSIP will search for "application/pidf+xml" or "application/xpidf+xml" in the multipart bodies.

This extends the general multipart support as implemented by #1070.

Although this is not recommended by the spec, some (IMS?) servers don't like the Route header, so it will be a good idea if PJSIP is able to route the request to the server without having to put Route header.

One idea to accomplish this is to utilize an proprietary/internal parameter in the Route header (e.g. "hide" parameter). By using this, PJSIP would process the route set as usual, but the Route header will not be printed to the request.

Previously the parser only supports parsing simple integer value inside Retry-After header and will barf if the header contains comment. Thanks Samuel Vinson and Nigel Hsiung for reporting.

Support parsing quoted arguments, for example:

  --add-buddy '"My Buddy" <sip:mybuddy@theirprovider.com>'

The arguments can be quoted with both single or double quotes.

Thanks Scott M Ober for the patch.

Some endpoint (SNOM SoftPhone in this case) sends RTP packets with no payload, presumably for keep-alive. This caused PJMEDIA to report PJMEDIA_RTP_EINLEN (Invalid RTP packet length) error, while it should be accepted.

Currently pjmedia expects than callee sends RTP packet using the codec specified as the first in format list in SDP answer. This is not always the case of course, as SIP allows callee to send multiple codecs in the SDP answer and send RTP using any of it, so this is a perfectly valid scenario.

This feature adds support for pjmedia to refresh its audio device list, thereby allowing application to use/stop using audio devices that are plugged/unplugged when the application is running.

Application would still need to do its own implementation to detect audio devices plug/unplug event. Then it can call the function pjmedia_aud_dev_refresh().

Note that this feature is not implemented on PortAudio, legacy, and all Symbian sound backends.

This feature adds support for pjmedia to refresh its video device list, thereby allowing application to use/stop using video devices that are plugged/unplugged when the application is running.

Application would still need to do its own implementation to detect video devices plug/unplug event. Then it can call the function pjmedia_vid_dev_refresh().

This is duplicate of ticket #13 for trunk.

The RTCP framework should send RTCP RR when it's not transmitting anything.

Currently the transmission of RTCP was driven by transmission of RTP, so when an endpoint is only actively listening (and not sending anything, although stream direction is send-recv), then no RTCP packet is sent.

Add PJMEDIA_AUD_DEV_CAP_INPUT_SOURCE capability (as an alias of PJMEDIA_AUD_DEV_CAP_INPUT_ROUTE) to support changing audio input source. The value of this capability is PJMEDIA_AUD_DEV_ROUTE_CUSTOM + the_desired_audio_input_source_constants.

For example:

// 6 is VOICE_RECOGNITION
unsigned aud_source = PJMEDIA_AUD_DEV_ROUTE_CUSTOM | 6;
pjmedia_aud_param_set_cap(&param, PJMEDIA_AUD_DEV_CAP_INPUT_SOURCE,
			  &aud_source);

Valid audio input source values can be found in ​Android doc here.

Thanks to Johan Lantz for the suggestion.

Currently PJSIP SIMPLE only supports basic online/offline status.

The proxy sample barf again when handling strict-routed request, this time because it thinks that the request is destined for it thus it rejects the request with 404/Not Found.

While PJSIP has been supporting tel: URI for a long time, support for this is currently disabled in PJSUA-LIB, mainly for cosmetic reason.

This ticket enables tel: URI support in PJSUA-LIB, according to the following specification:

1. tel: URI in account ID shall be supported, and the tel: URI shall be used as the URI in From header for all requests originated from that account, such as :
   • client registration
   • outgoing calls (including re-INVITE and UPDATE)
   • outgoing REFER
   • outgoing MESSAGE
   • presence PUBLISH (can't do this unless route set is configured)
   • outgoing SUBSCRIBE
   • and any other requests
2. shall be able to send requests to tel: URI from an account, as long as route set is configured for the account. This includes, but not limited to:
   • outgoing calls (including re-INVITE and UPDATE)
   • outgoing REFER
   • outgoing UPDATE
   • outgoing MESSAGE
   • outgoing SUBSCRIBE (can't add tel: URI as buddy)
   • and any other requests (although sending REGISTER and PUBLISH to a tel: URI is probably not appropriate)
3. shall be able to receive incoming requests with tel: URI in Request URI and/or To header of:
   • incoming calls
   • incoming MESSAGE
   • incoming REFER
   • incoming SUBSCRIBE
   • and any other requests

The unit tests shall be updated with above test scenarios.

The latest Intel IPP 6.1 has changed few things which is incompatible with some of our build scripts (especially Linux).

Also since the latest IPP supports Mac OS X, add this as the supported IPP platform too.

Add support for using different PortAudio, Speex, and/or GSM library installation than the one in the third_party directory. This is useful to make use of specific features in the system that have been configured in the installed library. For example, PortAudio library that comes with OpenBSD has support for sndio, which is not normally available/enabled in our PortAudio.

New ./aconfigure options are to be implemented:

--with-external-speex   Use external Speex development files, not the one in
                        "third_party" directory. When this option is set,
                        make sure that Speex is accessible to use (hint: use
                        CFLAGS and LDFLAGS env var to set the include/lib
                        paths)
--with-external-gsm     Use external GSM codec library, not the one in
                        "third_party" directory. When this option is set,
                        make sure that the GSM include/lib files are
                        accessible to use (hint: use CFLAGS and LDFLAGS env
                        var to set the include/lib paths)
--with-external-pa      Use external PortAudio development files, not the
                        one in "third_party" directory. When this option is
                        set, make sure that PortAudio is accessible to use
                        (hint: use CFLAGS and LDFLAGS env var to set the
                        include/lib paths)

Currently there is no mechanism to notify other party about new video orientation, so in this ticket, we implement a feature which allows the video sender to transmit video with the "proper" orientation at all time, i.e: head upside whatever the device orientation is. When a device orientation changes, the application can tell the library and the library will rotate the video, resize it accordingly, and insert blank/black in left+right sides. The receiver/decoder side will then receive the video in the correct orientation, nothing to do on the decoder side.

The feature is available on iOS and Android only.

Currently pjsua will always open both speaker and microphone. Some applications, such as Push-To-Talk, may have microphone off most of the time and prefer to open only the speaker.

While working on pjsua2 sound device, there seem to be a related issue, calling setPlaybackDev() & setCaptureDev() consecutively may not work as expected, i.e: for changing both playback and record devices, perhaps due to asynchronous dev open/close operations, this is experienced on Linux platforms. So, perhaps we should consider to add an API to set both at once, e.g: setPlaybackAndCaptureDev().

Currently only add and remove bindings are supported.

To perform registration query, create a registration client without registering any contacts, and set the "renew" parameter of pjsip_regc_register() to 0.

Thanks Roman Puls for the suggestion.

From the ​RFC 7587:

The Opus encoder can output encoded frames representing 2.5, 5, 10,
   20, 40, or 60 ms of speech or audio data.  Further, an arbitrary
   number of frames can be combined into a packet, up to a maximum
   packet duration representing 120 ms of speech or audio data.

For example, for a packet duration of 40ms, we can receive 1 frame of 40ms, 2x20ms, 4x10ms, etc. However, currently pjmedia only expects constant frame length to be specified during stream creation.

Thank you to Marcus Froeschl for the suggestion and patch testing.

Currently pjsip_inv_update() requires SDP offer param, while some applications may need to send UPDATE without SDP, e.g: session timer.

Add support to generate a synthesized IPv6 address from IPv4 address. The ability to synthesize IPv6 addresses was added to getaddrinfo in iOS 9.2 and OS X 10.11.2 (reference: ​Apple's doc).

Thanks to Jamil Abdullayev for the patch.

Currently, there are a couple of problems when making video only call (aud_cnt = 0) with pjsua:

• sound device still opened
• incoming video-only call will be rejected with 488

This ticket will fix these issues.

Currently pjsua_vid_win API will return PJ_EINVAL if the video capture preview is native, such as on iOS. This means that application has to manually get the native window handle and use the OS API directly. However, this could be problematic, for example on iOS, the UIView window also contains a preview layer which is not exposed to the application. Hence, resizing the window container itself (by calling the iOS API) will not resize the preview layer itself.

Thus, since the underlying pjmedia videodev implementation already supports such capabilities, this ticket will enchance pjsua_vid_win API to support them as well.

PJLIB has been attempting to suppress the delivery of SIGPIPE when send() is called on a broken TCP socket, by applying SO_NOSIGPIPE to the socket. Unfortunately this option is not available on Linux.

This ticket adds the same functionality by adding MSG_NOSIGNAL flag to send() if this flag is detected (declared).

Support for Symbian ^{3 platform.}

Report and solution provided by Gabor Tanka can be found in ​this thread.

Found that kern-exec 0 is caused by access to RHostResolver instance for IPv6, while RHostResolver for IPv6 is also not opened/initialized when PJ_HAS_IPV6 is not set.

Implement sound device implementation for Symbian/S60 3rd edition platform.

The Symbian timer heap implementation uses 32bit integer value to calculate the interval. Since RTimer.After() function takes microsecond interval as 32bit integer too, this causes the timer heap to unable to schedule more than ~2100 seconds duration.

If caller tries to schedule more than ~2100 seconds duration, the timer would be triggered immediately, causing e.g.:

• PUBLISH refresh to be sent continuously, if server replies with "Expires: 3600"

Answerer SHOULD use matching payload type as the offer:

• RFC 3264 6.1:

In the case of RTP, if a particular codec was referenced with a specific payload type number in the offer, that same payload type number SHOULD be used for that codec in the answer.

• RFC 6184:

To simplify the handling and matching of these configurations, the same RTP payload type number used in the offer SHOULD also be used in the answer, as specified in [8]. An answer MUST NOT contain the payload type number used in the offer unless the configuration is exactly the same as in the offer.

Note:

• Implemented in SDP negotiator (instead of in endpoint as the preanswer generator), as the codecs need to be negotiated first.
• Initiating re-INVITE will revert the payload type numbers to the pjmedia defaults, which should be fine. If this is not acceptable, a simple solution will be providing alternatives for SDP generation for the re-INVITE:
  • using active local SDP,
  • no SDP.

Some video devices implementation (in pjmedia-videodev) assumes that the application has been initialized in certain way in order for it to work. For example, the SDL and QuickTime video device implementation on Mac OS X requires NSApplication and NSAutoreleasePool to be present in the application.

That normally is not a problem if the application is GUI application created with IDE such as XCode, since the IDE will create those for us. But for application created manually, such as console apps that are common in our samples, these objects have to be created manually.

This ticket implements a generic way to initialize system specific settings to the application. This new API in pj/os.h is proposed:

/* Type for main function. */
typedef int (*pj_main_func_ptr)(int argc, char *argv[]);

/**
 * Run the application. This function has to be called in the main thread
 * and after doing the necessary initialization according to the flags
 * provided, it will call main_func() function.
 *
 * @param main_func Application's main function.
 * @param argc	    Number of arguments from the main() function, which
 * 		    will be passed to main_func() function.
 * @param argv	    The arguments from the main() function, which will
 * 		    be passed to main_func() function.
 * @param flags     Flags for application execution, currently must be 0.
 *
 * @return          main_func()'s return value.
 */
int pj_run_app(pj_main_func_ptr main_func, int argc, char *argv[],
	       unsigned flags);

Console apps need to call the above pj_run_app() API, passing it's main routine as the argument.

Since the timer heap time calculation uses wall clock source, changes in the system time (for example because of DST, network time synch, time change, etc.) will cause the timer entries to be scheduled wrongly.

The solution is to use tick count based time which should be immune from the time change problem.

Thanks Phil Torre for the report.

TCP keep-alives packets are not dropped by the stack, and although the parser should be able to cope with leading CRLF's, pjsip_find_msg() doesn't.

The transport layer should drop these leading CR-LF bytes before parsing is invoked.

Thanks Helmut Wolf for reporting this.

When TCP connection is closed by remote, it will be marked as being shutdown so that it will not be used by subsequent requests. This is the correct behavior.

When application needs to send another message to the destination, the transport manager will see that there is a transport object for the destination, but this transport object is being shutdown, so the transport manager will create a new transport for the same destination. This is also the correct behavior.

But unfortunately the new transport created will have the same key as the transport being shutdown, so it will overwrite the entry in the hash table. Then the first transport is finally destroyed, it will set the hash table key to NULL, and this causes the second transport's entry to be inadvertly nullified.

Issues:

• Domain name verification, IETF RFC draft could be found ​here.
• Add callback in PJSUA-LIB to notify transport state (e.g. transport is connected or disconnected). There are several use cases for this:
  • application send re-REGISTER when the connection is down. This should reconnect the transport.
  • application can inspect TLS connection/verification error and reconfigure TLS transport respectively (for example, to disable domain verification if TLS connection fails due to certificate error (perhaps after asking for user confirmation))

Specifications:

• Verification in SSL socket:
  • If verify_peer is set, SSL socket will disconnect the connection whenever there is any error occurred in the verification.
  • If verify_peer is not set, SSL socket will ignore any error in verification and continue establishing SSL handshake. Application may inspect the verification result in the SSL socket info or do further verification (e.g: domain name verification) in on_connect_complete() callback and decide whether SSL connection will be closed.
  • On both cases, the application may inspect the verification result in the SSL socket info.
  • The verification result in SSL socket info should show all verification errors, for example when remote certificate is expired and untrusted, the verification result should show both errors.
  • As domain name verification rules may vary among applications (e.g: SIP requires exact match and allows SIP URI identity, while HTTP allows wildcard, '*' and doesn't not say anything about URI identity), domain name verification is in application level and the SSL socket will just provide remote identities parsed from remote certificate, i.e: subjectAltName extension and subject common name.

• Verification in SIP TLS transport:
  • Add destination host name into pjsip_tx_data. The TLS transport will use it to get the server name for TLS verification.
  • Introduce a new SIP transport callback to notify transport states, i.e: connected, disconnected.
  • Add a verification step, i.e: domain name verification, then include its result to verification status of SSL socket info.
  • If verify_server/client is set and there is TLS verification error, TLS transport will disconnect the transport and notify the application via the new callback with event type 'disconnected', application can examine disconnection status code and TLS verification result via transport state info.
  • If verify_server/client is not set and there is TLS verification error, TLS transport will notify the application via the new callback with event type 'connected', application can examine TLS verification result via transport state info.

Implement TLS support for PJSIP

Issues:

1. OpenSSL error code range is too wide (consist of 3 fields: lib, func, reason) to fit into PJLIB error numbering space, so currently, only lib and reason error codes are kept/packed in pj_status_t using this formula: (lib_err*300+reason_err), but the 'shifter' 300 turns out to be too small and caused ambiguity.
2. Export OpenSSL error code to PJSIP/PJSUA level. This can be useful as a backup or alternative solution of point 1 issue above.
3. Misc error code related bugs:
   • avoid multiple error code retrievals on a failure, the latest retrieval will get zero.
   • should immediately retrieve more error code for details after getting SSL_ERROR_SSL error.
   • server socket: after handshake error, OpenSSL may need to write something first (e.g: notify error to client socket), so better flush write BIO first before closing the TCP socket.
   • client socket: after getting PJ_EEOF (server initiated disconnect), better check for OpenSSL error code, if any, replace the PJ_EEOF with OpenSSL error code.

Known issues:

1. pjsip_tls_transport_start() won't apply ca_list_file when cert_file is not set (thanks Tal Fromm for the report).
2. lis_create_transport() doesn't apply cert settings (thanks Tal Fromm for the report).
3. Update cipher type in pjsip_tls_setting to follow the new SSL socket abstraction.
4. Also export the cipher setting to pjsua app option (thanks Tal Fromm for the suggestion).

Implement TLS transport for TURN protocol, using the new PJLIB SSL transport object.

The TURN client fails to retry authentication after it receives several 438 responses during an allocation session. It fails with this message:

6:29:02.950 udprel0x81ea28  Error: authentication failed (too many retries)

This is because the sess->auth_retry variable is not properly reset after it receives successful response, in the STUN session.

Thanks Peter Livesey for the report!

Any parameters set in pj_turn_alloc_param is ignored by pj_turn_session

The corresponding ticket for 1.0.x branch is #901

This is duplicate of ticket #889 for 1.0.x branch.

When TURN is configured with bad server name, couple of error scenarios will occur. For example, configure pjsua with the following:

--use-ice
--use-turn
--turn-srv turn.baddomain.com

Scenario 1:

Crash will occur with the above config.

Scenario 2:

if nameserver is configured (i.e add --nameserver cmdline option containing your local router IP address), the crash will not happen, but the application is not notified that the call has failed (i.e on_call_state() callback is not called).

After media subsystem is shutdown, events are not polled, causing TURN session pending destroy events to be unprocessed.

It crashed upon receiving re-INVITE or UPDATE. This problem is to be fixed by #1358

RFC 3265 states:

If the NOTIFY request fails (as defined above) due to a timeout
condition, and the subscription was installed using a soft-state
mechanism (such as SUBSCRIBE), the notifier SHOULD remove the
subscription.

If a NOTIFY request receives a 481 response, the notifier MUST remove
the corresponding subscription even if such subscription was
installed by non-SUBSCRIBE means (such as an administrative
interface).

A NOTIFY request is considered failed if the response times out, or a
non-200 class response code is received which has no "Retry-After"
header and no implied further action which can be taken to retry the
request (e.g., "401 Authorization Required".)

If the NOTIFY request fails (as defined above) due to an error
response, and the subscription was installed using a soft-state
mechanism, the notifier MUST remove the corresponding subscription.

Based on the above statements, subscription will be terminated after receiving these Notify response:

1. 408
2. 481
3. Non 2xx response without "Retry-After" header

This ticket will apply the third condition for terminating the subscription.

Thanks to Mark Michelson for the report.

The standard says that a SIPS dialog should be secure end to end. However some proxies or UASes disobey this requirement and give back plain SIP URI in the Record-Route or Contact header. We should terminate such call.

This ticket will implement Contact/Record-Route header checks in incoming initial/subsequent INVITE/UPDATE request or 2xx response to INVITE/UPDATE. Contact URI should use "sips" scheme and Record-Route URI should use either "sips" scheme or "transport=tls" param. As UAC, if it receives 200/OK response to initial/subsequent INVITE/UPDATE with insecure Contact/Record-Route, it will terminate the session. As UAS, if it receives initial/subsequent INVITE/UPDATE request with insecure Contact/Route headers, it will send response with SIP status code 480 and Warning header: 381 "SIPS required".

This feature can be disabled by setting pjsip_cfg()->endpt.disable_secure_dlg_check to PJ_TRUE (default is PJ_FALSE).

Here are some texts from the standard:

• All hops must use TLS (should we check if all Record-Route URIs use "sips" scheme or "transport=tls" param? Currently we'll just check the top Record-Route header)
  • ​http://tools.ietf.org/html/rfc5630#section-4:

    This document specifies that SIPS means that the SIP resource
    designated by the target SIPS URI is to be contacted securely, using
    TLS on each hop between the UAC and the remote UAS (as opposed to
    only to the proxy responsible for the target domain of the Request-
    URI).
    

• Contact header must use "sips" scheme:
  • in request ​http://tools.ietf.org/html/rfc5630#section-5.1.1:

    As mandated by [RFC3261], Section 8.1.1.8, in a request, "if the
    Request-URI or top Route header field value contains a SIPS URI, the
    Contact header field MUST contain a SIPS URI as well".
    

  • in target refresh ​http://tools.ietf.org/html/rfc5630#section-5.1.1.2:

    When a target refresh occurs within a dialog (e.g., re-INVITE
    request, UPDATE request), the UAC MUST include a Contact header field
    with a SIPS URI if the original request used a SIPS Request-URI.
    

  • in response ​http://tools.ietf.org/html/rfc3261#section-12.1.1:

    If the request that initiated the dialog contained a SIPS URI in
    the Request-URI or in the top Record-Route header field value, if
    there was any, or the Contact header field if there was no Record-
    Route header field, the Contact header field in the response MUST
    be a SIPS URI.