= Using Intel Integrated Performance Primitive (IPP) with PJMEDIA = [[BR]] Intel® Integrated Performance Primitives (Intel® IPP) is an extensive library of multi-core-ready, highly optimized software functions for multimedia data processing, and communications applications. The Intel IPP can be used with PJMEDIA library to provide the following codecs: || '''Codec''' || '''Bitrates (Kbps)''' || '''Clock rate/'''[[BR]]'''samples per frame''' || '''default'''[[BR]]'''frames/pkt''' || '''default'''[[BR]]'''bitrate''' || '''VAD/DTX''' || '''PLC''' || '''Note''' || || '''AMR NB''' || 4.75 to 12.2 || 8000/160 || 2 || 7400 || internal || internal || || || '''AMR WB''' || 6.6 to 23.85 || 16000/320 || 1 || 15850 || internal || internal || || || '''G.722.1''' || 16, 24, 32, 40 || 16000/320 || 1 || 16000 || pjmedia || internal || || || '''G.723.1''' || 5.3, 6.3 || 8000/240 || 1 || 5300 || internal || internal || || || '''G.726''' || 16, 24, 32, 40 || 8000/80 || 2 || - || pjmedia || pjmedia || The bitrate is negotiated in SDP || || '''G.728''' || 16 || 8000/80 || 2 || 16000 || pjmedia || internal || || || '''G.729A''' || 8 || 8000/80 || 2 || 8000 || internal || internal || 1) Annex D and Annex E is supported for receive direction. || [[BR]] '''Table of Contents''' [[PageOutline(2-3,,inline)]] [[BR]] ---- == Requirements == The following are required: * PJSIP version 1.0-rc1 or above * Intel IPP library for your platform * Intel IPP samples for your platform * Suitable compiler for both PJSIP and IPP for your platform The following IPP versions have been tested: - Intel IPP '''version 5.3 Update 3''', tested on Windows and Linux - Intel IPP '''version 6.1''', tested on Windows, Linux, and Mac OS X {{{ #!html
}}} '''Note that the use of Intel IPP requires a license from Intel, and in addition a license from the parties who own the royalty of the codec(s) may also be required.''' {{{ #!html
}}} [[BR]] {{{ #!html
}}} {{{ #!html
}}} [[BR]] ---- == Installation == === Windows Instructions === ==== Download and install IPP and IPP samples ==== 1. Download Intel IPP and Intel IPP samples from http://www.intel.com/cd/software/products/asmo-na/eng/302910.htm 1. Run the Intel IPP installer (e.g. ''w_ipp_ia32_p_5.3.3.082.exe''). The default installation will install it to {{{C:\Program Files\Intel\IPP\5.3.3.082}}} folder, and this is the folder that will be referred to by this tutorial. 1. Unzip the Intel IPP samples (e.g. ''w_ipp-samples_p_5.3.095.zip'') to a folder. This tutorial assumes that the samples are installed in {{{C:\ipp-samples}}}, but it can be installed anywhere else with some modifications to the instructions below. ==== Build the IPP samples ==== 1. Go to the IPP samples unzipped folder (e.g. {{{C:\ipp-samples}}}). 1. Go to {{{speech-codecs}}} folder. 1. Open the {{{readme.htm}}} file, and follow the instructions there to build the samples from the source. - typically the build process will involve just setting the IPPROOT environment variable and execute one of the {{{build*.bat}}} batch file. The build process produces the following: - (for IPP v5.3) '''{{{usc.lib}}}''' file somewhere under {{{speech-codecs\bin}}} directory, e.g. {{{ipp-samples\speech-codecs\bin\win32_cl9\lib}}} directory. - (for IPP v6.x) '''{{{speech.lib}}}''' file somewhere under {{{speech-codecs\_bin}}} directory, e.g. {{{ipp-samples\speech-codecs\_bin\win32_cl9\lib}}} directory. If you have any problems with installing and/or building the IPP or IPP samples, please read the IPP documentation or contact Intel for support. ==== Configure Visual Studio ==== 1. Add Intel IPP include and library paths to Visual Studio paths. For example if 32bit Intel IPP is installed in {{{C:\Program Files\Intel\IPP\5.3.3.082\ia32}}} directory: - add to the VS include path: - {{{C:\Program Files\Intel\IPP\5.3.3.082\ia32\include}}} - add to the VS library path: - {{{C:\Program Files\Intel\IPP\5.3.3.082\ia32\lib}}} - {{{C:\Program Files\Intel\IPP\5.3.3.082\ia32\stublib}}} 1. Add Intel IPP samples include and library path to Visual Studio paths. For example, if Intel IPP samples are located in {{{C:\ipp-samples}}}: - add to the include path: - {{{C:\ipp-samples\speech-codecs\core\usc\include}}} - add to the library path (please replace {{{win32_cl9}}} with the actual directory name where the file '''{{{usc.lib}}}''' is found (or '''{{{speech.lib}}}''' for IPP 6.x). Please see the IPP build instructions above): - {{{C:\ipp-samples\speech-codecs\bin\win32_cl9\lib}}} Note that you still need to enable IPP support in your {{{config_site.h}}} in order to use the IPP features. This will be explained in later sections. ==== Issues with building statically linked application ==== There are '''link errors''' when building the application using '''statically linked CRT''' ({{{/MT[d]}}} compiler option), something like the following: {{{ msvcrt.lib(ti_inst.obj) : error LNK2005: "private: __thiscall type_info::type_info(class type_info const &)" (??0type_info@@AAE@ABV0@@Z) already defined in LIBCMTD.lib(typinfo.obj) msvcrt.lib(ti_inst.obj) : error LNK2005: "private: class type_info & __thiscall type_info::operator=(class type_info const &)" (??4type_info@@AAEAAV0@ABV0@@Z) already defined in LIBCMTD.lib(typinfo.obj)'' }}} This is caused by that IPP samples is by default built using '''dynamically linked CRT''' ({{{/MD}}} compiler option). A simple way to solve this is by modifying the {{{makefile}}} of {{{speech-codec}}} samples, for example by inserting the following line to {{{ipp-samples\speech-codecs\Makefile}}} line 46: {{{ M_FLAG = /Zl ## omitting the default C runtime library name }}} IPP samples application ({{{*.exe}}}) may not build successfully, however {{{speech.lib}}} will be built successfully and linkable with any CRT linkage types. === Linux Instructions === ==== Download and install IPP and IPP samples ==== 1. Download Intel IPP and Intel IPP samples from http://www.intel.com/cd/software/products/asmo-na/eng/302910.htm 1. Extract the archive to a temporary folder 1. Run the installer (e.g. ''./install.sh'') and follow the on screen instructions. The default installation options will install it to ''"/opt/intel/ipp/"'' directory (e.g. ''"/opt/intel/ipp/5.3.1.062/ia32/"'' in my case). 1. Extract IPP samples to a directory (e.g. to ''"~/Desktop/ipp-samples"''). ==== Build the IPP samples ==== 1. Go to the directory where IPP samples have been extracted to (e.g. {{{~/Desktop/ipp-samples}}}). 1. Go to {{{speech-codecs}}} folder. 1. Open the {{{readme.htm}}} file, and follow the instructions there to build the samples from the source. - typically you'd just need to execute one of the '''build*.sh''' script after setting the IPPROOT environment variable. The build process produces the following: - (for IPP v5.3) '''{{{libusc.a}}}''' file somewhere under {{{speech-codecs/bin}}} directory, e.g. {{{ipp-samples/speech-codecs/bin/linux32_gcc3/lib}}} directory. - (for IPP v6.x) '''{{{libspeech.a}}}''' file somewhere under {{{speech-codecs/_bin}}} directory, e.g. {{{ipp-samples/speech-codecs/_bin/linuxem64t_gcc4/lib}}} directory. The directory name corresponds to the platform where the samples are built for. If you have any problems with installing and/or building the IPP or IPP samples, please read the IPP documentation or contact Intel for support. ==== Running "configure" with IPP support ==== The {{{configure}}} script needs to be invoked with the appropriate IPP options in order to set up IPP include and library paths. There are three relevant arguments: {{{ --with-ipp=DIR Specify the Intel IPP location --with-ipp-samples=DIR Specify the Intel IPP samples location --with-ipp-arch=ARCH Specify the Intel IPP ARCH suffix, e.g. "64" or "em64t". Default is blank for IA32. }}} For this tutorial, we'll use two environment variables to hold the location of IPP and IPP samples. Sample configure session for Intel IPP 5.3 for IA32 architecture: {{{ $ cd /your/pjproject/directory $ export IPPROOT=/opt/intel/ipp/5.3.1.062/ia32/ $ export IPPSAMPLES=~/Desktop/ipp-samples $ ./configure --enable-ipp --with-ipp=$IPPROOT --with-ipp-samples=$IPPSAMPLES }}} Sample configure session for Intel IPP 6.1 for em64t architecture: {{{ $ cd /your/pjproject/directory $ export IPPROOT=/opt/intel/ipp/6.1.2.051/em64t/ $ export IPPSAMPLES=~/Desktop/opt/intel/ipp-samples $ ./configure --enable-ipp --with-ipp=$IPPROOT --with-ipp-samples=$IPPSAMPLES --with-ipp-arch=em64t }}} Note that you still need to enable IPP support in your {{{config_site.h}}} in order to use the IPP features. This will be explained in the next section. === Mac OS X Instructions (Warning: in construction, may not work) === ==== Download and install Intel Compiler and IPP samples ==== 1. IPP cannot be obtained separately, you have to install Intel Compiler to get it. 1. Download Intel Compiler and Intel IPP samples from http://www.intel.com/cd/software/products/asmo-na/eng/302910.htm 1. Extract the archive to a temporary folder 1. Run the installer and follow the on screen instructions. The default installation options will install it to ''"/Library/Frameworks"'' directory (e.g. ''"/Library/Frameworks/Intel_IPP.framework/Versions/Current/"'' in my case). 1. Extract IPP samples to a directory (e.g. to ''"~/Desktop/ipp-samples"''). ==== Build the IPP samples ==== 1. Go to the directory where IPP samples have been extracted to (e.g. {{{~/Desktop/ipp-samples}}}). 1. Go to {{{speech-codecs}}} folder. 1. Open the {{{readme.htm}}} file, and follow the instructions there to build the samples from the source. - typically you'd just need to execute one of the '''build*.sh''' script after setting the IPPROOT environment variable. The build process produces the following: - (for IPP v6.x) '''{{{libspeech.a}}}''' file somewhere under {{{speech-codecs/_bin}}} directory, e.g. {{{ipp-samples/speech-codecs/_bin/osxem64t_icc111/lib}}} directory. The directory name corresponds to the platform where the samples are built for. If you have any problems with installing and/or building the IPP or IPP samples, please read the IPP documentation or contact Intel for support. ==== Running "configure" with IPP support ==== The {{{configure}}} script needs to be invoked with the appropriate IPP options in order to set up IPP include and library paths. There are three relevant arguments: {{{ --with-ipp=DIR Specify the Intel IPP location --with-ipp-samples=DIR Specify the Intel IPP samples location --with-ipp-arch=ARCH Specify the Intel IPP ARCH suffix, e.g. "64" or "em64t". Default is blank for IA32. }}} For this tutorial, we'll use two environment variables to hold the location of IPP and IPP samples. Sample configure session for Intel IPP 6.1 for em64t architecture: {{{ $ cd /your/pjproject/directory $ export IPPROOT=/Library/Frameworks/Intel_IPP.framework/Versions/Current/ $ export IPPSAMPLES=~/Desktop/opt/intel/ipp-samples $ ./configure --enable-ipp --with-ipp=$IPPROOT --with-ipp-samples=$IPPSAMPLES --with-ipp-arch=em64t }}} Note that you still need to enable IPP support in your {{{config_site.h}}} in order to use the IPP features. This will be explained in the next section. [[BR]] ---- == PJMEDIA Configuration == === Enable IPP Support === By default, IPP support is disabled in PJMEDIA. To enable IPP support, declare the following in your {{{config_site.h}}}: {{{ #define PJMEDIA_HAS_INTEL_IPP 1 }}} Note that you still need to set this macro even after IPP has been enabled with the {{{configure}}} command. === Enable/Disable Individual IPP Codec === Once IPP support is enabled, by default all IPP codecs are enabled. To disable parcular codecs '''at compile time''', use the following constructs: {{{ #define PJMEDIA_HAS_INTEL_IPP_CODEC_AMR 0 #define PJMEDIA_HAS_INTEL_IPP_CODEC_AMRWB 0 #define PJMEDIA_HAS_INTEL_IPP_CODEC_G729 0 #define PJMEDIA_HAS_INTEL_IPP_CODEC_G723_1 0 #define PJMEDIA_HAS_INTEL_IPP_CODEC_G726 0 #define PJMEDIA_HAS_INTEL_IPP_CODEC_G728 0 #define PJMEDIA_HAS_INTEL_IPP_CODEC_G722_1 0 }}} You can also enable/disable the individual IPP codec at run-time by setting the particular codec priority to zero. === Verifying Intel IPP Codecs Support === To verify that Intel IPP codecs supports are present, you can run [http://www.pjsip.org/pjsua.htm pjsua] and enter "Cp" (codec priority) command from the menu. It should display all IPP codecs that are enabled at compile time, e.g.: {{{ >>> Cp List of codecs: 130 speex/16000/1 129 speex/8000/1 128 speex/32000/1 128 iLBC/8000/1 128 GSM/8000/1 128 PCMU/8000/1 128 PCMA/8000/1 128 G722/16000/1 128 AMR/8000/1 128 AMR-WB/16000/1 128 G729/8000/1 128 G723/8000/1 128 G726-32/8000/1 128 G728/8000/1 128 G7221/16000/1 128 G7221/16000/1 0 L16/44100/1 0 L16/44100/2 0 L16/8000/1 0 L16/8000/2 0 L16/11025/1 0 L16/11025/2 0 L16/16000/1 0 L16/16000/2 0 L16/22050/1 0 L16/22050/2 0 L16/32000/1 0 L16/32000/2 0 L16/48000/1 0 L16/48000/2 Enter codec id and its new priority (e.g. "speex/16000 200"), empty to cancel: Codec name ("*" for all) and priority: }}} === Manipulating IPP Codecs === To change the IPP codec properties at run-time (e.g. to enable/disable, change the priority, bitrate, vad/plc setting, etc.), use the usual codec manipulation API that is available from both PJSUA-LIB or PJMEDIA. More information is available in [#using Using in Application] section below. [[BR]] ---- == Static Linking == === Windows Instructions === By default, PJMEDIA will automatically link with Intel IPP libraries when Visual Studio is used. This is done by using using {{{#pragma comment(lib)}}} constructs in the source code. This is convenient, however it will only link with the stub libraries and the Intel IPP DLL's will be required when distributing the application. If application wants to link with the different types of the Intel IPP libraries (for example, the static libraries), it must set {{{PJMEDIA_AUTO_LINK_IPP_LIBS}}} option to zero and specify the Intel IPP libraries in the application's input library specification manually. Example: 1. Disable default automatic linking: {{{ #define PJMEDIA_AUTO_LINK_IPP_LIBS 0 }}} 1. Add {{{C:\Program Files\Intel\IPP\5.3.3.082\ia32\lib}}} to the VS library path 1. Include the following libraries in the application project setting: {{{ ippsmerged.lib ippsemerged.lib ippscmerged.lib ippscemerged.lib ippsrmerged.lib ippsremerged.lib ippcorel.lib usc.lib }}} 1. Insert IPP static initialization function, i.e: {{{ippStaticInit()}}} declared in {{{ippcore.h}}}, into the startup point of the application, e.g: {{{ #include main() { /* Init IPP before calling any IPP functions. */ ippStaticInit(); ... } }}} === Linux/Mac Instructions === By default, the configuration and build system will link with Intel IPP shared libraries. This is controlled by the {{{IPP_LIBS}}} setting in the auto-generated {{{build.mak}}} file: {{{ export IPP_LIBS=-lusc -lippsc -lipps -lippsr -lippcore -lguide }}} If you want to configure static linking, you can override this variable in your {{{user.mak}}} file, or change the value in the {{{build.mak}}}. The later is not recommended since the {{{build.mak}}} file will be overwritten everytime {{{configure}}} is run. Sample to override the variable in {{{user.mak}}}: {{{ export IPP_LIBS=-lusc -lippscmerged .. }}} [[BR]] Additional Information on IPP Linkage Models: - [http://software.intel.com/en-us/articles/intel-integrated-performance-primitives-intel-ipp-intel-ipp-linkage-models-quick-reference-guide IPP Linkage Models] - [http://software.intel.com/file/6279 Choosing the best IPP Linkage Models for Your Applications] [[BR]] ---- == Using in Applications == #using === Codec Registration === Before application can use any of the IPP codecs above, the IPP codec factory must be registered to the codec manager. If application is using [http://www.pjsip.org/pjsip/docs/html/group__PJSUA__LIB.htm PJSUA-LIB], this will be done automatically by PJSUA-LIB. If application is using [http://www.pjsip.org/pjmedia/docs/html/index.htm PJMEDIA] directly, then it must register the IPP codec factory by calling: {{{ pjmedia_codec_ipp_init(media_endpt); }}} === Codec Settings === Codec settings are managed by setting the appropriate field in {{{settings}}} part of [http://www.pjsip.org/pjmedia/docs/html/structpjmedia__codec__param.htm pjmedia_codec_param] as usual. [[BR]] ---- == Setting DLL/Shared Library Path == By default, the build process links with the IPP dynamic libraries. Because of this, you'd need to set the DLL/shared library path before running PJSIP application that links with Intel IPP. === Setting up DLL Path === On Windows, you'd need to add Intel IPP's DLL location to your PATH variable. === Setting up LD_LIBRARY_PATH === On Linux, you'd need to add Intel IPP shared library location to your {{{LD_LIBRARY_PATH}}} environment variable: {{{ $ export LD_LIBRARY_PATH="$IPPROOT/sharedlib:$LD_LIBRARY_PATH" }}} [[BR]] ---- == Open Issues == === Bitrate Selection === For codecs that support multiple bitrates (such as G.723.1 and AMR), the encoding bitrate is currently selected by setting [http://www.pjsip.org/pjmedia/docs/html/structpjmedia__codec__param.htm pjmedia_codec_param.info.avg_bps] field. This mechanism currently contradicts the normal convention of not modifying {{{pjmedia_codec_param.info}}} part of {{{pjmedia_codec_param}}}. === AMR Options === Negotiation of various AMR options in SDP is currently not supported. The AMR mode is locked to bandwidth-optimized, and when AMR with octet-aligned packet is received, the behavior is undefined. === MacOS X Support === Although IPP is said to be supported for MacOS X (Intel), we don't know yet how to configure this support. === Visual Studio 6 Support === Visual Studio 6 is not supported by the Intel IPP samples build system. If you wish to use Visual Studio 6 to build the samples, replace the {{{search_compilers.bat}}} in the {{{tools\env}}} directory under the IPP samples directory with the one from the attachment of this page below.