wiki:Intel_IPP_Codecs

Version 41 (modified by ming, 12 years ago) (diff)

--

Using Intel Integrated Performance Primitive (IPP) with PJMEDIA


Intel® Integrated Performance Primitives (Intel® IPP) is an extensive library of multi-core-ready, highly optimized software functions for multimedia data processing, and communications applications.

The Intel IPP can be used with PJMEDIA library to provide the following codecs:

Codec Bitrates (Kbps) Clock rate/
samples per frame 		default
frames/pkt 		default
bitrate 		VAD/DTX PLC Note
AMR NB 4.75 to 12.2 8000/160 2 7400 internal internal
AMR WB 6.6 to 23.85 16000/320 1 15850 internal internal
G.722.1 16, 24, 32, 40 16000/320 1 16000 pjmedia internal
G.723.1 5.3, 6.3 8000/240 1 5300 internal internal
G.726 16, 24, 32, 40 8000/80 2 - pjmedia pjmedia The bitrate is negotiated in SDP
G.728 16 8000/80 2 16000 pjmedia internal
G.729A 8 8000/80 2 8000 internal internal 1) Annex D and Annex E is supported for receive direction.


Table of Contents

  1. Requirements
  2. Installation
    1. Windows Instructions
    2. Linux Instructions
    3. Mac OS X Instructions
  3. PJMEDIA Configuration
    1. Enable IPP Support
    2. Enable/Disable? Individual IPP Codec
    3. Verifying Intel IPP Codecs Support
    4. Manipulating IPP Codecs
  4. Static Linking
    1. Windows Instructions
    2. Linux/Mac? Instructions
  5. Using in Applications
    1. Codec Registration
    2. Codec Settings
  6. Setting DLL/Shared Library Path
    1. Setting up DLL Path
    2. Setting up LD_LIBRARY_PATH
  7. Open Issues
    1. Bitrate Selection
    2. AMR Options
    3. Visual Studio 6 Support


Requirements

The following are required:

  • PJSIP version 1.0-rc1 or above for Intel IPP 5.x, and PJSIP version 1.6 or above for Intel IPP 6.x.
  • Intel IPP library for your platform
  • Intel IPP samples for your platform
  • Suitable compiler for both PJSIP and IPP for your platform

The following IPP versions have been tested:

  • Intel IPP version 5.3 Update 3, tested on Windows and Linux
  • Intel IPP version 6.1, tested on Windows, Linux, and Mac OS X. Note that this requires PJSIP version 1.6 or later.
  • Intel IPP version 7.0 Update 2, tested on Windows and Linux. Note this requires PJSIP version 1.10 or later (ticket #1200).

Note that the use of Intel IPP requires a license from Intel, and in addition a license from the parties who own the royalty of the codec(s) may also be required.



Installation

Windows Instructions

Download and install IPP and IPP samples

  1. Download Intel IPP and Intel IPP samples from ​http://software.intel.com/en-us/intel-ipp
  2. Run the Intel IPP installer (e.g. w_ipp_ia32_p_5.3.3.082.exe). The default installation will install it to C:\Program Files\Intel\IPP\5.3.3.082 folder, and this is the folder that will be referred to by this tutorial.
  3. Unzip the Intel IPP samples (e.g. w_ipp-samples_p_5.3.095.zip) to a folder. This tutorial assumes that the samples are installed in C:\ipp-samples, but it can be installed anywhere else with some modifications to the instructions below.

Build the IPP samples

  1. Go to the IPP samples unzipped folder (e.g. C:\ipp-samples).
  2. Go to speech-codecs folder.
  3. Open the readme.htm file, and follow the instructions there to build the samples from the source.
    • typically the build process will involve just setting the IPPROOT environment variable and execute one of the build*.bat batch file.

The build process produces the following:

  • (for IPP v5.3) usc.lib file somewhere under speech-codecs\bin directory, e.g. ipp-samples\speech-codecs\bin\win32_cl9\lib directory.
  • (for IPP v6.x or above) speech.lib file somewhere under speech-codecs\_bin directory, e.g. ipp-samples\speech-codecs\_bin\win32_cl9\lib directory.

If you have any problems with installing and/or building the IPP or IPP samples, please read the IPP documentation or contact Intel for support.

Configure Visual Studio

  1. Add Intel IPP include and library paths to Visual Studio paths. For example if 32bit Intel IPP is installed in C:\Program Files\Intel\IPP\5.3.3.082\ia32 directory:
    • add to the VS include path:
      • C:\Program Files\Intel\IPP\5.3.3.082\ia32\include
    • add to the VS library path:
      • C:\Program Files\Intel\IPP\5.3.3.082\ia32\lib
      • C:\Program Files\Intel\IPP\5.3.3.082\ia32\stublib
    • for IPP v7.x, also add Intel compiler library path C:\Program Files\Intel\ComposerXE-2011\compiler\lib\ia32
  2. Add Intel IPP samples include and library path to Visual Studio paths. For example, if Intel IPP samples are located in C:\ipp-samples:
    • add to the include path:
      • C:\ipp-samples\speech-codecs\core\usc\include
    • add to the library path (please replace win32_cl9 with the actual directory name where the file usc.lib is found (or speech.lib for IPP 6.x or above). Please see the IPP build instructions above):
      • C:\ipp-samples\speech-codecs\bin\win32_cl9\lib

Note that you still need to enable IPP support in your config_site.h in order to use the IPP features. This will be explained in later sections.

Issues with building statically linked application

There are link errors when building the application using statically linked CRT (/MT[d] compiler option), something like the following: 

msvcrt.lib(ti_inst.obj) : error LNK2005: "private: 
  __thiscall type_info::type_info(class type_info const &)"
 (??0type_info@@AAE@ABV0@@Z) already defined in LIBCMTD.lib(typinfo.obj)

msvcrt.lib(ti_inst.obj) : error LNK2005: "private: class type_info 
  & __thiscall type_info::operator=(class type_info const &)" 
  (??4type_info@@AAEAAV0@ABV0@@Z) already defined in LIBCMTD.lib(typinfo.obj)''

This is caused by that IPP samples is by default built using dynamically linked CRT (/MD compiler option). A simple way to solve this is by modifying the makefile of speech-codec samples, for example by inserting the following line to ipp-samples\speech-codecs\Makefile line 46: 

M_FLAG = /Zl ## omitting the default C runtime library name

IPP samples application (*.exe) may not build successfully, however speech.lib will be built successfully and linkable with any CRT linkage types.

Issues with building Intel IPP 7.0 on Visual C++ 2005

There may be link errors something like the following: 

ipps_t.lib: fatal error LNK1103: debugging information corrupt; recompile module

This is a compatibility issue of VS2005, check and install this ​VS2005 hotfix.


Linux Instructions

Download and install IPP and IPP samples

  1. Download Intel IPP and Intel IPP samples from ​http://www.intel.com/cd/software/products/asmo-na/eng/302910.htm
  2. Extract the archive to a temporary folder
  3. Run the installer (e.g. ./install.sh) and follow the on screen instructions. The default installation options will install it to "/opt/intel/ipp/" directory (e.g. "/opt/intel/ipp/5.3.1.062/ia32/" in my case).
  4. Extract IPP samples to a directory (e.g. to "~/Desktop/ipp-samples").

Build the IPP samples

  1. Go to the directory where IPP samples have been extracted to (e.g. ~/Desktop/ipp-samples).
  2. Go to speech-codecs folder.
  3. Open the readme.htm file, and follow the instructions there to build the samples from the source.
    • typically you'd just need to execute one of the build*.sh script after setting the IPPROOT environment variable.

The build process produces the following:

  • (for IPP v5.3) libusc.a file somewhere under speech-codecs/bin directory, e.g. ipp-samples/speech-codecs/bin/linux32_gcc3/lib directory.
  • (for IPP v6.x) libspeech.a file somewhere under speech-codecs/_bin directory, e.g. ipp-samples/speech-codecs/_bin/linuxem64t_gcc4/lib directory.

The directory name corresponds to the platform where the samples are built for.

If you have any problems with installing and/or building the IPP or IPP samples, please read the IPP documentation or contact Intel for support.

Running "configure" with IPP support

The configure script needs to be invoked with the appropriate IPP options in order to set up IPP include and library paths. There are three relevant arguments: 

  --with-ipp=DIR          Specify the Intel IPP location
  --with-ipp-samples=DIR  Specify the Intel IPP samples location
  --with-ipp-arch=ARCH    Specify the Intel IPP ARCH suffix, e.g. "64" or
                          "em64t". Default is blank for IA32.

For this tutorial, we'll use two environment variables to hold the location of IPP and IPP samples.

Sample configure session for Intel IPP 5.3 for IA32 architecture: 

$ cd /your/pjproject/directory
$ export IPPROOT=/opt/intel/ipp/5.3.1.062/ia32/
$ export IPPSAMPLES=~/Desktop/ipp-samples
$ ./configure --enable-ipp --with-ipp=$IPPROOT --with-ipp-samples=$IPPSAMPLES

Sample configure session for Intel IPP 6.1 for em64t architecture: 

$ cd /your/pjproject/directory
$ export IPPROOT=/opt/intel/ipp/6.1.2.051/em64t/
$ export IPPSAMPLES=~/Desktop/opt/intel/ipp-samples
$ ./configure --enable-ipp --with-ipp=$IPPROOT --with-ipp-samples=$IPPSAMPLES --with-ipp-arch=em64t

Sample configure session for Intel IPP 7.0 for Linux64 architecture: 

$ cd /your/pjproject/directory
$ export IPPROOT=~/Desktop/opt/intel/composerxe-2011.2.137/ipp
$ export IPPSAMPLES=~/Desktop/opt/intel/ipp-samples
$ ./configure --enable-ipp --with-ipp=$IPPROOT --with-ipp-samples=$IPPSAMPLES

Note that you still need to enable IPP support in your config_site.h in order to use the IPP features. This will be explained in the next section.


Mac OS X Instructions

Download and install Intel Compiler and IPP samples

  1. IPP cannot be obtained separately, you have to install Intel Compiler to get it.
  2. Download Intel Compiler and Intel IPP samples from ​http://www.intel.com/cd/software/products/asmo-na/eng/302910.htm
  3. Extract the archive to a temporary folder
  4. Run the installer and follow the on screen instructions. The default installation options will install it to "/Library/Frameworks?" directory (e.g. "/Library/Frameworks?/Intel_IPP.framework/Versions/Current/" in my case).
  5. Extract IPP samples to a directory (e.g. to "~/Desktop/ipp-samples").

Build the IPP samples

  1. Open Terminal application
  2. Go to the directory where IPP samples have been extracted to (e.g. ~/Desktop/ipp-samples).
  3. Go to speech-codecs folder.
  4. Open the readme.htm file, and follow the instructions there to build the samples from the source.
    • typically you'd just need to execute one of the build*.sh script after setting the IPPROOT environment variable.
  5. Important: since Intel C Compiler (icc) is installed, specify argument gcc4 when executing the build*.sh, to build the samples with GCC compiler. Without this, the samples will be built with icc and it will result in link error when compiling PJSIP applications with gcc.
  6. Sample command to build the samples: 
    $ cd ~/Desktop/ipp-samples/speech-codecs
$ export IPPROOT=/Library/Frameworks/Intel_IPP.framework/Versions/Current/
$ ./buildem64t.sh gcc4

The build process produces the following:

  • (for IPP v6.x) libspeech.a file somewhere under speech-codecs/_bin directory, e.g. ipp-samples/speech-codecs/_bin/osxem64t_gcc4/lib directory.

The directory name may be different, depending on which target architecture is selected to be build.

If you have any problems with installing and/or building the IPP or IPP samples, please read the IPP documentation or contact Intel for support.

Running "configure" with IPP support

The configure script needs to be invoked with the appropriate IPP options in order to set up IPP include and library paths. There are three relevant arguments: 

  --with-ipp=DIR          Specify the Intel IPP location
  --with-ipp-samples=DIR  Specify the Intel IPP samples location

There are third IPP argument, the --with-ipp-arch option, but this is not relevant for Mac OS X. Do not use this option.

For this tutorial, we'll use two environment variables to hold the location of IPP and IPP samples.

Sample configure session for Intel IPP 6.1: 

$ cd /your/pjproject/directory
$ export IPPROOT=/Library/Frameworks/Intel_IPP.framework/Versions/Current/
$ export IPPSAMPLES=~/Desktop/ipp-samples
$ ./configure --enable-ipp --with-ipp=$IPPROOT --with-ipp-samples=$IPPSAMPLES

Note that you still need to enable IPP support in your config_site.h in order to use the IPP features. This will be explained in the next section.

Issues and solutions

  1. Link or run-time error: "file not found: libiomp5.dylib", or run-time error: 
    dyld: Library not loaded: libiomp5.dylib
Referenced from: /opt/intel/Compiler/11.1/076/Frameworks/mkl/lib/em64t/libmkl_intel_thread.dylib
Reason: image not found
Trace/BPT trap
    or 
    dyld: Library not loaded: libiomp5.dylib
  Referenced from: /opt/intel/Compiler/11.1/080/Frameworks/ipp/Libraries/libippsc-6.1.dylib
  Reason: image not found
Trace/BPT trap
    Please see this article for solution (use the first suggestion): ​http://software.intel.com/en-us/articles/xcode-link-error-file-not-found-libiomp5dylib/
  2. Link error: 
    Undefined symbols:
  "_USC_G728_Fxns", referenced from:
      _ipp_codec in libpjmedia-codec-i386-apple-darwin9.8.0.a(ipp_codecs.o)
  "_USC_G729AFP_Fxns", referenced from:
      _ipp_codec in libpjmedia-codec-i386-apple-darwin9.8.0.a(ipp_codecs.o)
  "_USC_AMRWB_Fxns", referenced from:
      _ipp_codec in libpjmedia-codec-i386-apple-darwin9.8.0.a(ipp_codecs.o)
  ...
    or 
    Undefined symbols:
  "___svml_cosf4", referenced from:
      _ownCOS_G729_32f in libspeech.a(owng729fp.o)
      _ownCOS_G729_32f in libspeech.a(owng729fp.o)
      _ownCOS_G729_32f in libspeech.a(owng729fp.o)
      _ownCOS_G729_32f in libspeech.a(owng729fp.o)
  "__intel_fast_memcpy", referenced from:
      _apiG729FPEncode in libspeech.a(encg729fp.o)
      _apiG729FPEncode in libspeech.a(encg729fp.o)
      _UpdateExcitationSignal in libspeech.a(consealg728.o)
      _BandwidthExpansionModul in libspeech.a(adapters.o)
  "__intel_fast_memset", referenced from:
      _apiG722Decoder_Init in libspeech.a(g722api.o)
      _apiG722Encoder_Init in libspeech.a(g722api.o)
      _CodewordImpConv_G729_32f in libspeech.a(owng729fp.o)
      _FixedCodebookSearch_G723_16s in libspeech.a(owng723.o)
  "___svml_acosf4", referenced from:
      _ownACOS_G729_32f in libspeech.a(owng729fp.o)
      _ownACOS_G729_32f in libspeech.a(owng729fp.o)
      _ownACOS_G729_32f in libspeech.a(owng729fp.o)
      _ownACOS_G729_32f in libspeech.a(owng729fp.o)
ld: symbol(s) not found
collect2: ld returned 1 exit status
make[2]: *** [../bin/pjsua-i386-apple-darwin10.3.0] Error 1
make[1]: *** [pjsua] Error 2
    That's probably because the IPP samples were compiled with Intel compiler and not gcc. See Building IPP samples above.


PJMEDIA Configuration

Enable IPP Support

By default, IPP support is disabled in PJMEDIA. To enable IPP support, declare the following in your config_site.h: 

#define PJMEDIA_HAS_INTEL_IPP	1

Note that you still need to set this macro even after IPP has been enabled with the configure command.

Enable/Disable? Individual IPP Codec

Once IPP support is enabled, by default all IPP codecs are enabled. To disable parcular codecs at compile time, use the following constructs: 

#define PJMEDIA_HAS_INTEL_IPP_CODEC_AMR     0
#define PJMEDIA_HAS_INTEL_IPP_CODEC_AMRWB   0
#define PJMEDIA_HAS_INTEL_IPP_CODEC_G729    0
#define PJMEDIA_HAS_INTEL_IPP_CODEC_G723_1  0
#define PJMEDIA_HAS_INTEL_IPP_CODEC_G726    0
#define PJMEDIA_HAS_INTEL_IPP_CODEC_G728    0
#define PJMEDIA_HAS_INTEL_IPP_CODEC_G722_1  0

You can also enable/disable the individual IPP codec at run-time by setting the particular codec priority to zero.

Verifying Intel IPP Codecs Support

To verify that Intel IPP codecs supports are present, you can run ​pjsua and enter "Cp" (codec priority) command from the menu. It should display all IPP codecs that are enabled at compile time, e.g.: 

>>> Cp
List of codecs:
  130   speex/16000/1
  129   speex/8000/1
  128   speex/32000/1
  128   iLBC/8000/1
  128   GSM/8000/1
  128   PCMU/8000/1
  128   PCMA/8000/1
  128   G722/16000/1
  128   AMR/8000/1
  128   AMR-WB/16000/1
  128   G729/8000/1
  128   G723/8000/1
  128   G726-32/8000/1
  128   G728/8000/1
  128   G7221/16000/1
  128   G7221/16000/1
  0     L16/44100/1
  0     L16/44100/2
  0     L16/8000/1
  0     L16/8000/2
  0     L16/11025/1
  0     L16/11025/2
  0     L16/16000/1
  0     L16/16000/2
  0     L16/22050/1
  0     L16/22050/2
  0     L16/32000/1
  0     L16/32000/2
  0     L16/48000/1
  0     L16/48000/2

Enter codec id and its new priority (e.g. "speex/16000 200"), empty to cancel:
Codec name ("*" for all) and priority:

Manipulating IPP Codecs

To change the IPP codec properties at run-time (e.g. to enable/disable, change the priority, bitrate, vad/plc setting, etc.), use the usual codec manipulation API that is available from both PJSUA-LIB or PJMEDIA.

More information is available in Using in Application section below.


Static Linking

Windows Instructions

By default, PJMEDIA will automatically link with Intel IPP libraries when Visual Studio is used. This is done by using using #pragma comment(lib) constructs in the source code. This is convenient, however it will only link with the stub libraries and the Intel IPP DLL's will be required when distributing the application.

If application wants to link with the different types of the Intel IPP libraries (for example, the static libraries), it must set PJMEDIA_AUTO_LINK_IPP_LIBS option to zero and specify the Intel IPP libraries in the application's input library specification manually.

Example:

  1. Disable default automatic linking: 
    #define PJMEDIA_AUTO_LINK_IPP_LIBS  0
  2. Add C:\Program Files\Intel\IPP\5.3.3.082\ia32\lib to the VS library path
  3. Include the following libraries in the application project setting: 
    ippsmerged.lib ippsemerged.lib ippscmerged.lib ippscemerged.lib ippsrmerged.lib ippsremerged.lib ippcorel.lib usc.lib
  4. Insert IPP static initialization function, i.e: ippStaticInit() declared in ippcore.h, into the startup point of the application, e.g: 
    #include <ippcore.h>
main() {
    /* Init IPP before calling any IPP functions. */
    ippStaticInit();
    ...
}

Linux/Mac? Instructions

By default, the configuration and build system will link with Intel IPP shared libraries. This is controlled by the IPP_LIBS setting in the auto-generated build.mak file: 

export IPP_LIBS=-lusc -lippsc -lipps -lippsr -lippcore -lguide

If you want to configure static linking, you can override this variable in your user.mak file, or change the value in the build.mak. The later is not recommended since the build.mak file will be overwritten everytime configure is run.

Sample to override the variable in user.mak: 

export IPP_LIBS=-lusc -lippscmerged ..


Additional Information on IPP Linkage Models:


Using in Applications

Codec Registration

Before application can use any of the IPP codecs above, the IPP codec factory must be registered to the codec manager. If application is using ​PJSUA-LIB, this will be done automatically by PJSUA-LIB. If application is using ​PJMEDIA directly, then it must register the IPP codec factory by calling: 

    pjmedia_codec_ipp_init(media_endpt);

Codec Settings

Codec settings are managed by setting the appropriate field in settings part of ​pjmedia_codec_param as usual.


Setting DLL/Shared Library Path

By default, the build process links with the IPP dynamic libraries. Because of this, you'd need to set the DLL/shared library path before running PJSIP application that links with Intel IPP.

Setting up DLL Path

On Windows, you'd need to add Intel IPP's DLL location to your PATH variable.

Setting up LD_LIBRARY_PATH

On Linux, you'd need to add Intel IPP shared library location to your LD_LIBRARY_PATH environment variable: 

$ export LD_LIBRARY_PATH="$IPPROOT/sharedlib:$LD_LIBRARY_PATH"


Open Issues

Bitrate Selection

For codecs that support multiple bitrates (such as G.723.1 and AMR), the encoding bitrate is currently selected by setting ​pjmedia_codec_param.info.avg_bps field. This mechanism currently contradicts the normal convention of not modifying pjmedia_codec_param.info part of pjmedia_codec_param.

AMR Options

  • Negotiation of various AMR optional parameters in SDP is currently not supported. The RFC 3267 section 8.1 specifies a list of optional parameters for AMR/AMR-WB, however, the parameters supported by the IPP codecs are just octet-align and mode-set.
  • Payload verification is missing. A sample case, when the AMR mode is locked, after a successful SDP negotiation, to bandwidth-optimized, but the incoming AMR payloads are actually octet-aligned, the behavior will be undefined.

Visual Studio 6 Support

Visual Studio 6 is not supported by the Intel IPP samples build system. If you wish to use Visual Studio 6 to build the samples, replace the search_compilers.bat in the tools\env directory under the IPP samples directory with the one from the attachment of this page below.

Attachments (1)

Download all attachments as: .zip