wiki:Video_Users_Guide

Version 3 (modified by bennylp, 8 years ago) (diff)

--

PJSIP Video User's Guide

Table of Contents

  1. Requirements
  2. Building with Video Support
    1. Building with GNU Build System (for Mac OS X, Linux, etc.)
    2. Building with Visual Studio
    3. GUI Sample Application
  3. Video API (pjsua-lib)
    1. Device enumeration API
    2. Video preview API
    3. Video Configuration
    4. Video Window API
    5. Video Codec API

Video is available on PJSIP version 2.0 and later. This document describes how to use the video feature with PJSIP.


Requirements

All platforms:

  1. SDL version 1.3.
    • Note that this is the newer and not yet released SDL library, hence most likely it won't be installed by default on your system. You can download the source snapshot or from the Mercurial repository from http://www.libsdl.org/hg.php.
  2. ffmpeg development library.
  3. Optional: Qt development SDK for building the video GUI sample.

Linux:

  1. Video4Linux2 (v4l2) development library

Windows:

  1. ?

Mac OS X:

-

Mobile platforms (iPhone, Symbian, Windows Mobile, etc):

  • Video is not supported on this release yet!


Building with Video Support

Building with GNU Build System (for Mac OS X, Linux, etc.)

1. Running aconfigure

Video requirements will be detected by the configure script. Pay attention to the following output (the sample below was taken on a Mac):

checking SDL availability..... 1.3
checking for avdevice_version in -lavdevice... yes
checking for av_register_all in -lavformat... yes
checking for avcodec_init in -lavcodec... yes
checking for sws_scale in -lswscale... yes
checking for av_malloc in -lavutil... yes
checking for avcore_version in -lavcore... yes
checking for v4l2_open in -lv4l2... no

Once again, we need SDL version 1.3 for the video to work.

2. Configuring video support

Add this in your config_site.h:

#define PJMEDIA_HAS_VIDEO	1

3. Build the library

As usual with:

make dep && make clean && make

Once the build is successful, you will have pjsua executable containing video support. Additionally, you may want to build the GUI sample application too.


Building with Visual Studio

Visual Studio users will need to configure things manually as explained below.

1. Configure directories

Add include and library paths for the required components:

  1. SDL
  2. ffmpeg
  3. !DirectX (?)
  4. Base classes (?)

2. Configure Video Settings

Add these to your config_site.h:

#define PJMEDIA_HAS_VIDEO	    1
#define PJMEDIA_HAS_FFMPEG	    1
#define PJMEDIA_VIDEO_DEV_HAS_SDL   1

3. Rebuild the Solution

Once the build is successful, you will have pjsua executable containing video support. Additionally, you may want to build the GUI sample application too.


GUI Sample Application

We have a GUI sample application with video support. The project is located under pjsip-apps/src/vidgui. It is not built by default, and you need Qt SDK to build it.

GNU Build System (Mac OS X, Linux, etc)

Follow these steps to build vidgui sample:

$ cd pjsip-apps/src/vidgui
$ make

Visual Studio

Follow these steps to build vidgui sample with Visual Studio:

  1. Open command prompt, and
    cd pjsip-apps\src\vidgui
    
  2. Generate project files:
    qmake -tp vc
    
  3. Open vidgui.vcproj project.
  4. Save the solution, and build the project


Video API (pjsua-lib)

This section explains and lists the Video API as it was available when this document is written. For a more up to date lists, please see Video API reference documentation. Please see this page for detailed reference of the API.

The Video API is classified into the following categories.

Device enumeration API

The following API is available:

/**
 * Get the number of video devices installed in the system.
 *
 * @return		The number of devices.
 */
PJ_DECL(unsigned) pjsua_vid_dev_count(void);

/**
 * Retrieve the video device info for the specified device index.
 *
 * @param id		The device index.
 * @param vdi		Device info to be initialized.
 *
 * @return		PJ_SUCCESS on success, or the appropriate error code.
 */
PJ_DECL(pj_status_t) pjsua_vid_dev_get_info(pjmedia_vid_dev_index id,
                                            pjmedia_vid_dev_info *vdi);

/**
 * Enum all video devices installed in the system.
 *
 * @param info		Array of info to be initialized.
 * @param count		On input, specifies max elements in the array.
 *			On return, it contains actual number of elements
 *			that have been initialized.
 *
 * @return		PJ_SUCCESS on success, or the appropriate error code.
 */
PJ_DECL(pj_status_t) pjsua_vid_enum_devs(pjmedia_vid_dev_info info[],
					 unsigned *count);

In addition, the PJMEDIA videodev also provides this API to detect change in device availability:

/**
 * Refresh the list of video devices installed in the system. This function
 * will only refresh the list of videoo device so all active video streams will
 * be unaffected. After refreshing the device list, application MUST make sure
 * to update all index references to video devices (i.e. all variables of type
 * pjmedia_vid_dev_index) before calling any function that accepts video device
 * index as its parameter.
 *
 * @return		PJ_SUCCESS on successful operation or the appropriate
 *			error code.
 */
PJ_DECL(pj_status_t) pjmedia_vid_dev_refresh(void);

Video preview API

The video preview API can be used to show the output of capture device to a video window:

/**
 * Parameters for starting video preview with pjsua_vid_preview_start().
 * Application should initialize this structure with
 * pjsua_vid_preview_param_default().
 */
typedef struct pjsua_vid_preview_param
{
    /**
     * Device ID for the video renderer to be used for rendering the
     * capture stream for preview.
     */
    pjmedia_vid_dev_index	rend_id;
} pjsua_vid_preview_param;


/**
 * Start video preview window for the specified capture device.
 *
 * @param id		The capture device ID where its preview will be
 * 			started.
 * @param prm		Optional video preview parameters. Specify NULL
 * 			to use default values.
 *
 * @return		PJ_SUCCESS on success, or the appropriate error code.
 */
PJ_DECL(pj_status_t) pjsua_vid_preview_start(pjmedia_vid_dev_index id,
                                             pjsua_vid_preview_param *prm);

/**
 * Get the preview window handle associated with the capture device, if any.
 *
 * @param id		The capture device ID.
 *
 * @return		The window ID of the preview window for the
 * 			specified capture device ID, or NULL if preview
 * 			does not exist.
 */
PJ_DECL(pjsua_vid_win_id) pjsua_vid_preview_get_win(pjmedia_vid_dev_index id);

/**
 * Stop video preview.
 *
 * @param id		The capture device ID.
 *
 * @return		PJ_SUCCESS on success, or the appropriate error code.
 */
PJ_DECL(pj_status_t) pjsua_vid_preview_stop(pjmedia_vid_dev_index id);

Video Configuration

Video settings are mostly configured on pjsua_acc_config with the following fields:

    /**
     * Maximum number of simultaneous active video streams to be allowed
     * for calls on this account. Setting this to zero will disable video
     * in calls on this account, regardless of other video settings.
     *
     * Default: 1
     */
    unsigned         max_video_cnt;

    /**
     * Specify whether incoming video should be shown to screen by default.
     * This applies to incoming call (INVITE), incoming re-INVITE, and
     * incoming UPDATE requests.
     *
     * Regardless of this setting, application can detect incoming video
     * by implementing \a on_call_media_state() callback and enumerating
     * the media stream(s) with #pjsua_call_get_info(). Once incoming
     * video is recognised, application may retrieve the window associated
     * with the incoming video and show or hide it with
     * #pjsua_vid_win_set_show().
     *
     * Default: PJ_FALSE
     */
    pj_bool_t        vid_in_auto_show;

    /**
     * Specify whether outgoing video should be activated by default when
     * making outgoing calls and/or when incoming video is detected. This
     * applies to incoming and outgoing calls, incoming re-INVITE, and
     * incoming UPDATE. If the setting is non-zero, outgoing video
     * transmission will be started as soon as response to these requests
     * is sent (or received).
     *
     * Regardless of the value of this setting, application can start and
     * stop outgoing video transmission with #pjsua_call_set_vid_strm().
     *
     * Default: PJ_FALSE
     */
    pj_bool_t        vid_out_auto_transmit;

    /**
     * Specify the default capture device to be used by this account. If
     * \a vid_out_auto_transmit is enabled, this device will be used for
     * capturing video.
     *
     * Default: PJMEDIA_VID_DEFAULT_CAPTURE_DEV
     */
    pjmedia_vid_dev_index vid_cap_dev;

    /**
     * Specify the default rendering device to be used by this account.
     *
     * Default: PJMEDIA_VID_DEFAULT_RENDER_DEV
     */
    pjmedia_vid_dev_index vid_rend_dev;

Video Window API

A video window is a rectangular area in your monitor to display video content. The video content may come from remote stream, local camera (in case of preview), AVI playback, or any other video playback. Application mostly will be interested in the native handle of the video window so that it can embed it in its application window, however we also provide simple and commonly used API for manipulating the window.

/**
 * This structure describes video window info.
 */
typedef struct pjsua_vid_win_info
{
    /**
     * Renderer device ID.
     */
    pjmedia_vid_dev_index rdr_dev;

    /**
     * Native window handle.
     */
    pjmedia_vid_dev_hwnd hwnd;

    /**
     * Window show status. The window is hidden if false.
     */
    pj_bool_t	show;

    /**
     * Window position.
     */
    pjmedia_coord pos;

    /**
     * Window size.
     */
    pjmedia_rect_size size;

} pjsua_vid_win_info;


/**
 * Enumerates all video windows.
 *
 * @param id		Array of window ID to be initialized.
 * @param count		On input, specifies max elements in the array.
 *			On return, it contains actual number of elements
 *			that have been initialized.
 *
 * @return		PJ_SUCCESS on success, or the appropriate error code.
 */
PJ_DECL(pj_status_t) pjsua_vid_enum_wins(pjsua_vid_win_id wids[],
					 unsigned *count);


/**
 * Get window info.
 *
 * @param wid		The video window ID.
 * @param wi		The video window info to be initialized.
 *
 * @return		PJ_SUCCESS on success, or the appropriate error code.
 */
PJ_DECL(pj_status_t) pjsua_vid_win_get_info(pjsua_vid_win_id wid,
                                            pjsua_vid_win_info *wi);

/**
 * Show or hide window.
 *
 * @param wid		The video window ID.
 * @param show		Set to PJ_TRUE to show the window, PJ_FALSE to
 * 			hide the window.
 *
 * @return		PJ_SUCCESS on success, or the appropriate error code.
 */
PJ_DECL(pj_status_t) pjsua_vid_win_set_show(pjsua_vid_win_id wid,
                                            pj_bool_t show);

/**
 * Set video window position.
 *
 * @param wid		The video window ID.
 * @param pos		The window position.
 *
 * @return		PJ_SUCCESS on success, or the appropriate error code.
 */
PJ_DECL(pj_status_t) pjsua_vid_win_set_pos(pjsua_vid_win_id wid,
                                           const pjmedia_coord *pos);

/**
 * Resize window.
 *
 * @param wid		The video window ID.
 * @param size		The new window size.
 *
 * @return		PJ_SUCCESS on success, or the appropriate error code.
 */
PJ_DECL(pj_status_t) pjsua_vid_win_set_size(pjsua_vid_win_id wid,
                                            const pjmedia_rect_size *size);

Video Codec API

API for managing video codecs:

/**
 * Enum all supported video codecs in the system.
 *
 * @param id		Array of ID to be initialized.
 * @param count		On input, specifies max elements in the array.
 *			On return, it contains actual number of elements
 *			that have been initialized.
 *
 * @return		PJ_SUCCESS on success, or the appropriate error code.
 */
PJ_DECL(pj_status_t) pjsua_vid_enum_codecs( pjsua_codec_info id[],
					    unsigned *count );


/**
 * Change video codec priority.
 *
 * @param codec_id	Codec ID, which is a string that uniquely identify
 *			the codec (such as "H263/90000"). Please see pjsua
 *			manual or pjmedia codec reference for details.
 * @param priority	Codec priority, 0-255, where zero means to disable
 *			the codec.
 *
 * @return		PJ_SUCCESS on success, or the appropriate error code.
 */
PJ_DECL(pj_status_t) pjsua_vid_codec_set_priority( const pj_str_t *codec_id,
						   pj_uint8_t priority );


/**
 * Get video codec parameters.
 *
 * @param codec_id	Codec ID.
 * @param param		Structure to receive video codec parameters.
 *
 * @return		PJ_SUCCESS on success, or the appropriate error code.
 */
PJ_DECL(pj_status_t) pjsua_vid_codec_get_param(
					const pj_str_t *codec_id,
					pjmedia_vid_codec_param *param);


/**
 * Set video codec parameters.
 *
 * @param codec_id	Codec ID.
 * @param param		Codec parameter to set. Set to NULL to reset
 *			codec parameter to library default settings.
 *
 * @return		PJ_SUCCESS on success, or the appropriate error code.
 */
PJ_DECL(pj_status_t) pjsua_vid_codec_set_param( 
					const pj_str_t *codec_id,
					const pjmedia_vid_codec_param *param);