Custom Query (2195 matches)

Audio device open APIs, as described in the specifications, treat any negative device ID value indifferently, i.e: as default or first available device. However, some new audio device APIs differentiate -1 and -2 as PJMEDIA_AUD_DEFAULT_CAPTURE_DEV and PJMEDIA_AUD_DEFAULT_PLAYBACK_DEV, such as pjmedia_aud_dev_default_param().

Unfortunately some device open API implementations are currently not normalizing the negative audio index before calling pjmedia_aud_dev_default_param(). So for example, on a system without input device, calling pjmedia_snd_port_create_player() with device index param set to -1 (=PJMEDIA_AUD_DEFAULT_CAPTURE_DEV) will return error PJMEDIA_EAUD_NODEFDEV.

Thanks Daljeet Singh for the report.

Since 2.0, we require codec factory to implement "destroy" op (see #1294).

Bug in the multipart message will cause user defined headers to be printed without newlines.

In some unattended call transfer scenario, it looks like we miss sending some NOTIFY requests, and especially the NOTIFY containing 200/OK sipfrag status. This causes transferer to keep the old call active instead of disconnecting it.

The tonegen (tone generator) source files were missing in WinCE/EVC4 project files.

Missing keyframe is a major and common issue in real-time video session. Packet lost or discarding packets by jitter buffer (opening video devs may take few seconds) in the beginning of video session could be the main cause.

We experienced this problem a lot in sipit! When using H.264, missing keyframe causes pjsua got no video (decoding failure), sometime video could be successfully decoded after call hold and resume. When using H.263, initial video came foggy (only the moving object part had clear view, background/other part was gray).

Solutions/ideas:

1. Prevention:
   • for receiving direction:
     • prevent jitter buffer to discard first packet and following packets with same timestamp,
     • and/or, as long as there is space in jitter buffer, avoid dropping any video packets, just adjust the playback time (make it faster/slower), if possible.
   • for sending direction: force sending keyframes in some short periods in the beginning of video session for few seconds.
2. Implementing media feedback/control mechanisms (for both directions):
   • Full Intra Request in ​RFC 5104 3.5.1
   • Picture Loss Indication feedback in ​RFC 4585 6.3.1
   • Fast Update Command for the Full Picture in ​RFC 5168 7.1. Note that some implementations refuse to send us SIP INFO request because we don't list "INFO" in our SIP header "Allow".

Note:

• Missing keyframe detection has been implemented in this ticket along with SIP INFO mechanism.
• Check #1437 for RTCP-FB PLI mechanism.

While the media transport API interface have changed significantly from 0.8 because of SRTP (ticket #467), it still has some drawbacks, especially related to subsequent offer/answer. This ticket will make it right.

Some of the discussions for the draft modification is written here: 0.9/Media_Interface_Change

When account proxy is modified, i.e: via pjsua_acc_modify(), re-registration will be automatically performed, but the re-registration doesn't use the new proxy setting.

Modifiable settings to be implemented:

• pjsua_acc_config.mwi_expires: subscription expiration interval (compile-time default & run-time)
• pjsua_acc_config.mwi_enabled: to enable or disable MWI subscription. It can be updated via pjsua_acc_modify() to start or stop MWI subscription.

When there is a MWI subscription state change, pjsua callback on_mwi_state() will be called.

MWI subscription will be terminated when deleting the associated account.

Thanks Peter Koletzki and Matynka Bartosz for the feedbacks.

The proxy use case of PJSIP will introduce deadlock and other racing condition when using the transaction. Consider the following case:

Thread 1:

• UAC side receives response and is forwarding it to UAS transaction

Thread 2:

• UAS gets transport disconnection event and instructing UAC to CANCEL

Also there is a synchronization issue where a transaction is holding a stale pointer to the other transaction.

These problems can be solved by changing the transaction to use the Group Lock (introduced in PJSIP 2.1) and arrange the usage as follows:

1. Create the UAC transaction by specifying UAS's group lock as the group lock to use.
2. This way, both transactions will be using the same lock which will prevent deadlock.
3. The transactions also make use of the reference counting mechanism of the group lock. This way, both transactions will be destroyed simultaneously ONLY when both have indicated that they can be destroyed. This will prevent the stale pointer problem.

Thank you Mike Evans for reporting the problem and volunteering to test the initial patch.

Using asynchronous dispatch to the main queue may cause application to use objects prematurely (such as using UIView before it's initialized). This ticket will fix it by doing the dispatch synchronously instead.

Previously when the sink port is busy, the connection is rejected due to conf switch limitation (that a port can only has one transmitter), however it is rather uncomfortable in practice (e.g: attended call transfer), so this ticket will change the behavior: the destination port should be released first so the new connection can be made.

During cross-compilation, currently configure script will automatically detect the libraries on the build machine instead of the target host, so this ticket will disable autodetection in cross-compilation. Users now need to supply the configure script with the option "--with-A" to use feature A.

Modify configure-iphone to use the latest version of gcc instead of using the first available version since the latest version provides better support for latest iOS versions.

In addition to checking encoding name and clock rate, pjmedia_sdp_neg_fmt_match() API will check encoding parameters as well.

From: ​https://tools.ietf.org/html/rfc4566#page-25

a=rtpmap:<payload type> <encoding name>/<clock rate> [/<encoding parameters>]

For audio streams, <encoding parameters> indicates the number of audio channels. This parameter is OPTIONAL and may be omitted if the number of channels is one, provided that no additional parameters are needed.

For video streams, no encoding parameters are currently specified.

See ticket #600 (Solution 1).

When creating video port using pjmedia_vid_port_create(), video port may open a video stream with different parameters than requested. On output, the parameter will be set with the actual parameter used to open the video stream.

Video port with active role will now create its own clock so that it is independent of the video capture device's clock. This allows video port to support fps which is different than video device's fps.

When pjsua_vid.c's call_modify_video() is called, the current SDP will be cloned then the media transport attributes will be added to the SDP by calling pjmedia_transport_encode_sdp(). This can cause duplicate attributes if the current SDP already has media transport attributes.

To reproduce using pjsua: Enable ICE, establish a call, modify the video stream (such as "vid call rx off 1")

Often applications that implement pjsip_module's on_tx_request() callback need to find out what context is associated with the message (for example, the PJSUA-LIB call id or account id). Without the module data field, this can only be done by looking up the message headers such as From or Call-ID, and this is inefficient.

Currently the UDP and ICE media transport has the capability to switch destination RTP/RTCP address to the source address of the RTP/RTCP packets, if they are different than the original address set in SDP received from remote.

It may be better to put this functionality in the stream instead, for the following reason:

1. the stream has more information about the packet, such as RTP pt, SSRC, and info from RTCP statistic
2. some application that uses media transport wants to inspect and possibly filter the source address
3. some firewall such as Microsoft Forefront Threat Management Gateway wants us to send the RTP to the address in SDP, regardless of the source address of the RTP packet

Currently the UDP and ICE media transport has the capability to switch destination RTP/RTCP address to the source address of the RTP/RTCP packets, if they are different than the original address set in SDP received from remote.

The problem is audio playback is broken if more than one remotes are transmitting to us simultaneously, e.g. on early media forking scenario.

This ticket modifies the behavior so that the transmission won't switch as long as we're receiving RTP/RTCP packets from the "correct" remote address. Also when the remote peer address is still in probation, RTP/RTCP packets from that address will be silently discarded, to avoid playback confusion.

Note that there is a more ambitious proposal in #865.

This ticket will be the placeholder for various fixes/modifications in order to comply with the guidelines/recommendations in Apple's official doc ​Supporting IPv6 DNS64/NAT64 Networks.

This issue is related to #1834. INVITE session uses an alternating set of memory pools where the last used is reset after a cycle of SDP negotiation, this requires SDP negotiator to retain some of its internal states using the current pool in each SDP negotiation cycle. This ticket will refactor the management of internal state initial_sdp.

The initial_sdp contains local capability (original complete local SDP) provided by application when setting local offer. Currently it needs to be retained for initial and subsequent SDP negotiation operations, one of its usage is to be the default SDP when application supplied NULL SDP in pjmedia_sdp_neg_set_local_answer().

There are some scenarios of SDP negotiation that may cause initial_sdp to be invalid, e.g:

• remote sends re-offer, so local supplies initial answer and initial_sdp is updated using pool 1, but SDP negotiation fails (e.g: due to no matching codec) and the pool 1 is then reset
• local sends re-offer, so initial_sdp is updated using pool 1, but SDP negotiation fails and the pool 1 is then reset
• SDP negotiation cancellation (due to no remote answer) may not immediately cause initial_sdp to be invalid, but as initial_sdp is allocated using provisional pool, it is considered risky as provisional pool is not active pool, it may be reset by other component on any event.

Here are SDP offer/answer operations and actions to be done on initial_sdp:

• updating local capability via pjmedia_sdp_neg_modify_local_offer, pjmedia_sdp_neg_send_local_offer, or pjmedia_sdp_neg_set_local_answer:
  • save current initial_sdp to a temporary variable (for reverting back later if needed) and update initial_sdp with the new local SDP offer
• (local/remote) offer cancelled via pjmedia_sdp_neg_cancel_offer:
  • revert back initial_sdp when the offerer is local
• negotiation failure (pjmedia_sdp_neg_negotiate):
  • revert back initial_sdp
• negotiation success (pjmedia_sdp_neg_negotiate):
  • nothing, the provisional pool will become the active pool

Enhancements to pjsua:

• the --video option now works for local accounts too
• added new commands:
  • vid acc show: show current video settings in acc
  • vid acc enable|disable: enable/disable video in acc
  • vid acc autotx on|off: set video auto transmit on/off.
  • vid acc autorx on|off: set incoming video autoshow on/off
  • vid acc cap ID: set acc capture dev
  • vid acc rend ID: set acc render dev

Related to this ticket, see also ticket #364

Accept the following non-standard scenarios in the SDP negotiator:

1. Receiving answer with non-matching media lines (e.g. offer has m=audio and m=video lines, and answer only has m=audio line). In this case set the unmatched local media line with port zero.
2. Generating subsequent offer using sdp_neg_modify_local_offer() with non-matching media lines (e.g. current/active SDP's have m=audio and m=video lines, and the new offer only has m=audio line). In this case, the negotiator will fix the new offer by adding the missing media line, with port number set to zero. Also the negotiator will overwrite the session ID and version number in the SDP origin field to make sure that the session ID stays the same and version number has been increased.

Add good_ns_ttl and bad_ns_ttl in pj_dns_settings.

This ticket continues the work on PJSIP's AutomatedTesting which was added in PJSIP 1.6 by ticket #1041.

Some PJSIP config (e.g. PJSIP_MAX_TSX_COUNT) may better be implemented as a run-time configs instead of compile time, to enable multiple applications to be built on a single PJSIP tree.

Currently SRTP crypto and keying method is configurable via callback, i.e: on_create_media_transport_srtp() in PJSUA or onCreateMediaTransportSrtp() in PJSUA2 (only crypto). And after some review and internal discussions, we decided to deprecate the callback and move the settings to account config, here are some reasons:

• SRTP setting type used in callback param is pjmedia_srtp_setting, which has some fields that are PJMEDIA specifics, e.g: close_member_tp, cb, user_data. So this ticket will also create a new SRTP settings specifically for PJSUA and PJSUA2.
• media transport settings are usually configurable via account setting (instead of PJSUA/PJSUA2 callback), so it'd better to use the same pattern for SRTP.

Here are sample codes to enable DTLS-SRTP in outgoing SDP offer:

• using PJSUA:

   acc_cfg.srtp_opt.keying_count = 2;
   acc_cfg.srtp_opt.keying[0] = PJMEDIA_SRTP_KEYING_DTLS_SRTP;
   acc_cfg.srtp_opt.keying[1] = PJMEDIA_SRTP_KEYING_SDES; // optional, as a fallback for handling incoming call using SRTP with SDES
  

• using PJSUA2:

  acc_cfg.mediaConfig.srtpOpt.keyings.push_back(PJMEDIA_SRTP_KEYING_DTLS_SRTP);
  acc_cfg.mediaConfig.srtpOpt.keyings.push_back(PJMEDIA_SRTP_KEYING_SDES); // optional, as a fallback for handling incoming call using SRTP with SDES
  

Currently each open operation of iOS audio device will reset AVAudioSession category to AVAudioSessionCategoryPlayAndRecord (and without any option), so there is a risk that any audio session settings set by application to be reseted or overridden. To avoid that, the category setting will be moved to audio device factory initialisation.

The resample API already supports multi-channel input/output, however the channel number is not used for some resample backends (e.g. libresample).

Sample cases:

• pjsip_inv_verify_request2() fails to collect all extensions supported when Supported header formed in multiple rows (only the first row processed), such as:

  Supported: replaces, 100rel, norefersub
  Supported: timer
  

• similar to above, for Require header.

1) pj_timer_heap_poll examines the timer heap’s current size without taking its lock As in the summary, pj_timer_heap_poll examines ht->cur_size before taking the lock. Another thread could be manipulating this at the same time.

2) Possible deadlock due to “lock order inversion” between transport and transaction locks Normally, when sending a message, the transaction lock is taken (in pjsip_tsx_send_msg), followed by the transport lock (in tcp_send_msg). However, when sending the first message on a new connection, we receive a callback from the ioqueue layer to say that the TCP connection is connected (on_connect_complete), causing us to take the transport lock, and then take the transaction lock (in send_msg_callback). Taking locks in different orders in different places is a common cause of deadlock so we would like to prevent that happening.

According to ​http://tools.ietf.org/html/rfc3261#section-15:

   However, the callee's UA MUST NOT send a BYE on a confirmed dialog
   until it has received an ACK for its 2xx response or until the server
   transaction times out.

When the application is behind firewall, maybe the stack needs to periodically send keep-alive packet (probably to STUN server) to keep the NAT binding open.

Add support for clients on IPv6-only NAT64 environment to communicate with IPv4 servers and endpoints.

Please refer to the NAT64 wiki.

When receiving SIP message within/creating dialog, PJSIP should rewrite the Contact (and first Route if it is an initial INVITE transaction) if it is using IPv4 address, as otherwise, PJSIP will send a new request (within dialog) to that IPv4 address.

Thanks Kyle Kurz for the report.

When the port is not receiving signal from any other ports (e.g. when no other ports is transmitting to this port), the bridge periodically transmit NULL frame to the port to keep the port "alive" (for example, a stream port needs this heart-beat to periodically transmit silence frame to keep NAT binding alive).

This NULL frame should be sent to the port at the port's ptime rate. So if the port's ptime is greater than the bridge's ptime, the bridge needs to delay the NULL frame until it's the right time to do so.

This bug transmits NULL frame at the bridge's ptime interval and not the port's ptime interval.

Port Linux ALSA audio device backend implementation from the old sound device API to the pjmedia_audiodev framework.

The native WMME audio device implementation (wmme_dev.c) provides richer API than PortAudio, and by this ticket it is now used as the default audio device backend on Windows platforms.

coreaudio_dev now supports recording using a different sample rate than the hardware's sample rate by using Audio Converter for sample rate conversion.

Multi-thread Python application will need to register its threads to PJSUA before the thread can call any of the PJSUA functions.

Thanks Johan for the patch.

The jitter buffer provides pjmedia_jbuf_get_state() API to monitor its state, however this API is not accessible to application because jitter buffer instance is private to the stream/session.

This ticket adds new API to retrieve the jitter buffer state from the stream/session instance.

The application is located under pjsip-apps/src/swig/java/android. Check this wiki for build info.

The ​pj_sockaddr_parse() was added in version 1.3 to parse "HOSTPART[:PORT]" into a ​pj_sockaddr structure, possibly resolving the hostpart into IP address.

The new API is ​pj_sockaddr_parse2(), which just returns the hostpart as string and optional port number. This is useful if caller does not want to resolve the hostpart into IP address (for example, if caller wants to resolve the hostpart with DNS SRV).

New PJLIB API to parse string containing socket address (IP address and optional port):

PJ_DECL(pj_status_t) pj_sockaddr_parse(int af, unsigned options,
				       const pj_str_t *str,
				       pj_sockaddr *addr);

This function supports parsing several formats. Sample IPv4 inputs and their default results::

• "10.0.0.1:80": address 10.0.0.1 and port 80.
• "10.0.0.1": address 10.0.0.1 and port zero.
• "10.0.0.1:": address 10.0.0.1 and port zero.
• "10.0.0.1:0": address 10.0.0.1 and port zero.
• ":80": address 0.0.0.0 and port 80.
• ":": address 0.0.0.0 and port 0.

This will also work if host resolution is enabled (i.e. gethostbyname() works):

• "localhost": address 127.0.0.1 and port 0.
• "localhost:": address 127.0.0.1 and port 0.
• "localhost:80": address 127.0.0.1 and port 80.

Sample IPv6 inputs and their default results:

• "[fec0::01]:80": address fec0::01 and port 80
• "[fec0::01]": address fec0::01 and port 0
• "[fec0::01]:": address fec0::01 and port 0
• "[fec0::01]:0": address fec0::01 and port 0
• "fec0::01": address fec0::01 and port 0
• "fec0::01:80": address fec0::01:80 and port 0
• "::": address zero (::) and port 0
• "[::]": address zero (::) and port 0
• "[::]:": address zero (::) and port 0
• ":::": address zero (::) and port 0
• "[::]:80": address zero (::) and port 0
• ":::80": address zero (::) and port 80

Note: when the IPv6 socket address contains port number, the IP part of the socket address should be enclosed with square brackets, otherwise the port number will be included as part of the IP address (see "fec0::01:80" example above).

The APIs will be useful for application that needs to work with multiple sound devices concurrently. It can also help application to simplify the tasks of improving media framework clock as described here.

Specifications

1. Audio device configuration settings to be supported:
   • PJSUA: ​pjmedia_snd_port_param.
   • PJSUA2: only playback and record device IDs.
2. Application can apply further settings whenever needed by:
   • PJSUA: querying the sound device port and apply further settings directly to it using PJMEDIA Sound Device Port APIs.
   • PJSUA2: inheriting class ExtraAudioDevice and apply the settings using the same approach as PJSUA above. This needs be done in C++.
3. Audio device must be registered to the conference bridge:
   • PJSUA: it must have API to query the conference bridge port.
   • PJSUA2: it must be a descendant of ​AudioMedia.

Sample code using PJSUA

enum { EXTRA_SND_DEV_ID  = 3; };

pjmedia_snd_port_param ext_param;
pjsua_ext_snd_dev *ext_snd_dev;
pjsua_conf_port_id ext_id;

/* Generate params (with default values) */
status = pjmedia_snd_port_param_default(&ext_param);
status = pjmedia_aud_dev_default_param(EXTRA_SND_DEV_ID, &ext_param.base);

/* Create the extra audio device */
status = pjsua_ext_snd_dev_create(&ext_param, &ext_snd_dev);
ext_id = pjsua_ext_snd_dev_get_conf_port(ext_snd_dev);

/* Connect extra audio dev mic to main audio dev */
status = pjsua_conf_connect(ext_id, 0);

/* Connect main audio dev mic to extra audio dev */
status = pjsua_conf_connect(0, ext_id);

...

/* Destroy extra audio dev (after no longer used) */
pjsua_ext_snd_dev_destroy(ext_snd_dev);

Sample code using PJSUA2

/* Use Null Audio Device as main media clock. This is useful for improving
 * media clock (see also https://trac.pjsip.org/repos/wiki/FAQ#tx-timing)
 * especially when sound device clock is jittery.
 */
ep.audDevManager().setNullDev();

/* Install extra audio device */
ExtraAudioDevice *auddev2 = new ExtraAudioDevice(-1, -1);
try {
    auddev2->open();
} catch (...) {
    std::cout << "Extra sound device failed" << std::endl;
}

/* Create WAV player and play the WAV to extra audio speaker */
AudioMediaPlayer amp;
amp.createPlayer(PATH_TO_WAV_FILE);
if (auddev2->isOpened())
    amp.startTransmit(*auddev2);

/* Wait for the WAV playback */
pj_thread_sleep(5000);

...

/* Destroy extra audio device (after no longer used) */
delete auddev2;

Add PJSUA API to register transport factory that has been created by application. This can be useful if application wants to implement custom SIP transport and use it with pjsua.

Thanks to Peter Koletzki for the patch.

Currently there is no API to re-initiate presence subscription for a buddy when previous subscription failed. Although the library periodically refreshes the presence subscription for all buddies, some application may want to refresh the buddy's presence subscription immediately (for example upon receiving incoming MESSAGE), and this function is used for that purpose.

Currently, any incoming re-offer will be auto answered with 200/OK and the SDP answer is generated based on the current call setting. With this new callback, application will have a chance to reject the re-offer (so the existing media session will remain active), or accept the re-offer with updated call setting.

New callbacks:

• on_stream_created(): this callback is called to notify application when stream is created and about to be registered to conference bridge. Application may use this callback to add media processing ports to the stream.
• on_stream_destroyed(): this callback is called to notify application when stream is about to be destroyed. If application created media processing ports to the stream, it may use this callback to clean-up resources associated with the media stream. Also if application manages it's own media (see [FAQ#pjsua-lib-perf this FAQ entry]), application will need to use this callback to signal media processing that stream is about to be destroyed.

Thanks Norman Franke and Truong Thanh Quang for the suggestions.

The pjsua_acc_config.contact_params setting was used to add user defined parameters to the Contact URI. Some applications may need to add user defined parameters to the Contact header, and this will be supported by this ticket.

For consistency with RFC 3261, this will modify the semantic of the existing pjsua_acc_config.contact_params field as follows:

• pjsua_acc_config.contact_params is now used to specify additional Contact header parameters
• a new field, pjsua_acc_config.contact_uri_params, will be added, and this will be used to specify additional Contact URI parameters (the previous semantic of pjsua_acc_config.contact_params)

Some application would like to inspect the response of REGISTER response, and currently this is not possible since the response is not reported in on_reg_state() callback.

This ticket adds a new callback, on_reg_state2(), which also adds the whole response message as well as possibly other infos as parameter.

This ticket depends on #923 and #924

Screenshot:

jbsim - System and network impairments simulator
Copyright (C) 2008-2009 Teluu Inc. (http://www.teluu.com)

This program emulates various system and network impairment
conditions as well as application parameters and apply it to
an input WAV file. The output is another WAV file as well as
a detailed log file (in CSV format) for troubleshooting.

Usage:
 jbsim [OPTIONS]

General OPTIONS:
  --codec, -c NAME       Set the audio codec
                         Default: PCMU
  --input, -i FILE       Set WAV reference file to FILE
                         Default: ../../tests/pjsua/wavs/input.8.wav
  --output, -o FILE      Set WAV output file to FILE
                         Default: jbsim.wav
  --duration, -d SEC     Set test duration to SEC seconds
                         Default: 20
  --log-file, -l FILE    Save simulation log file to FILE
                         Note: FILE will be in CSV format with semicolon separator
                         Default: jbsim.out
  --help, -h             Display this screen

Simulation OPTIONS:
  --loss, -x PCT         Set packet average loss to PCT percent
                         Default: 0
  --loss-corr PCT        Set the loss correlation to PCT percent. Default: 0
  --min-lost-burst N     Set minimum packet lost burst (default:0)
  --max-lost-burst N     Set maximum packet lost burst (default:20)
  --min-jitter, -j MSEC  Set minimum network jitter to MSEC
                         Default: 0
  --max-jitter, -J MSEC  Set maximum network jitter to MSEC
                         Default: 0
  --snd-burst, -b VAL    Set RX sound burst value to VAL frames.
                         Default: 1
  --tx-ptime, -t MSEC    Set transmitter ptime to MSEC
                         Default: 0 (not set, use default)
  --rx-ptime, -r MSEC    Set receiver ptime to MSEC
                         Default: 0 (not set, use default)
  --no-vad, -U           Disable VAD/DTX in transmitter
  --no-plc, -p           Disable PLC in receiver
  --jb-prefetch, -P      Enable prefetch bufferring in jitter buffer
  --jb-min-pre, -m MSEC  Jitter buffer minimum prefetch delay in msec
  --jb-max-pre, -M MSEC  Jitter buffer maximum prefetch delay in msec
  --jb-max, -X MSEC      Set maximum delay that can be accomodated by the
                         jitter buffer msec.

Application may need to inspect additional info in the NOTIFY message, hence the need for the event parameter.

A new callback is added to pjsua_callback to allow application to be notified when the library has started or stopped registration:

    /**
     * Notify application when registration or unregistration has been
     * initiated. Note that this only notifies the initial registration
     * and unregistration. Once registration session is active, subsequent
     * refresh will not cause this callback to be called.
     *
     * @param acc_id	    The account ID.
     * @param renew	    Non-zero for registration and zero for
     * 			    unregistration.
     */
    void (*on_reg_started)(pjsua_acc_id acc_id, pj_bool_t renew);

Implement a new callback to notify application that sound device is about to be opened or closed. This is useful if application wants to keep track whether the sound device is currently active or idle, and also will be very useful if application manages the sound device by itself since this callback notifies the application when the library needs to use the sound device and when it's safe to close it, taking into account the sound device idle timer.

Prototype of the callback:

    /**
     * Callback when the sound device is about to be opened or closed.
     * This callback will be called even when null sound device or no
     * sound device is configured by the application (i.e. the
     * #pjsua_set_null_snd_dev() and #pjsua_set_no_snd_dev() APIs).
     * This API is mostly useful when the application wants to manage
     * the sound device by itself (i.e. with #pjsua_set_no_snd_dev()),
     * to get notified when it should open or close the sound device.
     *
     * @param operation	The value will be set to 0 to signal that sound
     * 			device is about to be closed, and 1 to be opened.
     *
     * @return		The callback must return PJ_SUCCESS at the moment.
     */
    pj_status_t (*on_snd_dev_operation)(int operation);

When stream is configured to send multiple frames per packet, currently this will cause the stream to encode and pack N frames simultaneously in one call to put_frame(). This may put significant load on the CPU since we are encoding that many frames at once and some device may be sensitive to the delay. It has been reported that this may cause audio disruption on iPhone 4S.

This ticket implements new interface to the codec to enable encoding frame one by one on each put_frame() cycle and only pack them together once we have enough frame to send. The new codec interface looks like the following:

    /**
     * Encode a single audio frame without performing any packing to
     * the output. Normally the input frame should have exactly the same
     * ptime as the codec frame time, however when the codec frame ptime
     * is 10 ms, this may be called with input frame containing 20ms
     * worth of samples.
     *
     * @param codec	The codec instance.
     * @param input	Input audio frame.
     * @param out_size	The length of buffer in the output frame.
     * @param output	The output frame.
     *
     * @return		PJ_SUCCESS on success or the appropriate error code.
     */
    pj_status_t (*encode_single)(pjmedia_codec *codec,
				 const struct pjmedia_frame *input,
				 unsigned out_size,
				 struct pjmedia_frame *output);

    /**
     * Pack one or more encoded audio frames into packet, suitable to be
     * placed as payload of a transport such as RTP.
     *
     * @param codec	The codec instance.
     * @param pack_cfg	Codec packing settings to specify how audio frames
     * 			should be arranged in the output buffer. Use NULL
     * 			to use codec's default packing setting.
     * @param frame_cnt	Number of input frames.
     * @param inputs	Input frames, which is an array of encoded frames.
     * @param out_size	Size of the output buffer.
     * @param output	Frame buffer to receive the output.
     *
     * @return		PJ_SUCCESS on success.
     */
    pj_status_t (*pack)(pjmedia_codec *codec,
		        pjmedia_codec_pack_cfg *pack_cfg,
			unsigned frame_cnt,
			pjmedia_frame inputs[],
			unsigned out_size,
			struct pjmedia_frame *output);

The stream will be modified to support this feature.

There will also be a new field in codec param to enable/disable the new feature on per stream basis:

struct pjmedia_codec_param
{
    ...

    struct {
        ...
	unsigned    pack:1;	    /**< Enable encode_single()&pack()  */
        ...
    } setting;
};

Only if pjmedia_codec_param.setting.pack is set to 1, the stream will use the new interface, and only if the codec supports it.

The first implementation will add this feature to G.711 and OpenCore AMR codecs.

New mixing algorithm for conference bridge: rather than doing multiplication of signal from sources, we can just add the signal from the sources. The benefits of this approach are it doesn't weaken the input signals, and it is faster since the multiplication is replaced by simple addition.

We already have the facility to insert custom header(s) into outgoing requests via pjsua_msg_data facility, but this lacks the capability to add the header into REGISTER request.

Two new features are introduced by this ticket:

1. Indentation

Indentation of log message to give grouping context of the messages. Example:

21:13:35.787   pjsua_call.c  Answering call 1: code=200
21:13:35.787  pjsua_media.c  ...Call 1: updating media..
21:13:35.787  pjsua_media.c  .....Media session call01:0 is destroyed
21:13:35.787  pjsua_media.c  ....Audio channel update..
21:13:35.787  strm0x22f04d8  .....VAD temporarily disabled
21:13:35.788  strm0x22f04d8  .....Encoder stream started
21:13:35.788  strm0x22f04d8  .....Decoder stream started
21:13:35.788    pjsua_app.c  ...Call 1 media 0 [type=audio], status is Active
21:13:35.788  pjsua_media.c  ...Conf disconnect: 2 -x- 0
21:13:35.788  pjsua_media.c  ...Conf connect: 3 --> 0
21:13:35.788  pjsua_media.c  ...Conf connect: 0 --> 3

Use pj_log_push_indent() and pj_log_pop_indent() to use this new feature. The PJ_LOG_HAS_INDENT must be set in the log decoration flag (it is set by default by PJSUA-LIB).

The indentation is applied on per thread basis.

The complete features introduced by this ticket are:

• new functions: pj_log_add_indent(int), pj_log_push_indent(), pj_log_pop_indent()
• new log decor: PJ_LOG_HAS_INDENT
• new macros: PJ_LOG_ENABLE_INDENT (1), PJ_LOG_INDENT_SIZE (1), PJ_LOG_INDENT_CHAR ('.')

Sample code:

#include <pj/log.h>
#define THIS_FILE  "sample.c"

void inner()
{
    PJ_LOG(4,(THIS_FILE, "We're in inner"));
}

void outer()
{
    PJ_LOG(4,(THIS_FILE, "Calling inner"));
    pj_log_push_indent();
    inner();
    pj_log_pop_indent();
}

int main()
{
    pj_log_set_decor(pj_log_get_decor() | PJ_LOG_HAS_INDENT);
    outer();
    return 0;
}

The sample code above will produce:

21:13:35.787       sample.c  Calling inner
21:13:35.787       sample.c  .We're in inner

2. Thread Switch Indication

An exclamation mark will be placed before a log message if the caller thread is different than the caller thread of the previous message.

Example:

21:13:35.787       sample.c  Thread A is calling the log
21:13:35.787       sample.c  ..This one is still from thread A
21:13:35.787       sample.c !Thread B appears (indentation has changed too)
21:13:35.787       sample.c  Thread B again
21:13:35.787       sample.c !..Thread A resumes (indentation changed back)

This feature is controlled by the new PJ_LOG_HAS_THREAD_SWC log decor. It is set by default by PJSUA-LIB.

New logging flag PJ_LOG_HAS_THREAD_ID can be used to include the caller thread ID in the log message. This would be useful to troubleshoot threading related problems such as deadlocks.

Application would have to activate this flag explicitly to use this feature, either with pj_log_set_decor() or via pjsua_logging_config.decor, since this option is not used by default.

The user credentials lookup callback is currently only passing realm and account name info, and for some cases it may be insufficient. Additional info such as 'pjsip_rx_data' is considered to provide better flexibility for the application so that it can fetch any information from the message in order to lookup the user credentials.

Thanks Mike Evans for the suggestion.

pjsua.py

Implement new objected oriented Python wrapper for pjsua API. This is a new Python module, and it is implemented purely in Python, on top of the C module.

_pjsua C python module

This is the implementation of the Python C module. It is based on the old py_pjsua.c module with some changes and new features. The py_pjsua.c module is now deprecated.

On IP Address change case, app might want to close the transport before connecting state, however it is not possible to get the transport instance before registration process is finished.

This ticket is mainly for getting registration transport instance before getting the registration result (via pjsua callback on_reg_state).

Example scenario:

1. Connecting to network A (3G)
2. Check if a transport is saved. Finding None so go on.
3. Register requested (and account created)
4. Transport created in PJSIP
5. Connecting to network B (WIFI)
6. Register request on network B (old account destroyed and new one created)
7. Check if a transport is saved. Finding None so go on.
8. Transport A and transport B connect (may happen in an iphone since the 3G connection is never really lost).
9. Transport A is saved and then transport B is saved in its stead
10. If on transport connect we also check if there is already a transport connected and then release the old one (which we are doing at the time), then if by chance transport B connects first, then network A would actually cause a release of transport B which is the latest one
11. Otherwise, transport A is never released.

on_reg_started callback can be used for getting registration transport instance, but we need to include pjsua_internal.h to get account's pjsip_regc instance (to eventually get the registration transport). With this ticket, it can be done as follow:

static void on_reg_started2(pjsua_acc_id acc_id, pjsua_reg_info *info)
{
    pjsip_regc_info regc_info;
    pjsip_regc_get_info(info->regc, &regc_info);

    // transport instance is in 'regc_info.transport'
    ...
}

Currently the REGISTER request will be sent with Route header(s), if outbound proxies are set in the global config or proxies are set in the account config.

Some provider would like the REGISTER to be sent directly to the registrar, and this is not possible with existing settings.

This ticket adds a new account option to control if and how the Route headers will be added to the REGISTER request.

Currently bad NOTIFY message body (for presence) will be responded automatically with 488 (Not Acceptable Here), and application callback will not be called. Sample of bad NOTIFY requests:

• NOTIFY containing non-PIDF/X-PIDF
• NOTIFY with multipart message bodies but without PIDF/X-PIDF part
• NOTIFY containing bad PIDF/X-PIDF document

This ticket adds PJSIP_PRES_BAD_CONTENT_RESPONSE compile time setting with the following spec:

/**
 * Specify the status code value to respond to bad message body in NOTIFY
 * request for presence. Scenarios that are considered bad include non-
 * PIDF/XML and non-XPIDF/XML body, multipart message bodies without PIDF/XML
 * nor XPIDF/XML part, and bad (parsing error) PIDF and X-PIDF bodies
 * themselves.
 *
 * Default value is 488. Application may change this to 200 to ignore the
 * Unrecognised content (this is useful if the application wishes to handle
 * the content itself). Only non-3xx final response code is allowed here.
 *
 * Default: 488 (Not Acceptable Here)
 */
#ifndef PJSIP_PRES_BAD_CONTENT_RESPONSE
#   define PJSIP_PRES_BAD_CONTENT_RESPONSE	488
#endif

PJSUA-LIB has a feature to update account Contact with the public address as it is learned from the REGISTER response. It does this by unregistering the current binding, while at the same time establishing a brand new registration session with a new Contact.

This method would not work well in all cases though, as some server would immediately close the TCP connection as soon as unregister request is received. In this case, it will be better to perform the Contact URI update in a single REGISTER request instead.

Implement new packet loss concealment (PLC) module based on the new WSOLA algorithm.

New API in <pj/errno.h>:

PJ_PERROR usage:

  PJ_PERROR(2, (__FILE__, PJ_EBUSY, "Error making %s", "coffee"));

pj_perror() API:

   PJ_DECL(void) pj_perror(int log_level, const char *sender, 
                           pj_status_t status,
		           const char *title_fmt, ...);

New API to register all known codecs that are implemented in PJMEDIA-CODEC, and to automatically deregister them on shutdown:

/* file <pjmedia-codec/audio_codecs.h> */

/**
 * Codec configuration. Call #pjmedia_audio_codec_config_default() to initialize
 * this structure with the default values.
 */
typedef struct pjmedia_audio_codec_config
{
    /** Speex codec settings. See #pjmedia_codec_speex_init() for more info */
    struct {
	unsigned	option;		/**< Bitmask of options.	*/
	unsigned	quality;	/**< Codec quality.		*/
	unsigned	complexity;	/**< Codec complexity.		*/
    } speex;

    /** iLBC settings */
    struct {
	unsigned	mode;		/**< iLBC mode.			*/
    } ilbc;

    /** Passthrough */
    struct {
	pjmedia_codec_passthrough_setting setting; /**< Passthrough	*/
    } passthrough;

} pjmedia_audio_codec_config;


/**
 * Initialize pjmedia_audio_codec_config structure with default values.
 *
 * @param cfg		The codec config to be initialized.
 */
PJ_DECL(void) pjmedia_audio_codec_config_default(pjmedia_audio_codec_config *cfg);

/**
 * Register all known audio codecs implemented in PJMEDA-CODEC library to the
 * specified media endpoint.
 *
 * @param endpt		The media endpoint.
 * @param c		Optional codec configuration, or NULL to use default
 * 			values.
 *
 * @return		PJ_SUCCESS on success or the appropriate error code.
 */
PJ_DECL(pj_status_t)
pjmedia_codec_register_audio_codecs(pjmedia_endpt *endpt,
                                    const pjmedia_audio_codec_config *c);

Note to codec writers:

A new mandatory destroy callback has been added to factory operation. You must implement this callback (hint: just point it to your existing function that deinitialize your codec factory).

Note to app writers:

You don't need to initialize each codec now, just use pjmedia_codec_register_audio_codecs() above. Similarly you should remove all the calls to codec deinitialization routines since that will be done automatically when the media endpoint is destroyed (leaving these calls as they are is harmless though; they will render to no-ops).

This is to replace the (newly added) pjsua_get_aud_stream() API that is unsafe, since pjsua-lib may close the stream while the application is holding the stream pointer.

New API is proposed by this ticket:

PJ_DECL(void) pjsua_destroy2(unsigned flags);

The flags argument may contain bitmask of the following options:

• PJSUA_DESTROY_NO_RX_MSG: allow sending outgoing messages (such as unregistration, event unpublication, BYEs, unsubscription, etc.), but do not wait for responses. This is useful to perform "best effort" clean up without delaying the shutdown process waiting for responses. But note that if the requests are challenged with authentication (and usually they are), this would not accomplish much if any.
• PJSUA_DESTROY_NO_TX_MSG: do not send any outgoing messages at all. This flag is useful if application knows that the network which the messages are to be sent on is currently down.
• PJSUA_DESTROY_NO_NETWORK: combination of the two above.

Introducing new application, pjsystest, to perform various tests on the target system or device. Currently it contains several audio test functions such as:

• simple audio playback
• simple audio capture
• audio device test
• latency test

In the future, the tests can be extended to cover other areas such as NAT.

Some applications want to get access to the message body in the incoming NOTIFY requests, for example when the NOTIFY contains multipart bodies (support for receiving multipart bodies in NOTIFY was implemented in #1146).

Included in this package:

• new pcap.[hc] file in PJLIB-UTIL to parse and read packets from PCAP file.
• new application (pcaputil.c) in samples collection to read RTP packets from PCAP file, decode the payload, and write the PCM samples to WAV file.

Scenario:

• run two pjsua with --use-100rel
• pjsua 1 make call to pjsua 2
• pjsua 2 answer with 183
• pjsua 2 answer with 200 after receiving PRACK

When OSX sound device is opened on non-native sampling rate, no audio input is audible.

Currently iLBC wrapper for iOS passes the whole PCM input packet (containing multiple frames) to the encoder, and this way the encoder always returns zero frame output. This ticket will alter the wrapper behaviour to feed only one PCM input frame at a time to encoder.

Thanks Yeffry Zakizon for the report.

When making a call and before media is received, loud noise/distortion may be played to the speaker.

There are two problems here:

1. Problem with upsampling that introduces noise, and
2. Problem with resampling/upsampling, specific when it's used in the conference bridge and only when two or more sources are mixed together.

For the first problem:

The following file was upsampled with pjsua, from the original WAV file of 22050 Hz to 44100 Hz. You can hear clearly about the noise:

​http://www.pjsip.org/tmp/upsampling-noise.wav

For the second problem:

The following file was recorded by mixing two 16KHz sources and upsampled to 44.1KHz:

​http://www.pjsip.org/tmp/conference_noise.wav

Sample case:

• Worker thread is receiving an RTP packet, it gets the ICE lock and about to get the SRTP lock, other thread is trying to send an RTP packet, it successfully acquire the SRTP lock, but unable to acquire ICE lock since it is hold by the worker thread, deadlock.

Solution:

• Make the locking order between SRTP & its member transport uniform or independent.

The DNS resolver encounters PJ_ENOMEM error when parsing and duplicating response.

This is copy of ticket #156 for trunk.

This ticket will notify when IP change handling using pjsua_handle_ip_change() is completed by adding PJSUA_IP_CHANGE_OP_COMPLETED to pjsua_ip_change_op.

Currently pjsua_set_null_snd_dev() will force thread creation for its clock source. This behaviour will prevent application to achieve single-threaded environment.

The original report/suggestion can be found ​here.

The proposed patch can be found ​here

Reported that there are object slicing issues in AudioDevInfo/VideoDevInfo when storing video/audio format detail info (i.e: MediaFormatVideo/MediaFormatAudio) into MediaFormatVector, this is due to the definition of MediaFormatVector that is a vector of MediaFormat (no pointer):

typedef std::vector<MediaFormat> MediaFormatVector;

This ticket will replace MediaFormatVector with MediaFormatVideoVector and MediaFormatAudioVector.

Thanks Ryan Wallach for the report and the suggestion.

Currently, SRTP optional mode is implemented by putting 'crypto' attributes in unsecured media line (with RTP/AVP transport) in the SDP offer. This non-standard approach is somehow getting popular.

The suggested approach is to offer duplicated media line, for secured and unsecured version, in the SDP. This approach is syntactically standard-friendly, however it may lack the way to tell UAS to choose/activate only one of the duplicated media.