Workaround for frame bursts and drifts from audio devices

[bennylp]

Audio burst:

Most audio devices will call record and playback callbacks in burst, rather than in perfect interleave (for example, burst of record callbacks, followed by burst of playback callbacks, and so on). Because of this, audio buffering needs to be implemented in few pjmedia components (conference bridge, splitcomb, echo canceller, etc.), and the maximum buffering is controlled by PJMEDIA_SOUND_BUFFER_COUNT macro.

This is not ideal, because:

1. it adds audio latency of up to PJMEDIA_SOUND_BUFFER_COUNT frames, while theoretically the delay should be limited to the maximum burst level of the device.
2. it will add even more end to end latency when the sound device is connected directly to the stream (rather than via conference bridge), because of how the jitter buffer is implemented.
3. pjmedia components need to handle the burst, while this problem should have been localized and fixed in sound device.

This ticket attempts to fix this problem by fixing the burst in sound device port. The idea is sound device port will apply a fixed delay to buffer the burst, like a fixed jitter buffer (and the delay will be detected at run-time) and will call the get_frame() and put_frame() callback in a perfect interleave. Then the rest of PJMEDIA components can just use 1 frame for buffering.

Audio drifts:

Local audio device drifts would cause clicking noise as underflow and overflow occurs.

[bennylp]

In r1664:

• Added delay buffer implementation in pjmedia. It's disabled by default for now.

[bennylp]

More work in r1715:

Changed the way sound buffering works when PJMEDIA_SOUND_USE_DELAYBUF is enabled (note: default is still disabled for now). When it is enabled, the PJMEDIA_SOUND_BUFFER_COUNT macro now will be used as the maximum delay to be accommodated by the delay buffer, and PJMEDIA components that previously use PJMEDIA_SOUND_BUFFER_COUNT value (such as conference bridge, splitcomb, and echo canceller) will set its buffer count to 1.

In the delay buffer itself, there are two problems:

• if we set LEARN_COUNT and WAITING_COUNT too low (for example, 8 and 8 in previous version), the delay buffer could not find the right value for the level. For example, in my system it set the level to 5, while the stable level value should be 6. Because it sets the level lower than the stable level, overflows occur.
• Setting LEARN_COUNT and WAITING_COUNT to high numbers (for example 4 and 48) solves above problem, but it will make learning time too long, and this is unacceptable since the delay buffer drops audio frames during learning time.

To solve the problems above, the delay buffer now adds 2 frames to the detected level, compared to just 1 frame in previous version. This setting is set as SAFE_MARGIN macro in delaybuf.c. With adding 2 frames it seems that now it can find the correct level without having to set long learning time period. So the default setting is now:

#define WAITING_COUNT  4
#define LEARN_COUNT    16
#define SAFE_MARGIN    2

[nanang]

Attachment ticket438.patch:

• added WSOLA to delaybuf, this is a great feature of time domain samples modification to compensate clock rate drift between PUT & GET.
• replaced snd_buf mechanism to delaybuf in conference, to manage passive ports buffer, so now conference should be driftproof.
• updated modules that use delaybuf, because of a small API changing of the delaybuf.
• added small part of drift case test on sound device port.

[nanang]

Attachment ticket438.2.patch:

• as attachment ticket438.patch.
• added new delaybuf API pjmedia_delay_buf_reset.
• modified conference to reset delaybuf when passive ports have no listener.
• modified delaybuf, in learning mode it will perform normal operation (instead of ignore frame on put & returning zero frame on get), after learning finished, the max delay will be set and buffer content will be adjusted appropriately.

[bennylp]

Patch ticket438.2.patch committed in r1833 with some modifications:

• added mutex
• changed some logging
• added options argument in delaybuf_create()