Opened 17 years ago
Last modified 17 years ago
#438 closed enhancement
Workaround for frame bursts from audio devices — at Version 3
Reported by: | bennylp | Owned by: | bennylp |
---|---|---|---|
Priority: | critical | Milestone: | release-0.9.0 |
Component: | pjmedia | Version: | trunk |
Keywords: | Cc: | nanang | |
Backport to 1.x milestone: | Backported: |
Description (last modified by bennylp)
Most audio devices will call record and playback callbacks in burst, rather than in perfect interleave (for example, burst of record callbacks, followed by burst of playback callbacks, and so on). Because of this, audio buffering needs to be implemented in few pjmedia components (conference bridge, splitcomb, echo canceller, etc.), and the maximum buffering is controlled by PJMEDIA_SOUND_BUFFER_COUNT macro.
This is not ideal, because:
- it adds audio latency of up to PJMEDIA_SOUND_BUFFER_COUNT frames, while theoretically the delay should be limited to the maximum burst level of the device.
- it will add even more end to end latency when the sound device is connected directly to the stream (rather than via conference bridge), because of how the jitter buffer is implemented.
- pjmedia components need to handle the burst, while this problem should have been localized and fixed in sound device.
This ticket attempts to fix this problem by fixing the burst in sound device port. The idea is sound device port will apply a fixed delay to buffer the burst, like a fixed jitter buffer (and the delay will be detected at run-time) and will call the get_frame() and put_frame() callback in a perfect interleave. Then the rest of PJMEDIA components can just use 1 frame for buffering.
Change History (3)
comment:1 Changed 17 years ago by bennylp
- Description modified (diff)
comment:2 Changed 17 years ago by bennylp
- Cc nanang added
More work in r1715:
Changed the way sound buffering works when PJMEDIA_SOUND_USE_DELAYBUF is enabled (note: default is still disabled for now). When it is enabled, the PJMEDIA_SOUND_BUFFER_COUNT macro now will be used as the maximum delay to be accommodated by the delay buffer, and PJMEDIA components that previously use PJMEDIA_SOUND_BUFFER_COUNT value (such as conference bridge, splitcomb, and echo canceller) will set its buffer count to 1.
In the delay buffer itself, there are two problems:
- if we set LEARN_COUNT and WAITING_COUNT too low (for example, 8 and 8 in previous version), the delay buffer could not find the right value for the level. For example, in my system it set the level to 5, while the stable level value should be 6. Because it sets the level lower than the stable level, overflows occur.
- Setting LEARN_COUNT and WAITING_COUNT to high numbers (for example 4 and 48) solves above problem, but it will make learning time too long, and this is unacceptable since the delay buffer drops audio frames during learning time.
To solve the problems above, the delay buffer now adds 2 frames to the detected level, compared to just 1 frame in previous version. This setting is set as SAFE_MARGIN macro in delaybuf.c. With adding 2 frames it seems that now it can find the correct level without having to set long learning time period. So the default setting is now:
#define WAITING_COUNT 4 #define LEARN_COUNT 16 #define SAFE_MARGIN 2
comment:3 Changed 17 years ago by bennylp
- Description modified (diff)
- Status changed from new to assigned
In r1664: