Changes between Version 4 and Version 5 of JBuf_Spec

Timestamp:

Feb 10, 2023 10:38:13 AM (15 months ago)

Author:

nanang

Comment:

--

JBuf_Spec

@@ -22 +22 @@
 === Adaptive to jitter change ===
 
-The jitter buffer MUST be able to adapt to change in network jitter, increasing or decreasing the prefetch value as necessary.
+The jitter buffer MUST be able to adapt to change in network jitter, increasing or decreasing the prefetch value and the buffering latency as necessary. Jitter may also be introduced by sound device. See '''Low latency''' section below for more info about the adaptivity.
 
 === Low latency ===
 
 The jitter buffer MUST attempt to provide the minimum buffering possible on the incoming packet (without sacrificing the continuous flow requirement above), to minimize latency.
+
+==== Burst level ====
+
+In the jitter buffer operation, packets are added-to and removed-from jitter buffer continuously. Ideally the adding & removing operations are done alternatingly one after another, this is when burst level is 1.
+
+However, at least from our experience, this is rarely the case. Network jitter happens and it may change from time to time. We have also seen jitter in sound device clock, permanently or temporary (e.g: due to CPU load spike). The combined jitters, i.e: from network (packet producer) and from sound-device (packet consumer), will cause add/remove operation bursts. The burst level represents the number of consecutive add/remove operations in a row, for example, burst level 3 means the jitter buffer will tend to see three {{{add}}} operations in a row before seeing {{{remove}}} operation and vice versa. We consider the burst level is very important for optimum latency calculation, i.e: the latency should not be shorter than the burst level as otherwise the sound-device/consumer may suffer from starvation (no packet available in the jitter buffer when needed).
+
+As the network condition may change continuously and so does the sound device clock, the burst level needs to be monitored continuously. As mentioned before that the burst level is very important for latency calculation, we need to be careful in changing the burst level, which eventually will cause jitter buffer to adjust the latency. To avoid starvation, increasing burst level should be done instantly and decreasing it should be done slowly.
+
+==== Progressive discard ====
+
+We consider that the optimum latency at any time should be about the burst level, for example when burst level is 3, the latency should be about 60ms (for audio frame length 20ms). So when the latency (or the number of frames buffered by jitter buffer) is longer than the burst level, we should discard some frames to reach the optimum latency. The progressive discard will drop frames at various rates depending on the difference between the actual latency and the optimum/target latency. There are some configurable macro settings that affects the discard rate: [https://docs.pjsip.org/en/latest/api/generated/pjmedia/group/group__PJMEDIA__CONFIG.html#c.PJMEDIA_JBUF_PRO_DISC_MIN_BURST PJMEDIA_JBUF_PRO_DISC_MIN_BURST & PJMEDIA_JBUF_PRO_DISC_MAX_BURST], [https://docs.pjsip.org/en/latest/api/generated/pjmedia/group/group__PJMEDIA__CONFIG.html#c.PJMEDIA_JBUF_PRO_DISC_T1 PJMEDIA_JBUF_PRO_DISC_T1 & PJMEDIA_JBUF_PRO_DISC_T2], [https://docs.pjsip.org/en/latest/api/generated/pjmedia/group/group__PJMEDIA__CONFIG.html#c.PJMEDIA_JBUF_DISC_MIN_GAP PJMEDIA_JBUF_DISC_MIN_GAP].
+
+For example, when the optimum/target latency is 3 frames (or 60ms) and current latency is 10 frames, the jitter buffer will schedule to discard a frame with calculation as follow:
+ - {{{Overflow}}} or difference between actual & target latencies = 10 - 3 frames = 7 frames.
+ - Use the following formula for calculating the target time for adjusting the latency (i.e: by discarding the overflow of 7 frames above):
+   {{{
+        T = PJMEDIA_JBUF_PRO_DISC_T1 +
+            (PJMEDIA_JBUF_PRO_DISC_T2 - PJMEDIA_JBUF_PRO_DISC_T1) *
+            (burst_level - PJMEDIA_JBUF_PRO_DISC_MIN_BURST) /
+            (PJMEDIA_JBUF_PRO_DISC_MAX_BURST-PJMEDIA_JBUF_PRO_DISC_MIN_BURST);
+
+        /*
+           Default settings:
+           PJMEDIA_JBUF_PRO_DISC_T1 = 2000ms
+           PJMEDIA_JBUF_PRO_DISC_T2 = 10000ms
+           PJMEDIA_JBUF_PRO_DISC_MIN_BURST = 1
+           PJMEDIA_JBUF_PRO_DISC_MIN_BURST = 100
+         */
+   }}}
+   At this point, the target time is 2000 + (8000 * 3/99) = 2242ms or discard rate is {{{target-time / overflow}}} = 2242 / 7 = 320 ms per frame. So the jitter buffer will discard a frame with timestamp 320ms (or frame to be played 320ms later).
+   Few notes:
+   - If the frame with that timestamp is not available in the jitter buffer yet, the calculation will be done again later. If the burst level is changed when the calculation is redone, the frame to discard may be changed too (no longer frame with timestamp 320ms).
+   - If the scheduled frame timestamp is lower than {{{PJMEDIA_JBUF_DISC_MIN_GAP}}} (i.e: 200ms), the jitter buffer will use {{{PJMEDIA_JBUF_DISC_MIN_GAP}}} instead, so the discard rate will not be faster than be faster than {{{PJMEDIA_JBUF_DISC_MIN_GAP}}}.
+
+==== Static discard ====
+
+The jitter buffer also provides a more conservative discard algoritm. With this algo, the optimum latency at any time should be twice the burst level (as comparison, the progressive discard algo above assumes optimum latency should be about equal to the burst level). This algorithm discard rate is fixed to {{{PJMEDIA_JBUF_DISC_MIN_GAP}}}, so it will discard a frame every 200ms (the default value) until the target/optimum latency is reached.
+
 
 === Duplicate/old frame ===