Missing video keyframe

[nanang]

Missing keyframe is a major and common issue in real-time video session. Packet lost or discarding packets by jitter buffer (opening video devs may take few seconds) in the beginning of video session could be the main cause.

We experienced this problem a lot in sipit! When using H.264, missing keyframe causes pjsua got no video (decoding failure), sometime video could be successfully decoded after call hold and resume. When using H.263, initial video came foggy (only the moving object part had clear view, background/other part was gray).

Solutions/ideas:

1. Prevention:
   • for receiving direction:
     • prevent jitter buffer to discard first packet and following packets with same timestamp,
     • and/or, as long as there is space in jitter buffer, avoid dropping any video packets, just adjust the playback time (make it faster/slower), if possible.
   • for sending direction: force sending keyframes in some short periods in the beginning of video session for few seconds.
2. Implementing media feedback/control mechanisms (for both directions):
   • Full Intra Request in ​RFC 5104 3.5.1
   • Picture Loss Indication feedback in ​RFC 4585 6.3.1
   • Fast Update Command for the Full Picture in ​RFC 5168 7.1. Note that some implementations refuse to send us SIP INFO request because we don't list "INFO" in our SIP header "Allow".

Note:

• Missing keyframe detection has been implemented in this ticket along with SIP INFO mechanism.
• Check #1437 for RTCP-FB PLI mechanism.

[nanang]

(In [3901]) Re #1234: Initial version of keyframe request/response via SIP INFO.

[nanang]

(In [3904]) Re #1234: minor update, put libavcodec version check for constant AV_PICTURE_TYPE_I/FF_I_TYPE usage.

[bennylp]

(In [3920]) Re #1234: increase default interval between two INFOs from 500 to 3000 ms