Opened 13 years ago

Last modified 13 years ago

#1182 closed enhancement

vstream (video stream) implementation and sample — at Version 2

Reported by: bennylp Owned by: nanang
Priority: normal Milestone: release-2.0-dev
Component: pjmedia Version: 2.0-dev-branch
Keywords: Cc:
Backport to 1.x milestone: Backported:

Description (last modified by nanang)

  • Either extend existing stream.c to support video, or create a new vstream.c.
  • Use vstreamutil.c as demo.

There is already video stream implementation in the stream.c and sample in the vstreamutil.c, however it's just proof-of-concept version, so the to do/consider list will be:

  • proper picture bitstream generation from RTP packet, currently the video stream will just 'concatenate' incoming RTP to get a complete picture bitstream, and everytime it got end-of-picture flag, RTP header m==1, push the picture into jitter buffer, so in case of unordered RTP the picture bitstreams will be truncated
  • perhaps jitter buffer frame content/granularity should be RTP payload instead of a picture bitstream (picture bitstream will be generated right before decoding, instead of inline with incoming RTP packets), however current jitter buffer only has RTP sequence (no RTP timestamp), so need to research if the current jitter buffer fits this requirements along with optimization consideration (e.g: frame size, minimal memcpy)
  • audio-video sync
  • encoder & decoder size can be different, e.g: remote endpoint, which is an embedded/mobile, only supports QCIF while local endpoint, which is desktop, prefers CIF, so the issues will be:
    • which size should be put in the stream info (as currently it only has one 'format info' field)?
    • the decoder output format (e.g: format id & size) can only be ensured after a successful picture decoding (the format setting in opening the decoder is rather unusable and the local SDP may offer multiple sizes), need a way to notify application about the active format.

Change History (2)

comment:1 Changed 13 years ago by bennylp

  • Description modified (diff)

comment:2 Changed 13 years ago by nanang

  • Description modified (diff)
Note: See TracTickets for help on using tickets.