Video orientation support

[bennylp]

Support for encoding video orientation in SDP, stream, and (possibly) devices.

Background

• Mobile devices can be rotated (the rotation angle can be detected), which may lead to wrong video orientation in rendering/playback.

Objective

• The purpose is to render video in natural orientation (head/sky upside) or the preferred orientation of the capturer.

Fact

• Common video codecs works in predefined allowed sizes/orientation, e.g: CIF, QCIF.
• On iOS, capturing video to a file will store the video data in camera native orientation, i.e: wide-oriented, the capture orientation info (the device/camera orientation when the capture/record process is done) is stored in video metadata, so in playing back the file, the stored capture orientation info (may be combined with current device orientation info) is used to calculate rotation angle to get natural orientation in video rendering/playback.

Idea

• The orientation info is readable and read-only in capture.
• The renderer will apply the orientation info provided by the capturer.
• The orientation info is a rotation angle to be applied to a picture/video so it can be displayed naturally (or as the capturer preference).

Issue

• SDP/SIP doesn't seem to support this orientation thing. No way for sender to tell receiver about the video orientation info. Putting 'a=orient:x' in SDP is not really useful as it just receiver preference/capability, while in practice, the receiver can render any orientations and the capturer cannot really send portrait (codecs limitation, etc).