Working with Media

The Concept

Media objects are objects that are capable to process media. There are several type of media objects supported in pjsua:

• call, obviously, to transmit and receive media from remote person.
• WAV file player to play WAV file
• WAV file recorder to record audio to a WAV file
• WAV playlist, to playback multiple WAV files sequentially.

More media objects may be added in the future.

In pjsua Python module and in ​PJSUA API in general, all media objects are terminated in the central conference bridge so that they are easier to manipulate. When objects are plugged-in to the conference bridge, they will be given a slot number that identifies the objects in the bridge.

The conference bridge provides a simple but yet powerful API to manage audio routing between the audio objects. The principle is very simple, that is you connect audio source to audio destination, and the bridge will make the audio flows from the source to destination, and that's it. If more than one sources are transmitting to the same destination, then the audio will be mixed. If one source is transmitting to more than one destinations, the bridge will take care of duplicating the audio from the source to the multiple destinations.

Lets see a diagram of a conference bridge with several media objects.

The diagram above shows the conference bridge with the following objects:

• sound device, which by convention is always attached at slot number 0
• a WAV file to be played back at slot 1
• a WAV file for recording at slot 2
• an active call to Alice at slot 3
• an active call to Bob at slot 4

When a media object is plugged-in to the bridge, it will not be connected to anything, so media will not flow from/to the object. When no connections are made in the bridge (like the diagram above), then no media will be flowing in the bridge at all.

Sample: WAV File Playback

To playback the WAV file to the speaker, just connect the WAV playback object to the sound device, and that's it. Simple isn't it!

The diagram above shows the media flow from the WAV playback object to the sound device (see the red arrow).

Here's the code to connect the WAV playback object to the speaker:

  lib.conf_connect(1, 0)

The ​conf_connect() method takes two arguments:

• the first argument is the slot number of the source media object, in this case one (1) for the WAV file playback, and
• the second argument is the slot number of the destination media object, in this case zero for the sound device

Once you're done with the playback, just disconnect the connection to stop the playback:

  lib.conf_disconnect(1, 0)

Sample: Recording to WAV File

Or if you want to record the microphone to the WAV file, simply do this:

  lib.conf_connect(0, 2)

And the media will flow from the sound device to the WAV record file, as shown in the diagram below:

As usual, to stop or pause recording, just disconnect the connection:

  lib.conf_disconnect(0, 2)

(Note that disconnecting the WAV recorder as above does not close the WAV file, and you can resume recording by connecting a source to the WAV recorder again.)

Sample: Normal Call

For a normal call, we'd want to establish bidirectional audio with the remote person, which can be done easily by establishing bidirectional connection between the call (with Alice, in this case) and the sound device:

 lib.conf_connect(0, 3)
 lib.conf_connect(3, 0)

Of course on a real application, you would replace the number "3" above with the slot number of the call, which you can get with call.info().conf_slot.

The diagram below shows the interconnection in the bridge now:

Sample: Second Call

Suppose we want to talk with Alice and Bob at the same time. Since we already have bidirectional media connection with Alice, we just need to add the code below to establish bidirectional connection with Bob.

  lib.conf_connect(0, 4)
  lib.conf_connect(4, 0

Now we can talk to Alice and Bob at the same time, and we will hear audio from either party. The interconnection diagram in the bridge will be like this:

At this stage, we can talk to both Alice and Bob, but Alice and Bob can't talk or hear each other.

Sample: Conference Call

To make Alice and Bob talk to each other, just establish bidirectional media between them:

 lib.conf_connect(3, 4)
 lib.conf_connect(4, 3)

The snippet above will add the new connections as shown as lines with black color below:

Sample: Recording the Conference

While doing the conference, it perfectly makes sense to want to record the conference to a WAV file, and all we need to do is to connect the microphone and both calls to the WAV recorder:

 lib.conf_connect(0, 2)
 lib.conf_connect(3, 2)
 lib.conf_connect(4, 2)

The connections in the conference bridge now will look like the diagram below, with the snippet above adds the brown lines in the diagram: