Caps negotiation analysis with GstTracer


GstTracer is an yet to be merged (post 1.6) new feature to GStreamer core to expose events happening in the pipeline to tracer plugins. Examples of such events are:

  • Events/buffers/queries being pushed on pads
  • Elements/pads being created
  • Messages posted on the bus

The notification for those events happens live during the pipeline execution and the plugins can react to it. GstTracer discussion is happening at bugzilla and the latest version, while not merged, can be found here.

stats, for example, is a tracer plugin that logs to GST_DEBUG all those notifications using a structured output. This output can be very useful to do post-analysis of a pipeline execution.

Analyzing caps negotiation

By using the stats output it was possible to analyze the caps related queries performed in the pipeline and organize this information for 2 purposes:

  • Create caps query calls tree: the sequence of caps and accept-caps queries can be put up together in a tree of calls and it is possible to check how a caps query travels and transforms itself from element to element in the pipeline.
  • Count the number of repeated caps queries: Queries have a filter caps and a result

AdaptiveDemux merged (finally)

The adaptivedemux base class has just been merged to gst-plugins-bad package and, with it, the first port of our 3 adaptive demuxer elements has also landed. Dashdemux is now only a thin layer of code that handles the specifics of parsing and keeping an MPD representation while the base class is the real adaptive client that does the fragment download and pushing among other generic adaptive features. (More on the base class can be found here:

Various playback tests have been done this month but some corner case may have escaped. So, if you test the new dashdemux (please do it) and happen to find a regression or any other issue report them at Bugs reported for DASH will have a very high priority on my TODO list for the next weeks. There are already some bugs open with patches for the old dashdemux version and I'm going to go over them and check if the patch can be ported to the adaptivedemux baseclass so those get fixed for all 3 formats for once (base classes are great!)

As for the other 2


Adaptive Demuxer baseclass

If you haven't read yet, Sebastian has a good overview of adaptive streaming support (client-side) in GStreamer:

Currently, GStreamer works with all 3 adaptive formats out there: HLS with hlsdemux, SmoothStreaming with mssdemux and DASH with dashdemux. And while it all works quite well for the most common scenarios, all 3 elements are very similar and share a lot of code. Large portions of code were actually copy and pasted from one to another while they were being developed and in early stages of stabilization. At the moment it is common to have to fix the same issue or implement the same feature and copy over for the other elements. This is not nice.

The Solution

As most of the code is actually the same, the obvious solution is to write a base class: GstAdaptiveDemux. It will handle the common logic that is now copied on all 3 elements:

  • Receive the manifest from upstream and merge it into a single buffer
  • Start a thread for each stream available in the manifest and create the source element that will fetch the fragments and push downstream
  • Calculate the download rate of fragments to select the