Chapter 18. Pipeline manipulation

This chapter will discuss how you can manipulate your pipeline in several ways from your application on. Parts of this chapter are downright hackish, so be assured that you'll need some programming knowledge before you start reading this.

Topics that will be discussed here include how you can insert data into a pipeline from your application, how to read data from a pipeline, how to manipulate the pipeline's speed, length, starting point and how to listen to a pipeline's data processing.

18.1. Data probing

Probing is best envisioned as a pad listener. Technically, a probe is nothing more than a signal callback that can be attached to a pad. Those signals are by default not fired at all (since that may have a negative impact on performance), but can be enabled by attaching a probe using gst_pad_add_buffer_probe (), gst_pad_add_event_probe (), or gst_pad_add_data_probe (). Those functions attach the signal handler and enable the actual signal emission. Similarly, one can use the gst_pad_remove_buffer_probe (), gst_pad_remove_event_probe (), or gst_pad_remove_data_probe () to remove the signal handlers again.

Probes run in pipeline threading context, so callbacks should try to not block and generally not do any weird stuff, since this could have a negative impact on pipeline performance or, in case of bugs, cause deadlocks or crashes. More precisely, one should usually not call any GUI-related functions from within a probe callback, nor try to change the state of the pipeline. An application may post custom messages on the pipeline's bus though to communicate with the main application thread and have it do things like stop the pipeline.

In any case, most common buffer operations that elements can do in _chain () functions, can be done in probe callbacks as well. The example below gives a short impression on how to use them (even if this usage is not entirely correct, but more on that below):


#include <gst/gst.h>

static gboolean
cb_have_data (GstPad    *pad,
	      GstBuffer *buffer,
	      gpointer   u_data)
{
  gint x, y;
  guint16 *data = (guint16 *) GST_BUFFER_DATA (buffer), t;

  /* invert data */
  for (y = 0; y < 288; y++) {
    for (x = 0; x < 384 / 2; x++) {
      t = data[384 - 1 - x];
      data[384 - 1 - x] = data[x];
      data[x] = t;
    }
    data += 384;
  }

  return TRUE;
}

gint
main (gint   argc,
      gchar *argv[])
{
  GMainLoop *loop;
  GstElement *pipeline, *src, *sink, *filter, *csp;
  GstCaps *filtercaps;
  GstPad *pad;

  /* init GStreamer */
  gst_init (&argc, &argv);
  loop = g_main_loop_new (NULL, FALSE);

  /* build */
  pipeline = gst_pipeline_new ("my-pipeline");
  src = gst_element_factory_make ("videotestsrc", "src");
  if (src == NULL)
    g_error ("Could not create 'videotestsrc' element");

  filter = gst_element_factory_make ("capsfilter", "filter");
  g_assert (filter != NULL); /* should always exist */

  csp = gst_element_factory_make ("ffmpegcolorspace", "csp");
  if (csp == NULL)
    g_error ("Could not create 'ffmpegcolorspace' element");

  sink = gst_element_factory_make ("xvimagesink", "sink");
  if (sink == NULL) {
    sink = gst_element_factory_make ("ximagesink", "sink");
    if (sink == NULL)
      g_error ("Could not create neither 'xvimagesink' nor 'ximagesink' element");
  }

  gst_bin_add_many (GST_BIN (pipeline), src, filter, csp, sink, NULL);
  gst_element_link_many (src, filter, csp, sink, NULL);
  filtercaps = gst_caps_new_simple ("video/x-raw-rgb",
			   "width", G_TYPE_INT, 384,
			   "height", G_TYPE_INT, 288,
			   "framerate", GST_TYPE_FRACTION, 25, 1,
			   "bpp", G_TYPE_INT, 16,
			   "depth", G_TYPE_INT, 16,
			   "endianness", G_TYPE_INT, G_BYTE_ORDER,
			   NULL);
  g_object_set (G_OBJECT (filter), "caps", filtercaps, NULL);
  gst_caps_unref (filtercaps);

  pad = gst_element_get_pad (src, "src");
  gst_pad_add_buffer_probe (pad, G_CALLBACK (cb_have_data), NULL);
  gst_object_unref (pad);

  /* run */
  gst_element_set_state (pipeline, GST_STATE_PLAYING);

  /* wait until it's up and running or failed */
  if (gst_element_get_state (pipeline, NULL, NULL, -1) == GST_STATE_CHANGE_FAILURE) {
    g_error ("Failed to go into PLAYING state");
  }

  g_print ("Running ...\n");
  g_main_loop_run (loop);

  /* exit */
  gst_element_set_state (pipeline, GST_STATE_NULL);
  gst_object_unref (pipeline);

  return 0;
}
    

Compare that output with the output of "gst-launch-0.10 videotestsrc ! xvimagesink", just so you know what you're looking for.

The above example is not really correct though. Strictly speaking, a pad probe callback is only allowed to modify the buffer content if the buffer is writable, and it is only allowed to modify buffer metadata like timestamps, caps, etc. if the buffer metadata is writable. Whether this is the case or not depends a lot on the pipeline and the elements involved. Often enough, this is the case, but sometimes it is not, and if it is not then unexpected modification of the data or metadata can introduce bugs that are very hard to debug and track down. You can check if a buffer and its metadata are writable with gst_buffer_is_writable () and gst_buffer_is_metadata_writable (). Since you can't pass back a different buffer than the one passed in, there is no point of making a buffer writable in the callback function.

Pad probes are suited best for looking at data as it passes through the pipeline. If you need to modify data, you should write your own GStreamer element. Base classes like GstAudioFilter, GstVideoFilter or GstBaseTransform make this fairly easy.

If you just want to inspect buffers as they pass through the pipeline, you don't even need to set up pad probes. You could also just insert an identity element into the pipeline and connect to its "handoff" signal. The identity element also provides a few useful debugging tools like the "dump" property or the "last-message" property (the latter is enabled by passing the '-v' switch to gst-launch).