GStreamer Basic Tutorial 6: Media formats and pad's apabilities

target

Pad's apabilities are the basis of a GStreamer element ation, and since the framework is mostly automated, we hardly feel it.This tutorial compares the principles of bias and introduces:

  • What are Pad Capabilities
  • How to get this
  • When should I get this?
  • Why do you need to know them

introduce

Pads

Pads allow information to enter or leave an element as it has been shown before.This Capabilities (or simply Caps) specifies which information can be transmitted through the Pad.For example: "" RGB video, size 320x200 and 30 frames per second "or" 16 bits of audio sampling, 5.1 channels, 44.1k per second "can even be compressed like mp3/h264.

Pads supports multiple Capabilities (for example, a sink for a video can support RGB or YUV output), Capabilities can specify a range instead of a specific value (for example, an audio sink can support sampling rates from 1 to 48000).However, when data flows from one pad to another, it must be in a format that both sides can support.One form of data is supported by both pads so that Pads'apabilities are fixed (only one minute and no longer a data interval), a process known as negotiation.The following example illustrates this more clearly

In order for the two elements to be connected, they must have a common subset of Capabilities (otherwise they must not be connected to each other).This is the main purpose of Capabilities.As an application developer, we usually build pipeline s by connecting elements.In this example, you need to know the Caps of the Pad you are using elementation.

Pad Template

Pad is created by the Pad template, which lists all possible Capabilities for a Pad.Templates are useful for creating several similar Pads, but it is also early to determine whether two element ations can be connected: if even two Pad's templates do not have a common subset, there is no need for deeper negotiation.

Pad template checking is the first step in the negotiation process.As the process progresses, Pads formally initialize and determine their Capability (unless the negotiation fails).

Capabilities example

SINK template: 'sink'
  Availability: Always
  Capabilities:
    audio/x-raw-int
               signed: true
                width: 16
                depth: 16
                 rate: [ 1, 2147483647 ]
             channels: [ 1, 2 ]
    audio/x-raw-int
               signed: false
                width: 8
                depth: 8
                 rate: [ 1, 2147483647 ]
             channels: [ 1, 2 ]

This is a permanent sink pad for element ation.It supports two media formats, the original audio data (audio/x-raw-int), the number of 16-bit symbols and the number of 8-bit unsigned.Square brackets denote a range, for example, channels that range from 1 to 2.

SRC template: 'src'
  Availability: Always
  Capabilities:
    video/x-raw-yuv
                width: [ 1, 2147483647 ]
               height: [ 1, 2147483647 ]
            framerate: [ 0/1, 2147483647/1 ]
               format: { I420, NV12, NV21, YV12, YUY2, Y42B, Y444, YUV9, YVU9, Y41B, Y800, Y8  , GREY, Y16 , UYVY, YVYU, IYU1, v308, AYUV, A420 } 

video/x-raw-yur indicates that this source pad outputs video in YUV format.It supports a wide range of dimensions and frame rates, a series of YUV formats (listed in curly brackets).All these formats show different levels of image packaging and subsampling.

Final summary

You can use the gst-inspect-0.10 tool (described later in the tutorial) to look at Caps for element ation.

Keep in mind that some element ations look at the underlying hardware to determine the supported format and Caps of the Pad that can be provided (usually in READY or later).Therefore, the Caps shown here are different due to platform differences, and may even be different every time they run (which is rarely the case).

This tutorial creates two element s, displays their respective Pad templates, connects them, and sets the pipeline to a PLAY state.During each state switching process, the apabilities of Sink Elementment's Pad are displayed so that you can see how the negotiation process proceeds and how Pad's Caps are finalized.

A simple example of Pad Capabilities

#include <gst/gst.h>
  
/* Functions below print the Capabilities in a human-friendly format */
static gboolean print_field (GQuark field, const GValue * value, gpointer pfx) {
  gchar *str = gst_value_serialize (value);
  
  g_print ("%s  %15s: %s\n", (gchar *) pfx, g_quark_to_string (field), str);
  g_free (str);
  return TRUE;
}
  
static void print_caps (const GstCaps * caps, const gchar * pfx) {
  guint i;
  
  g_return_if_fail (caps != NULL);
  
  if (gst_caps_is_any (caps)) {
    g_print ("%sANY\n", pfx);
    return;
  }
  if (gst_caps_is_empty (caps)) {
    g_print ("%sEMPTY\n", pfx);
    return;
  }
  
  for (i = 0; i < gst_caps_get_size (caps); i++) {
    GstStructure *structure = gst_caps_get_structure (caps, i);
    
    g_print ("%s%s\n", pfx, gst_structure_get_name (structure));
    gst_structure_foreach (structure, print_field, (gpointer) pfx);
  }
}
  
/* Prints information about a Pad Template, including its Capabilities */
static void print_pad_templates_information (GstElementFactory * factory) {
  const GList *pads;
  GstStaticPadTemplate *padtemplate;
  
  g_print ("Pad Templates for %s:\n", gst_element_factory_get_longname (factory));
  if (!factory->numpadtemplates) {
    g_print ("  none\n");
    return;
  }
  
  pads = factory->staticpadtemplates;
  while (pads) {
    padtemplate = (GstStaticPadTemplate *) (pads->data);
    pads = g_list_next (pads);
    
    if (padtemplate->direction == GST_PAD_SRC)
      g_print ("  SRC template: '%s'\n", padtemplate->name_template);
    else if (padtemplate->direction == GST_PAD_SINK)
      g_print ("  SINK template: '%s'\n", padtemplate->name_template);
    else
      g_print ("  UNKNOWN!!! template: '%s'\n", padtemplate->name_template);
    
    if (padtemplate->presence == GST_PAD_ALWAYS)
      g_print ("    Availability: Always\n");
    else if (padtemplate->presence == GST_PAD_SOMETIMES)
      g_print ("    Availability: Sometimes\n");
    else if (padtemplate->presence == GST_PAD_REQUEST) {
      g_print ("    Availability: On request\n");
    } else
      g_print ("    Availability: UNKNOWN!!!\n");
    
    if (padtemplate->static_caps.string) {
      g_print ("    Capabilities:\n");
      print_caps (gst_static_caps_get (&padtemplate->static_caps), "      ");
    }
    
    g_print ("\n");
  }
}
  
/* Shows the CURRENT capabilities of the requested pad in the given element */
static void print_pad_capabilities (GstElement *element, gchar *pad_name) {
  GstPad *pad = NULL;
  GstCaps *caps = NULL;
  
  /* Retrieve pad */
  pad = gst_element_get_static_pad (element, pad_name);
  if (!pad) {
    g_printerr ("Could not retrieve pad '%s'\n", pad_name);
    return;
  }
  
  /* Retrieve negotiated caps (or acceptable caps if negotiation is not finished yet) */
  caps = gst_pad_get_negotiated_caps (pad);
  if (!caps)
    caps = gst_pad_get_caps_reffed (pad);
  
  /* Print and free */
  g_print ("Caps for the %s pad:\n", pad_name);
  print_caps (caps, "      ");
  gst_caps_unref (caps);
  gst_object_unref (pad);
}
  
int main(int argc, char *argv[]) {
  GstElement *pipeline, *source, *sink;
  GstElementFactory *source_factory, *sink_factory;
  GstBus *bus;
  GstMessage *msg;
  GstStateChangeReturn ret;
  gboolean terminate = FALSE;
  
  /* Initialize GStreamer */
  gst_init (&argc, &argv);
   
  /* Create the element factories */
  source_factory = gst_element_factory_find ("audiotestsrc");
  sink_factory = gst_element_factory_find ("autoaudiosink");
  if (!source_factory || !sink_factory) {
    g_printerr ("Not all element factories could be created.\n");
    return -1;
  }
  
  /* Print information about the pad templates of these factories */
  print_pad_templates_information (source_factory);
  print_pad_templates_information (sink_factory);
  
  /* Ask the factories to instantiate actual elements */
  source = gst_element_factory_create (source_factory, "source");
  sink = gst_element_factory_create (sink_factory, "sink");
  
  /* Create the empty pipeline */
  pipeline = gst_pipeline_new ("test-pipeline");
  
  if (!pipeline || !source || !sink) {
    g_printerr ("Not all elements could be created.\n");
    return -1;
  }
  
  /* Build the pipeline */
  gst_bin_add_many (GST_BIN (pipeline), source, sink, NULL);
  if (gst_element_link (source, sink) != TRUE) {
    g_printerr ("Elements could not be linked.\n");
    gst_object_unref (pipeline);
    return -1;
  }
  
  /* Print initial negotiated caps (in NULL state) */
  g_print ("In NULL state:\n");
  print_pad_capabilities (sink, "sink");
  
  /* Start playing */
  ret = gst_element_set_state (pipeline, GST_STATE_PLAYING);
  if (ret == GST_STATE_CHANGE_FAILURE) {
    g_printerr ("Unable to set the pipeline to the playing state (check the bus for error messages).\n");
  }
  
  /* Wait until error, EOS or State Change */
  bus = gst_element_get_bus (pipeline);
  do {
    msg = gst_bus_timed_pop_filtered (bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_ERROR | GST_MESSAGE_EOS |
        GST_MESSAGE_STATE_CHANGED);
  
    /* Parse message */
    if (msg != NULL) {
      GError *err;
      gchar *debug_info;
    
      switch (GST_MESSAGE_TYPE (msg)) {
        case GST_MESSAGE_ERROR:
          gst_message_parse_error (msg, &err, &debug_info);
          g_printerr ("Error received from element %s: %s\n", GST_OBJECT_NAME (msg->src), err->message);
          g_printerr ("Debugging information: %s\n", debug_info ? debug_info : "none");
          g_clear_error (&err);
          g_free (debug_info);
          terminate = TRUE;
          break;
        case GST_MESSAGE_EOS:
          g_print ("End-Of-Stream reached.\n");
          terminate = TRUE;
          break;
        case GST_MESSAGE_STATE_CHANGED:
          /* We are only interested in state-changed messages from the pipeline */
          if (GST_MESSAGE_SRC (msg) == GST_OBJECT (pipeline)) {
            GstState old_state, new_state, pending_state;
            gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
            g_print ("\nPipeline state changed from %s to %s:\n",
                gst_element_state_get_name (old_state), gst_element_state_get_name (new_state));
            /* Print the current capabilities of the sink element */
            print_pad_capabilities (sink, "sink");
          }
          break;
        default:
          /* We should not reach here because we only asked for ERRORs, EOS and STATE_CHANGED */
          g_printerr ("Unexpected message received.\n");
          break;
      }
      gst_message_unref (msg);
    }
  } while (!terminate);
  
  /* Free resources */
  gst_object_unref (bus);
  gst_element_set_state (pipeline, GST_STATE_NULL);
  gst_object_unref (pipeline);
  gst_object_unref (source_factory);
  gst_object_unref (sink_factory);
  return 0;
}

 

Workflow

Here print_field, print_caps, and print_pad_templates simply display the Capabilities structure in a readable way.If you want to learn about the intrinsic organization of GstCaps data structures, read the section on Pad Caps in the Gstreamer documentation below.

/* Shows the CURRENT capabilities of the requested pad in the given element */
static void print_pad_capabilities (GstElement *element, gchar *pad_name) {
  GstPad *pad = NULL;
  GstCaps *caps = NULL;
  
  /* Retrieve pad */
  pad = gst_element_get_static_pad (element, pad_name);
  if (!pad) {
    g_printerr ("Could not retrieve pad '%s'\n", pad_name);
    return;
  }
  
  /* Retrieve negotiated caps (or acceptable caps if negotiation is not finished yet) */
  caps = gst_pad_get_negotiated_caps (pad);
  if (!caps)
    caps = gst_pad_get_caps_reffed (pad);
  
  /* Print and free */
  g_print ("Caps for the %s pad:\n", pad_name);
  print_caps (caps, "      ");
  gst_caps_unref (caps);
  gst_object_unref (pad);
}

The gst_element_get_static_pad() method takes the name of the Pad from a given element.This Pad is static because it will always exist.To learn more about Pad, read the section on Pad in the GStreamer documentation.

We then invoke the gst_pad_get_negotiated_caps() method to obtain Pad's current Capabilities, depending on the status of the negotiation process, whether fixed or not.Capabilities may not exist at this point, and if we do, we call the gst_pad_get_caps_reffed() method to create a currently acceptable Pad Capabilities.This so-called currently acceptable Caps is Pad Template's Caps in NULL state, but this may change later because the actual hardware is also queried.

gst_pad_get_caps_reffed() is generally faster than gst_pad_get_caps(), and is sufficient if we don't need to change the Caps we get.And then we'll print out these Capabilities.

  /* Create the element factories */
  source_factory = gst_element_factory_find ("audiotestsrc");
  sink_factory = gst_element_factory_find ("autoaudiosink");
  if (!source_factory || !sink_factory) {
    g_printerr ("Not all element factories could be created.\n");
    return -1;
  }
  
  /* Print information about the pad templates of these factories */
  print_pad_templates_information (source_factory);
  print_pad_templates_information (sink_factory);
  
  /* Ask the factories to instantiate actual elements */
  source = gst_element_factory_create (source_factory, "source");
  sink = gst_element_factory_create (sink_factory, "sink");

In the previous tutorial, we used the gst_element_factory_make() method directly to create the element, ignoring the factory itself, and we'll add this section now.A GstElementFactory is responsible for instantiating an element through a factory name.

You can use gst_element_factory_find() to create a "videotestsrc" factory and instantiate multiple "videotestsrc" elements by gst_element_factory_create().gst_element_factory_make() is actually just an encapsulation of this process.

Through the project, Pad templates are virtually accessible, so we print this information as soon as the factory is set up.

We skip the pipeline creation and startup section and jump directly to the processing of state switch messages:

        case GST_MESSAGE_STATE_CHANGED:
          /* We are only interested in state-changed messages from the pipeline */
          if (GST_MESSAGE_SRC (msg) == GST_OBJECT (pipeline)) {
            GstState old_state, new_state, pending_state;
            gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
            g_print ("\nPipeline state changed from %s to %s:\n",
                gst_element_state_get_name (old_state), gst_element_state_get_name (new_state));
            /* Print the current capabilities of the sink element */
            print_pad_capabilities (sink, "sink");
          }
          break;

Here's just a simple print of the current Pad Caps each time the pipeline state switches.You can see how, in the output information, the initial Caps (Caps for Pad templates) change gradually until they are determined (contain only one type and have a fixed value).

conclusion

This tutorial shows:

  • What are Fill Function and Fill Template Function.

  • How to use gst_pad_get_current_caps() or retrieve them gst_pad_query_caps().

  • Depending on the state of the pipeline, they have different meanings (initially, they indicate all possible functions, then they indicate the currently negotiated upper limit for the ad).

  • pad is important for knowing in advance whether two elements can be linked together.

  • have access to Basic Tutorial 10: In the GStreamer tool Tools described in gst-inspect-1.0 to find Pad Caps.

The next tutorial shows how to manually inject and extract data from the GStreamer pipeline.

Reference resources: https://blog.csdn.net/sakulafly/article/details/21299519

166 original articles published, 495 awarded, 970,000 visits+
His message board follow

Posted on Wed, 11 Mar 2020 20:37:33 -0700 by Yossarian