Type of analytics meta data This meta represents a batch of buffers from one or more streams together with the relevant events to be able to interpret the buffers and to be able to reconstruct the original streams. When used for multiple streams and batching them temporarily, caps of type `multistream/x-analytics-batch(meta:GstAnalyticsBatchMeta)` should be used, with the original caps of each stream in an array-typed `streams` field. The original caps of each stream might be extended by additional fields and the order of the streams in the array corresponds to the order of the @streams array of the meta. In this case, empty buffers would be used without any #GstMemory and When used for a single stream, the original caps might be used together with the `meta:GstAnalyticsBatchMeta` caps feature and potentially extended by additional fields to describe the kind of batching and its configuration, e.g. that each batch is made of 25% overlapping 320x320 slices of the original video frame. The timestamp, duration and other metadata of each batch can be retrieved from the parent buffer of this meta. parent #GstAnalyticsBatchStream for this batch Number of streams Index of the stream in the meta's stream array The sticky events store before any of the mini objects in the @objects fields are processed Number of sticky events #GstMiniObject in this batch for this stream. Those are serialized mini objects: buffers, bufferlists and serialized events Number of objects Gets the #GstCaps from a stream The #GstCaps if there are any A #GstAnalyticsBatchStream Gets the #GstSegment from a stream The #GstSegment if there is one A #GstAnalyticsBatchStream Gets the current stream id from a stream The stream id if there is any A #GstAnalyticsBatchStream The caps feature to be used on streams that make use of this meta. Handle containing data required to use gst_analytics_cls_mtd APIs. This type is generally expected to be allocated on the stack. Instance identifier Instance of #GstAnalyticsRelationMeta where the analytics-metadata identified by @id is stored index of the class associated with @quarks ( and label) or a negative value on failure. Instance handle Quark of the class Get index of class represented by @quark Number of classes in this classification instance Instance handle Get number of classes Get confidence level for class at @index confidence level for @index, <0.0 if the call failed. instance handle Object class index Quark of this class (label) associated with @index Instance handle index of the class Get quark of the class at @index Get an id identifying #GstAnalyticsMtd type. opaque id of #GstAnalyticsMtd type The #GstAnalyticsModelInfo is an object storing artifical neural network model metadata describing the input and output tensors. These information's are required by inference elements. The ".modelinfo" files describe the additional metadata for a given serialized model file such as a `.tflite`, `.onnx` or `.pte` files. The ModelInfo files are ini-style. Each section is matched to a particular input or output tensor. The title of the section must match the name of the tensor in the model file. The fields used to match the modelinfo to the model are: `\[title\]`: The name of the tensor, must be unique `dims`: The dimensions as a comma-separated list of ints. -1 matches a dynamic dimension and is a wildcard `dir`: Either "input" or "output" `type`: The data type match #GstTensorDataType, one of: `int4` `int8` `int16` `int32` `int64` `uint4` `uint8` `uint16` `uint32` `uint64` `float16` `float32` `float64` `bfloat16` Based on these fields, the following metadata is applied to output tensors: `id`: The tensor ID so other elements can identity it, ideally registered in the [Tensor ID Registry](https://github.com/collabora/tensor-id-registry/blob/main/tensor-id-register.md). `group-id`: The group ID that groups related tensors together (e.g., all outputs from the same model) `dims-order`: The dimension ordering, either "row-major" or "col-major". Defaults to "row-major" if not specified. Those fields are applied to input tensors for normalization: `ranges`: semicolon-separated list of comma-separated pairs of floats, each representing (min, max) for a single channel or dimension. For per-channel normalization: `ranges=0.0,255.0;-1.0,1.0;0.0,1.0` (R,G,B) For single range (applies to all channels): `ranges=0.0,255.0` The inference elements will convert 8-bit input [0-255] to target ranges using: output[i] = input[i] * scale[i] + offset[i] where for each channel i: scale[i] = (max[i] - min[i]) / 255.0 offset[i] = min[i] Common ranges: `0.0,255.0` - No normalization (passthrough, scale=1.0, offset=0.0) `0.0,1.0` - Normalized to [0,1] range (scale≈0.00392, offset=0.0) `-1.0,1.0` - Normalized to [-1,1] range (scale≈0.00784, offset=-1.0) `16.0,235.0` - TV/limited range (scale≈0.859, offset=16.0) Other fields are ignored for now. The API is meant to be used by inference elements Passes to functions asking for a relation span when the span is infinite. The name of the modelinfo header section The current major version of the modelinfo format The current minor version of the modelinfo format The current version string for the modelinfo format. This MUST be updated whenever the format changes. A wildcard matching any type of analysis The #GstAnalyticsModelInfo is an object storing artifical neural network model metadata describing the input and output tensors. These information's are required by inference elements. Find the name of a tensor in the modelinfo that matches the given criteria. The function performs the following checks in order: 1. If @in_tensor_name is provided and exists in modelinfo, validate it matches 2. Search by index for the specified direction and validate 3. Search by dimensions and data type The tensor name if found, or %NULL otherwise. The caller must free this with g_free() when done. Instance of #GstAnalyticsModelInfo The tensor direction (input or output) The tensor index within the specified direction An optional tensor name hint to check first The tensor data type to match The number of dimensions The dimension sizes. Use -1 for dynamic dimensions. Free a modelinfo object allocated by gst_analytics_modelinfo_load(). This function should be called when the modelinfo is no longer needed to release the associated resources. Instance of #GstAnalyticsModelInfo Retrieve the dimension ordering for a given tensor. The dimension ordering specifies how multi-dimensional tensor data is laid out in memory: - Row-major (C/NumPy style): Last dimension changes fastest in memory - Column-major (Fortran style): First dimension changes fastest in memory If not specified in the modelinfo, defaults to row-major. The dimension order as #GstTensorDimOrder Instance of #GstAnalyticsModelInfo The name of the tensor Get the group ID that groups related tensors together (e.g., all outputs from the same model). The group ID is stored in the modelinfo section and is global for all tensors in the model. The group ID string, or %NULL if not found. The caller must free this with g_free() when done. Instance of #GstAnalyticsModelInfo Get the tensor ID from the modelinfo for the specified tensor name. The tensor ID is ideally registered in the [Tensor ID Registry](https://github.com/collabora/tensor-id-registry/blob/main/tensor-id-register.md). The tensor ID string, or %NULL if not found. The caller must free this with g_free() when done. Instance of #GstAnalyticsModelInfo The name of the tensor Calculate normalization scales and offsets to transform input data to the target range. This function calculates transformation parameters to convert from the actual input data range [input_min, input_max] to the target range expected by the model [target_min, target_max]: `normalized_value[i] = input[i] * output_scale[i] + output_offset[i]` The target ranges are read from the modelinfo `ranges` field: Semicolon-separated list of comma-separated pairs (min,max) for per-channel target ranges (e.g., "0.0,255.0;-1.0,1.0;0.0,1.0" for RGB channels with different target ranges). Common input ranges: - [0.0, 255.0]: 8-bit unsigned (uint8) - [-128.0, 127.0]: 8-bit signed (int8) - [0.0, 65535.0]: 16-bit unsigned (uint16) - [-32768.0, 32767.0]: 16-bit signed (int16) - [0.0, 1.0]: Normalized float - [-1.0, 1.0]: Normalized signed float The number of input ranges (@num_input_ranges) must equal the number of target ranges in the modelinfo. The function will return FALSE if they don't match. The caller must free @output_scales and @output_offsets with g_free() when done. %TRUE on success, %FALSE on error, if ranges field is not found, or if @num_input_ranges doesn't match the number of target ranges in the modelinfo Instance of #GstAnalyticsModelInfo The name of the tensor The number of input ranges (channels/dimensions) The minimum values of the actual input data for each channel The maximum values of the actual input data for each channel The number of output ranges/scale-offset pairs The scale values for normalization The offset values for normalization Get the group ID as a GQuark for efficient string comparison and storage. Using GQuark is more efficient than string comparison when you need to compare multiple group IDs. The GQuark of the group ID, or 0 if not found Instance of #GstAnalyticsModelInfo Get the tensor ID as a GQuark for efficient string comparison and storage. Using GQuark is more efficient than string comparison when you need to compare multiple IDs. The GQuark of the tensor ID, or 0 if not found Instance of #GstAnalyticsModelInfo The name of the tensor Retrieve all target ranges (min/max pairs) expected by the model for a given tensor. This function retrieves all target ranges from the `ranges` field in the modelinfo. Each range represents the expected input range for a channel or dimension that the model requires. The function reads from the `ranges` field: Semicolon-separated list of comma-separated pairs (min,max) for per-channel target ranges (e.g., "0.0,1.0;-1.0,1.0;0.0,1.0" for RGB channels with different normalization targets). The caller must free @mins and @maxs with g_free() when done. %TRUE if range information was found and valid, %FALSE otherwise Instance of #GstAnalyticsModelInfo The name of the tensor The number of ranges The minimum values for each target range The maximum values for each target range Retrieve the version string of the modelinfo file format. The version is in the format "Major.Minor" and is stored in the [modelinfo] section of the modelinfo file. The version string (e.g., "1.0"). The caller must free this with g_free() when done. Defaults to "1.0" if not specified. Instance of #GstAnalyticsModelInfo Load a modelinfo file associated with the given model file. This function attempts to load a `.modelinfo` file in the following order: 1. `{model_filename}.modelinfo` 2. `{model_filename_without_extension}.modelinfo` The modelinfo file contains metadata for the model's input and output tensors, including normalization ranges, dimension ordering, tensor IDs, etc. The loaded modelinfo must be freed with gst_analytics_modelinfo_free() when no longer needed. A new #GstAnalyticsModelInfo instance, or %NULL if the modelinfo file could not be found or loaded. Path to the model file (e.g., "model.onnx", "model.tflite") Tensor location is unknown Input tensor Output tensor Handle containing data required to use gst_analytics_mtd API. This type is generally expected to be allocated on the stack. Instance identifier Instance of #GstAnalyticsRelationMeta where the analytics-metadata identified by @id is stored Get instance id Id of @instance Instance of #GstAnalyticsMtd opaque id of the type Instance of #GstAnalyticsMtd Get analysis result type. Get instance size Size (in bytes) of this instance or 0 on failure. Instance of #GstAnalyticsMtd Gets the string version of the name of this type of analytics data the name The type of analytics data This structure must be provided when registering a new type of Mtd. It must have a static lifetime (never be freed). The name of the metadata type A pointer to a function that will be called when the containing meta is transform to potentially copy the data into a new Mtd into the new meta. A pointer to a function that will be called when the containing meta is cleared to potetially do cleanup (ex. _unref or release) resources it was using. Handle containing data required to use gst_analytics_od_mtd APIs. This type is generally expected to be allocated on the stack. Instance identifier Instance of #GstAnalyticsRelationMeta where the analytics-metadata identified by @id is stored Retrieve location confidence level. TRUE on success, otherwise FALSE. instance Confidence on object location Retrieve location and location confidence level. TRUE on success, otherwise FALSE. instance x component of upper-left corner of the object location y component of upper-left corner of the object location bounding box width of the object location bounding box height of the object location Confidence on object location Quark of the class of object associated with this location. Quark different from on success and 0 on failure. Instance handle Retrieve oriented location and location confidence level. TRUE on success, otherwise FALSE. instance x component of upper-left corner of the object location (pre-rotation) y component of upper-left corner of the object location (pre-rotation) bounding box width of the object location bounding box height of the object location Rotation of the bounding box in radians <0, 2xPI> with respect to the bounding box center (the rotation value is a clock-wise angle) Confidence on object location Get an id that represent object-detection metadata type Opaque id of the #GstAnalyticsMtd type No relation First analysis-meta is part of second analysis-meta First analysis-meta contain second analysis-meta. First analysis-meta relate to second analysis-meta. Used to express relations between two groups where each group's components correspond to the respective component in the other group. Only use for criteria. An opaque #GstMeta that can be used to hold various types of results from analysis processes. The content should be accessed through the API. Add analytic classification metadata to @instance. Added successfully Instance of #GstAnalyticsRelationMeta where to add classification instance length of @confidence_levels confidence levels labels of this classification. Order define index, quark, labels relation. This array need to exist as long has this classification meta exist. Handle updated to newly added classification meta. Add a relatable metadata to @meta. This method is meant to be used by new struct sub-classing GstAnalyticsRelatableMtd. A pointer to a memory area of size @size where to put the data Instance Implementation of relatable (#GstAnalyticsRelatableMtd) Size required Updated handle Added successfully Instance of #GstAnalyticsRelationMeta where to add classification instance Quark of the object type x component of bounding box upper-left corner y component of bounding box upper-left corner bounding box width bounding box height confidence level on the object location Handle updated with newly added object detection meta. Add an object-detetion metadata to @instance. Add analytic classification metadata to @instance. Added successfully Instance of #GstAnalyticsRelationMeta where to add classification instance confidence levels labels of this classification. Order define index, quark, labels relation. This array need to exist as long has this classification meta exist. Handle updated to newly added classification meta. Added successfully Instance of #GstAnalyticsRelationMeta where to add classification instance Quark of the object type x component of bounding box upper-left corner (pre-rotation) y component of bounding box upper-left corner (pre-rotation) bounding box width bounding box height bounding box rotation in radians <0, 2xPI> with respect to the bounding box center (the rotation value is a clock-wise angle) confidence level on the object location Handle updated with newly added object detection meta. Add an object-detetion metadata to @instance. Add analytics segmentation metadata to @instance. The rectangle (@masks_loc_x, @mask_loc_y, @mask_loc_w, @mask_loc_h) define a area of the image that correspond to the segmentation masks stored in @buffer. For example if the segmentation masks stored in @buffer describe the segmented regions for the entire image the rectangular area will be (@masks_loc_x = 0, @masks_loc_y = 0, @masks_loc_w = image_width, @masks_loc_h = image_height). TRUE if added successfully, otherwise FALSE Instance of #GstAnalyticsRelationMeta where to add segmentation instance. Buffer containing segmentation masks. @buffer must have a #GstVideoMeta attached Segmentation type Number of regions in the masks Arrays of region ids present in the mask. Left coordinate of the rectangle corresponding to the masks in the image. Top coordinate of the rectangle corresponding to the masks in the image. Width of the rectangle corresponding to the masks in the image. Height of the rectangle corresponding to the masks in the image. Handle update with newly added segmentation meta. Add a new #GstAnalyticsTensorMtd holding a #GstTensor to @instance. The #GstTensor needs to be filled. Added successfully Instance The number of dimensions in the tensor Handle update with newly added tensor mtd. Add tensor mtd to @instance. Add a new #GstAnalyticsTensorMtd holding a #GstTensor to @instance. Added successfully Instance semantically identify the contents of the tensor #GstTensorDataType of tensor data #GstBuffer holding tensor data Indicate tensor dimension indexing order number of tensor dimensions size of tensor in each dimension. A value of 0 means the dimension is dynamic. Handle update with newly added tensor mtd. Add tensor mtd to @instance. Added successfully Instance of GstAnalyticsRelationMeta where to add tracking mtd Tracking id Timestamp of first time the object was observed. Handle updated with newly added tracking meta. Add an analytic tracking metadata to @instance. Verify existence of relation(s) between @an_meta_first_d and @an_meta_second_id according to relation condition @cond_types. It optionally also return a shortest path of relations ( compliant with @cond_types) between @an_meta_first_id and @an_meta_second_id. TRUE if a relation between exit between @an_meta_first_id and @an_meta_second_id, otherwise FALSE. a #GstAnalyticsRelationMeta describing analysis-meta relation First analysis-meta Second analysis-meta Maximum number of relation between @an_meta_first_id and @an_meta_second_id. A value of 1 mean only only consider direct relation. condition on relation types. If not NULL this list will be filled with relation path between @an_meta_first_id and @an_meta_second_id. List value should be access with GSList API. Use GPOINTER_TO_INT(iter->data) where iter is a GSList element to get analysis-meta id on the relation path. Free this list with g_slist_free (@relations_path) after using. Fill @rlt if a analytics-meta with id == @an_meta_id exist in @meta instance, otherwise this method return FALSE and @rlt is invalid. TRUE if successful. Instance of #GstAnalyticsRelationMeta Id of #GstAnalyticsClsMtd instance to retrieve Will be filled with relatable meta TRUE if @rlt_mtd was updated, other wise FALSE GstAnalyticsRelationMeta instance where to query for GstAnalyticsRelatableMtd. Id of GstAnalyticsMtd involved in relation to query Type of relation to filter on. Type of GstAnalyticsMtd to filter on Opaque data to store state of the query. If @state point to NULL, the first analytics-metadata directly related to @an_meta_id will be set in @rlt_mtd. Doesn't need to be free. Handle updated to directly related relatable meta. Fill @rlt if a analytics-meta with id == @an_meta_id exist in @meta instance, otherwise this method return FALSE and @rlt is invalid. TRUE if successful. Instance of GstAnalyticsRelationMeta Id of GstAnalyticsMtd instance to retrieve Filter on a specific type of analysis, use %GST_ANALYTICS_MTD_TYPE_ANY to match any type Will be filled with relatable meta Analytics data pointer Instance of GstAnalyticsRelationMeta Id of GstAnalyticsMtd instance to retrieve Fill @rlt if a analytics-meta with id == @an_meta_id exist in @meta instance, otherwise this method return FALSE and @rlt is invalid. TRUE if successful. Instance of #GstAnalyticsRelationMeta Id of #GstAnalyticsODMtd instance to retrieve Will be filled with relatable meta Get relations between first and second analysis-meta. Ids (@an_meta_first_id and @an_meta_second_id) must be from a call to @gst_analytics_mtd_get_id (handle). relation description between first and second analysis-meta. a #GstAnalyticsRelationMeta Id of first analysis-meta Id of second analysis-meta Fill @rlt if a analytics-meta with id == @an_meta_id exist in @meta instance, otherwise this method return FALSE and @rlt is invalid. TRUE if successful. Instance of #GstAnalyticsRelationMeta Id of #GstAnalyticsSegmentationMtd instance to retrieve Will be filled with relatable meta Fill @rlt if a analytics-meta with id == @an_meta_id exist in @meta instance, otherwise this method return FALSE and @rlt is invalid. TRUE if successful. Instance of #GstAnalyticsRelationMeta Id of #GstAnalyticsTensorMtd instance to retrieve Will be filled with relatable meta Fill @rlt if a analytics-meta with id == @an_meta_id exist in @meta instance, otherwise this method return FALSE and @rlt is invalid. TRUE if successful. Instance of GstAnalyticsRelationMeta Id of GstAnalyticsMtd instance to retrieve Will be filled with relatable meta FALSE if end was reached and iteration is completed. Instance of GstAnalyticsRelationMeta Opaque data to store iteration state, initialize to NULL, no need to free it. Type of GstAnalyticsMtd to iterate on or use %GST_ANALYTICS_MTD_TYPE_ANY for any. Handle updated to iterated GstAnalyticsRelatableMtd. Sets the relation (#GstAnalyticsRelTypes) between @an_meta_first and @an_meta_second. Ids must have been obtained a call to @gst_analytics_mtd_get_id(handle). TRUE on success and FALSE on failure. Parameter to receive new maximum number of analysis-meta described by relation. a #GstAnalyticsRelTypes defining relation between two analysis-meta first meta id second meta id Get the meta info GstAnalyticsRelationMeta initialization parameters. Initial relations order. Buffer size in bytes to store relatable metadata This type of metadata holds information on which pixels belongs to a region of the image representing a type of object. It supports two types of segmentation, semantic or instance: * Semantic: All objects of the same type have the same id * Instance: Each instance of an object has a different id The results of the segmentation are stored in a #GstBuffer that has a #GstVideoMeta associated with it. This buffer is stored in the GstAnalyticsSegmentationMtd using #gst_analytics_relation_meta_add_segmentation_mtd(). The #GstBuffer containing the segmentation mask is image-like but the color values are arbitrary values, referred by region-id in this API, without meaning beyond specifying that two pixels in the original image with the same values in their corresponding mask value belong to the same region. To further describe a region, the #GstAnalyticsSegmentationMtd can be associated with other #GstAnalyticsMtd. Since region ids are generated by the segmentation process itself and are not always sequential, we use a map of indexes to region ids starting with 0 without discontinuity which facilitate N-to-N mapping with other #GstAnalyticsMtd. For example it can be associated with #GstAnalyticsClsMtd to describe the class of object matching the pixels of a segmented region. Example: Associate Instance Segmentation with Classification In the following example the segmentation process will fill segmask with values of 0 for background, 12 for the first region which correspond to a to a strawberry, 7 for the second region that also correspond to a strawberry in the image and 31 for the third region that correspond to a leaf in the image. region_ids is fill during segmentation post-processing region_ids: |region-index | region-id | |-------------|-----------| | 0 | 0 | | 1 | 12 | | 2 | 7 | | 3 | 31 | region_count = 4 ``` C GstAnalyticsSegmentationMtd segmtd; GstAnalyticsClassificationMtd clsmtd; GstBuffer *segmask, *img; guint *region_ids; gsize region_count, class_count; gfloat *class_confidence; GQuark *classes; ... (segmentation filling segmask based on img) gst_analytics_relation_meta_add_segmentation_mtd (rmeta, segmask, GST_SEGMENTATION_TYPE_INSTANCE, region_count, region_ids, &segmtd); class_count = region_count; ... (class-index must match and correspond to region-index) classes [0] = g_quark_from_string ("background"); classes [1] = g_quark_from_string ("strawberry"); classes [2] = g_quark_from_string ("strawberry"); classes [3] = g_quark_from_string ("leaf"); ... (set confidence level for each class associated with a region ... where -1.0 mean undefined.) class_confidence [0] = -1.0; class_confidence [1] = 0.6; class_confidence [2] = 0.9; class_confidence [3] = 0.8; gst_analytics_relation_meta_add_cls_mtd (rmeta, class_count, class_confidence, classes, &clsmtd); gst_analytics_relation_meta_set_relation (rmeta, GST_ANALYTICS_REL_TYPE_RELATE_TO, segmtd.id, clsmtd.id); ``` Example: Associate Semantic Segmentation with Classification Assuming the same context as for Instance Segmentation above but instead a semantic segmentation is performed, therefore region-id-12 and region-id-7 are now represented by the same region-id-12 region_ids: (here |region-index | region-id | |-------------|-----------| | 0 | 0 | | 1 | 12 | | 2 | 31 | Code remain the same except that we set all confidence level to undefined (-1.0). ``` ... (class-index must match and correspond to region-index) classes [0] = g_quark_from_string ("background"); classes [1] = g_quark_from_string ("strawberry"); classes [2] = g_quark_from_string ("leaf"); ... (set confidence level for each class associated with a region ... where -1.0 mean undefined.) class_confidence [0] = -1.0; class_confidence [1] = -1.0; class_confidence [2] = -1.0; gst_analytics_relation_meta_add_cls_mtd (rmeta, class_count, class_confidence, classes, &clsmtd); gst_analytics_relation_meta_set_relation (rmeta, GST_ANALYTICS_REL_TYPE_RELATE_TO, segmtd.id, clsmtd.id); ``` Example: Retrieving class associated with a segmentation region-id-12 This the typical case for an overlay as we visit the segmentation mask we we find region-id values ``` gsize idx; gst_analytics_segmentation_mtd_get_region_index (&segmtd, &idx, 12); gst_analytics_relation_meta_get_direct_related (rmeta, segmtd.id, GST_ANALYTICS_REL_TYPE_RELATE_TO, gst_analytics_cls_mtd_get_type (), NULL, &clsmtd); GQuark region_class = gst_analytics_cls_mtd_get_quark (&segmtd, idx) ... ``` Since: 1.26 Instance identifier Instance of #GstAnalyticsRelationMeta where the analytics-metadata identified by @id is stored Get segmentation mask data. Segmentation mask data stored in a #GstBuffer Instance Left coordinate of the rectangle corresponding to the mask in the image. Top coordinate of the rectangle corresponding to the mask in the image. Width of the rectangle corresponding to the mask in the image. Height of the rectangle corresponding to the mask in the image. Get the regions count. Number of regions segmented Instance Get id of the region corresponding to @index, which should be smaller than the return value of gst_analytics_segmentation_mtd_get_region_count() The region ID Instance Region index Get region index of the region identified by @id. TRUE if a region with @id exist, otherwise FALSE Instance Region index Region id Get an instance of #GstAnalyticsMtdType that represent segmentation metadata type. A #GstAnalyticsMtdType type Enum value describing supported segmentation type Segmentation where the belonging of each pixel to a class of objects is identified. Segmentation where the belonging of each pixel to instance of an object is identified. Hold tensor data semantically identify the contents of the tensor Indicate tensor layout #GstTensorDataType of tensor data #GstBuffer holding tensor data Indicate tensor elements layout in memory. number of tensor dimensions number of tensor dimensions Allocate a tensor with @num_dims dimensions. tensor allocated Number of dimension of the tensors Allocates a new #GstTensor of @dims_order ROW_MAJOR or COLUMN_MAJOR and with an interleaved layout. For example, a two-dimensional tensor with 32 rows and 4 columns, @dims would be the two element array `[32, 4]`. A newly allocated #GstTensor semantically identify the contents of the tensor #GstTensorDataType of tensor data #GstBuffer holding tensor data Indicate tensor dimension indexing order number of tensor dimensions size of tensor in each dimension. A value of 0 means the dimension is dynamic. Validate the tensor whether it mathces the reading order, dimensions and the data type. Validate whether the #GstBuffer has enough size to hold the tensor data. TRUE if the #GstTensor has the reading order from the memory matching @order, dimensions matching @num_dims, data type matching @data_type Otherwise FALSE will be returned. A #GstTensor The data type of the tensor The order of the tensor to read from the memory The number of dimensions that the tensor can have An optional array of dimensions, where G_MAXSIZE means ANY. Create a copy of @tensor. a new #GstTensor a #GstTensor to be copied Free tensor pointer to tensor to free Gets the dimensions of the tensor. The dims array form the tensor a #GstTensor The number of dimensions Sets the content of a #GstTensor of @dims_order ROW_MAJOR or COLUMN_MAJOR and with an interleaved layout. The #GstTensor must have exactly num_dims. For example, a two-dimensional tensor with 32 rows and 4 columns, @dims would be the two element array `[32, 4]`. TRUE if it coudl be set correctly a #GstTensor semantically identify the contents of the tensor #GstTensorDataType of tensor data #GstBuffer holding tensor data Indicate tensor dimension indexing order number of tensor dimensions size of tensor in each dimension. A value of 0 means the dimension is dynamic. Get a string version of the data type a constant string with the name of the data type a #GstTensorDataType Describe the type of data contain in the tensor. signed 4 bit integer tensor data signed 8 bit integer tensor data signed 16 bit integer tensor data signed 32 bit integer tensor data signed 64 bit integer tensor data unsigned 4 bit integer tensor data unsigned 8 bit integer tensor data unsigned 16 bit integer tensor data unsigned 32 bit integer tensor data unsigned 64 bit integer tensor data 16 bit floating point tensor data 32 bit floating point tensor data 64 bit floating point tensor data "brain" 16 bit floating point tensor data UTF-8 string A boolean value stored in 1 byte. A 64-bit complex number stored in 2 32-bit values. A 128-bit complex number stored in 2 64-bit values. A non-IEEE 8-bit floating point format with 4 exponent bits and 3 mantissa bits, with NaN and no infinite values (FN). See [this paper for more details](https://onnx.ai/onnx/technical/float8.html) A non-IEEE 8-bit floating point format with 4 exponent bits and 3 mantissa bits, with NaN, no infinite values (FN) and no negative zero (UZ). See [this paper for more details](https://onnx.ai/onnx/technical/float8.html) A non-IEEE 8-bit floating point format with 5 exponent bits and 2 mantissa bits. See [this paper for more details](https://onnx.ai/onnx/technical/float8.html) A non-IEEE 8-bit floating point format with 5 exponent bits and 2 mantissa bits, with NaN, no infinite values (FN) and no negative zero (UZ). See [this paper for more details](https://onnx.ai/onnx/technical/float8.html) Indicate to read tensor from memory in row-major or column-major order. elements along a row are consecutive in memory elements along a column are consecutive in memory Indicate tensor storage in memory. indicate the tensor is stored in a dense format in memory parent number of tensors Retrieves a tensor from the #GstTensorMeta, the index must be smaller than #GstTensorMeta.num_tensors a GstTensor A #GstTensorMeta The number of the tensor to get Get the first tensor from the #GstTensorMeta identified by @id. a GstTensor with id matching @id. Otherwise NULL will be returned. A #GstTensorMeta A #GQuark identifying tensor-encoding Finds the first tensor with the requsted ID in the meta The index of the tensor inthe meta, or -1 if its not found. a #GstTensorMeta The tensor id to look for Get the first tensor from the #GstTensorMeta identified by @tensor_id, matching the reading order, dimensions and the data type and optionally the dimensions. Validate whether the #GstBuffer has enough size to hold the tensor data. a matching #GstTensor, otherwise NULL A #GstTensorMeta A #GQuark identifying the tensor-encoding The data type of the tensor The order of the tensor to read from the memory The number of dimensions that the tensor can have An optional array of dimensions, where G_MAXSIZE means ANY. Sets tensors into the #GstTensorMeta a #GstTensorMeta The number of tensors in the @tensors array An array of poiners to #GstTensor This type of metadata holds a tensor. It can be used to store tensor as analytics-meta for their ability to relate to each others. For example in a multi-model analytics pipeline, we sometime have one model input match the output of the other model. In this context it can be useful to keep the ancestry relation between first tensor, output of first inference, and the second tensor, output from second inference. Another use-case for #GstAnalyticsTensorMtd is to transport tensors from inference element to a post-processing element using a computing graph framework, like ONNX. Essentially #GstAnalyticsTensorMtd is a GstBuffer encapsulated by a analytics-meta with additional parameters describing the tensor. Since 1.28 Instance identifier Instance of #GstAnalyticsRelationMeta where the analytics-metadata identified by @id is stored Get tensor a #GstTensor Instance of #GstAnalyticsTensorMtd Get an id that represent tensor metadata type Opaque id of the #GstAnalyticsMtd type Store information on results of object tracking Instance identifier Instance of #GstAnalyticsRelationMeta where the analytics-metadata identified by @id is stored Retrieve tracking information. Successfully retrieved info. Instance of tracking metadata Updated tracking id Updated timestamp of the tracking first observation. Updated timestamp of the tracking last observation. Has the tracking been lost Update successful Instance of GstAnalyticsTrackingMtd. Set tracking to lost GstAnalyticsTrackingMtd instance Timestamp of last time this object was tracked id representing the type of GstAnalyticsRelatableMtd Get the opaque id identifying the relatable type Adds a #GstAnalyticsBatchMeta to a buffer or returns the existing one The new #GstAnalyticsBatchMeta A writable #GstBuffer Attach a analysis-results-meta-relation meta (#GstAnalyticsRelationMeta)to @buffer. A #GstAnalyticsRelationMeta is a metadata describing relation between other analysis meta. It's more efficient to use #gst_buffer_add_analytics_relation_meta_full and providing the maximum number of analysis meta that will attached to a buffer. Newly attached #GstAnalyticsRelationMeta a #GstBuffer Attache a analysis-results relation-meta (#GstAnalyticsRelationMeta) to @buffer. A #GstAnalyticsRelationMeta is a metadata describing relation between other analysis meta. Newly attached #GstAnalyticsRelationMeta a #GstBuffer Initialization parameters Adds a #GstTensorMeta to a buffer The new #GstTensorMeta A writable #GstBuffer Gets the #GstAnalyticsBatchMeta from a buffer The #GstAnalyticsBatchMeta if there is one A #GstBuffer Retrives the meta or %NULL if it doesn't exist The #GstAnalyticsRelationMeta if there is one a #GstBuffer Gets the #GstTensorMeta from a buffer The #GstTensorMeta if there is wone A #GstBuffer Get an id identifying #GstAnalyticsMtd type. opaque id of #GstAnalyticsMtd type This type of metadata holds classification, it is generally used in relationship with another metadata type to enhance its content. For example, it can enhance the classifcation of an object detection held by the #GstAnalyticsODMtd metadata type. The #GstAnalyticsRelationMeta is a #GstMeta that can contain a large number of results from the analysis of a meta. Each result can be accessed by using its id, or more conviently, by using a #GstAnalyticsMtd. A matrix of relationships between the various metadata is also defined and can be filled by the analysis processes. This type of metadata holds the position of detected object inside the image, along with the probabily of each detection. This type of metadata holds tracking information. In many cases, it is desired to track an object across many frames. This type of metadata holds information about the tracking, for example, it can be used alongside a #GstAnalyticsODMtd to track an object. Calculate the intersection over the union (IoU) of the two areas defined by the bounding box 1 and bounding box 2. IoU is a measure of how much two regions overlap. IoU of bb1 and bb2. Bounding box 1, X coordinate Bounding box 1, Y coordinate Bounding box 1, width Bounding box 1, height Bounding box 2, X coordinate Bounding box 2, Y coordinate Bounding box 2, width Bounding box 2, height Calculate the intersection over the union (IoU) of the two areas defined by the bounding box 1 and bounding box 2. IoU is a measure of how much two regions overlap. IoU of bb1 and bb2. Bounding box 1, X coordinate Bounding box 1, Y coordinate Bounding box 1, width Bounding box 1, height Bounding box 2, X coordinate Bounding box 2, Y coordinate Bounding box 2, width Bounding box 2, height Load a modelinfo file associated with the given model file. This function attempts to load a `.modelinfo` file in the following order: 1. `{model_filename}.modelinfo` 2. `{model_filename_without_extension}.modelinfo` The modelinfo file contains metadata for the model's input and output tensors, including normalization ranges, dimension ordering, tensor IDs, etc. The loaded modelinfo must be freed with gst_analytics_modelinfo_free() when no longer needed. A new #GstAnalyticsModelInfo instance, or %NULL if the modelinfo file could not be found or loaded. Path to the model file (e.g., "model.onnx", "model.tflite") Gets the string version of the name of this type of analytics data the name The type of analytics data Get an id that represent object-detection metadata type Opaque id of the #GstAnalyticsMtd type Get number of relatable meta attached to instance Number of analysis-meta attached to this instance. Instance of #GstAnalyticsRelationMeta GType of GstAnalyticsRelationMeta Get the meta info Get an instance of #GstAnalyticsMtdType that represent segmentation metadata type. A #GstAnalyticsMtdType type Get a string version of the data type a constant string with the name of the data type a #GstTensorDataType Get an id that represent tensor metadata type Opaque id of the #GstAnalyticsMtd type id representing the type of GstAnalyticsRelatableMtd Get the opaque id identifying the relatable type