Graphics on video streams are more common that you would think. Digital cameras, for example, overlay icons showing different values such as brightness, exposure, flash status and battery charge. Reverse car cameras render the distance from the bumper to the nearest object and a virtual path for the car to follow given the steering wheel's current position. Medical equipment, navigation controls and home appliances all benefit from graphics overlaying. Even our mirrors and glasses present icons for us now!
The reason behind this is mainly because graphics transmit lots of information without taking much of the video space. There are many frameworks that will allow you to make an arrangement of video and icons: QT, GTK, Flutter, Swift, etc... However all these are designed to render graphics in a display. What if you wanted all these graphics in an off-screen destination? Think of a file, a network stream or a pointer in memory. Up until now there was no direct way to achieve this using well known technology.
Welcome GstQtOverlay!
GstQtOverlay is a GStreamer plug-in that allows you to render QT graphics in a GStreamer buffer. It does this using GPU acceleration. As a user, you pass in a GUI (Graphical User Interface) in the form of a QML text file and the engine overlays the result in a given memory. This basically allows you to use all the power of the QT framework within a GStreamer element linked to your pipeline! As per now, GstQtOverlay is "output-only" meaning that, while you can display virtually anything that can be displayed in a QT application, you cannot receive user input (a mouse click or a key press).
Render rich graphics on GStreamer video buffers. QML is a well known language among designers and GstQtOverlay allows them to design and test their GUI in their personal computer, only to later be deployed in the embedded system by a developer. And the best of all, graphics are immediately available in all the branches of the pipeline.
Optimized for Embedded Systems
One of the key aspects - and design criteria - was that it had to perform in real time when ran in embedded systems. To do so GstQtOverlay leverages QT's native OpenGL engine. While most modern SoC (System on Chip) have compliant OpenGL implementations, many of them require advanced tricks to achieve full performance and, even, a zero copy processing. GstQtOverlay decouples the platform by adding a HAL (Hardware Abstraction Layer) which deals with vendor specifics.
GstQtOverlay is highly optimized for NXP i.MX6 and NVIDIA Jetson boards. Both of them use the vendor's proprietary extensions to avoid unnecessary memory transfers that hurt performance. The more technical reader may appreciate the fact that the element uses Vivante's direct memory access when processing on the i.MX6 and NVMM memory when doing so in a Jetson board.
Results are impressive!
Learn more in our developers wiki.
Want to give it a try? Ask us for an evaluation version! Contact us if you have any question.
Purchase directly from our store.