• ridgerun

How to use Xilinx OpenCV on RidgeRun V4L2 FPGA and FPGA ISP

Updated: Jul 10, 2020

The Xilinx OpenCV library is now supported by V4L2 FPGA and FPGA ISP projects. Thanks to the adapters from the V4L2 FPGA project, it’s now possible to extend the functionality of your accelerators. The supported Xilinx OpenCV libraries are:

For the list of compatible functions, please visit our developer wiki List of Xilinx OpenCV functions compatible with V4L2_FPGA

Similarly to the FPGA ISP development flow, it’s possible to get the same advantages of data flow processing and pipelining by using Xilinx OpenCV, since they are both built for Vivado HLS.

Describing a custom accelerator

The example accelerator is composed by the following modules:

Figure 1 - Example of a custom accelerator

The image representation in V4L2 FPGA is different from the Xilinx OpenCV Mat representation, since the data stream in V4L2 FPGA is fixed to 64-bits. To connect the streams to the Xilinx OpenCV module, a data type conversion is required. The V4L2 FPGA also facilitates a couple of interface functions to carry out this job:

  • utils::stream::rraxi2xfMat(stream_in, xfmat_out, width, height): to transform from the stream to the Xilinx OpenCV Mat.

  • utils::stream::xfMat2rraxi(xfmat_in, stream_out, width, height): to transform from the Xilinx OpenCV Mat to stream.

Taking into account the functions listed above, it is possible to code of the accelerator as follows:

The variables which interconnect the several modules are stream variables (FIFO-based variables). Also, note that the interfaces are just placed at the input and at the output of the pipeline. To interface to a FPGA ISP module, the same process is required to transform the data types.

About the throughput

As explained in our previous blog, RidgeRun FPGA-ISP to easily create a Image Signal Processing ISP on FPGA, the throughput of the whole custom accelerator will be limited by the slowest part of the pipeline.

According to the tests of this accelerator, on the PicoEVB at 4K:

  • Theoretical throughput: 48 fps

  • Experimental throughput: 40 fps (limited by PCIe 2.0 x1 bandwidth)

For more information, please refer to FPGA Image Signal Processing with Xilinx OpenCV and V4L2-FPGA.


After running the tests on an image of 2048x1536, the time of solution was 803910 clock cycles.

Figure 2 - Original picture (grayscaled in software)

Figure 3 - Output image from the FPGA

One of the important remarks about the output image is that the result is approximated by the Xilinx OpenCV, since the operations are done by using fixed-point arithmetic instead of floating-point to save resources. This can be seen by an error image (Software OpenCV - Xilinx OpenCV) in the following picture.

Figure 4 - Error of Xilinx OpenCV respect to the same pipeline in OpenCV performed on software

Current development and research

  • Support for memory-based algorithms, which requires to pass the image more than once.

  • Driver development for ZYNQ-based platforms.

Learn more in

Related blog from RidgeRun : RidgeRun FPGA-ISP to easily create a Image Signal Processing ISP on FPGA

Any Questions? : support@ridgerun.com

Contact Us

Visit our Main Website for the RidgeRun online store, for pricing information of the RidgeRun products and Professional Services. Please email support@ridgerun.com for technical questions. Contact details for sponsoring the RidgeRun GStreamer projects are available at Sponsor Projects page.

86 views0 comments

Recent Posts

See All