FPGA Image Signal Processor

FPGA-ISP: Higher Throughput, Less Power Consumption

 

FPGA enters into the game of image processing when the application goes beyond the GPU domain. This allows the developers to connect peripherals directly to the FPGA or create really complex algorithms that can be accelerated on FPGA. In turn, this offers higher throughput with less power consumption. 

 

programming-code-tech-background.jpg

Examples of FPGA

Some examples to illustrate the scope of the FPGA on image processing:

 

Cameras connected directly to the FPGA on custom hardware in purely industrial applications.

Custom devices with sensors and actuators, where some actions are taken according to the image: color histogram under a threshold, illumination adjustment, monitoring the quality of a production line and defects detection.

Artificial intelligence on the edge and with low power consumption.

 

The FPGA Image Signal Processing (ISP) Library is a collection of accelerators and templated C++ modules to perform image processing on FPGA-based video streams for Vivado High-Level Synthesis (Vivado HLS*). Since FPGA ISP is purely based on hardware description, you can create low-latency ISP applications which execute in a highly predictable way.

 

The real power of FPGA ISP is the capability of interconnecting several ISP modules in cascade without having a penalty of executing each module individually in a sequential fashion. This capability is thanks to the pipelining and data flow programming models available when describing ISPs with High-Level Synthesis (see Figure 1).

fpga-concurrent-execution-model.png

Figure 1 - Concurrent execution model

Available FPGA Modules

Currently, FPGA ISP in its 0.8.0 release has the following modules:

  • 8-bit demosaicing (Debayer) to 32-bit ARGB

  • Color space converter (UYVY - ARGB)

  • Auto white balancer

  • Histogram equalization

  • Geometric Transformation Unit for perspective transformation

  • Lens undistort: Brown-Conrady and Fish-Eye

  • High-throughput grayscale convolution

  • Interpolation: Truncation, Nearest-Neighbors, Bilinear

  • Fast-Fourier Transform

  • Channel splitting and merging

  • Matrix-matrix multiplication and addition

More modules can be developed and optimized on-demand. Please contact us for more information about project sponsorship and our engineering hours.

Extending FPGA ISP Features

FPGA ISP is 100% compatible with V4L2 FPGA. This combination allows users to get a better experience with Image Processing on FPGAs, exposing them as V4L2 compliant devices on Linux. Moreover, FPGA ISP is fully based on the V4L2 FPGA development ecosystem, allowing you to use your accelerators with user-level software such as GStreamer.

Learn more.


Platforms Supported by FPGA ISP

FPGA ISP is compatible with the following platforms, thanks to Vivado HLS:

Artix7

Zynq-MP SoC

Kintex7

Vertex7

Zynq-7000

NOTE: Additional platforms may be supported. Contact us for additional information.

License

*Vivado HLS is a tool from Xilinx

Suggested Support Bundles

+11 support hours: $3499