FPGA Image Signal Processor
FPGA-ISP: Higher Throughput, Less Power Consumption
FPGA enters into the game of image processing when the application goes beyond the GPU domain. This allows the developers to connect peripherals directly to the FPGA or create really complex algorithms that can be accelerated on FPGA. In turn, this offers higher throughput with less power consumption.
Examples of FPGA
Some examples to illustrate the scope of the FPGA on image processing:
Cameras connected directly to the FPGA on custom hardware in purely industrial applications.
Custom devices with sensors and actuators, where some actions are taken according to the image: color histogram under a threshold, illumination adjustment, monitoring the quality of a production line and defects detection.
Artificial intelligence on the edge and with low power consumption.
The FPGA Image Signal Processing (ISP) Library is a collection of accelerators and templated C++ modules to perform image processing on FPGA-based video streams for Vivado High-Level Synthesis (Vivado HLS*). Since FPGA ISP is purely based on hardware description, you can create low-latency ISP applications which execute in a highly predictable way.
The real power of FPGA ISP is the capability of interconnecting several ISP modules in cascade without having a penalty of executing each module individually in a sequential fashion. This capability is thanks to the pipelining and data flow programming models available when describing ISPs with High-Level Synthesis (see Figure 1).
Figure 1 - Concurrent execution model
Available FPGA Modules
Currently, FPGA ISP in its 0.8.0 release has the following modules:
8-bit demosaicing (Debayer) to 32-bit ARGB
Color space converter (UYVY - ARGB)
Auto white balancer
Geometric Transformation Unit for perspective transformation
Lens undistort: Brown-Conrady and Fish-Eye
High-throughput grayscale convolution
Interpolation: Truncation, Nearest-Neighbors, Bilinear
Channel splitting and merging
Matrix-matrix multiplication and addition
More modules can be developed and optimized on-demand. Please contact us for more information about project sponsorship and our engineering hours.
Extending FPGA ISP Features
FPGA ISP is 100% compatible with V4L2 FPGA. This combination allows users to get a better experience with Image Processing on FPGAs, exposing them as V4L2 compliant devices on Linux. Moreover, FPGA ISP is fully based on the V4L2 FPGA development ecosystem, allowing you to use your accelerators with user-level software such as GStreamer.
Platforms Supported by FPGA ISP
FPGA ISP is compatible with the following platforms, thanks to Vivado HLS: