Exploring NVIDIA Holoscan SDK: Architecture, Custom Operators, And Performance On Jetson
A RidgeRun evaluation of Holoscan SDK, its graph-based application model, custom operators, and performance on NVIDIA Jetson.
Introduction
The NVIDIA Holoscan SDK is designed for developing real-time sensor processing and edge-AI applications. It allows developers to connect capture, processing, inference, visualization, and control stages in a single application graph.
Throughout this article, “Holoscan SDK” and “HSDK” refer specifically to NVIDIA Holoscan SDK version 5.0.
RidgeRun evaluated HSDK on an NVIDIA Jetson AGX Orin using an IMX274 camera. The evaluation included custom HSDK operators for V4L2 capture, raw Bayer processing, TensorRT inference, and display. We also compared the measured pipelines with GStreamer and DeepStream.
This article explains the HSDK application model, the development changes, the operators used in the RidgeRun pipelines, and the results of the camera and inference benchmarks.
What Is Holoscan SDK?
NVIDIA Holoscan is an ecosystem for developing real-time sensor-processing and edge-AI applications. It brings together the software components, modules, operators, and integration paths used to build applications for cameras, sensors, and other devices.
An application built with the Holoscan SDK is represented as a graph of operators. Each operator performs one part of the workload, such as capturing a frame, converting a format, preparing a tensor, running inference, displaying an image, or publishing data to another process.
In HSDK the graph is compiled into an execution plan before the application runs. The execution plan defines how the connected operators are scheduled and executed by the runtime.
An operator performs one unit of work. It can capture a frame, convert a data format, prepare a tensor, run inference, display an image, or publish data to another process. Operators expose named input and output ports, and samples move between them according to the graph’s data and timing contracts.
A typical sensor-processing application follows this path:
This model keeps the responsibilities of the application visible. During application design, developers can choose a compatible camera source without rewriting the inference stage. They can also build a variant without a display stage when measuring inference only, or add operators for filtering, sensor fusion, format conversion, post-processing, or communication with another process.
Graph changes must be made before the execution plan is compiled. If the graph changes, the execution plan must be rebuilt or recompiled. HSDK does not dynamically update a graph while it is running.
The graph also gives developers a place to reason about the data being moved: what a sample contains, where its memory resides, which operator owns it, and when the next operator is allowed to execute. HSDK builds on this graph model by making more of the data, timing, and execution decisions visible in the application design.
Key Capabilities of HSDK
HSDK builds on the graph model by making the decisions that control data flow and execution more explicit. Connections identify which ports exchange data, payload types define what moves between operators, and timing contracts describe when an operator can run and which samples it should use.
The application then moves through composition, compilation, and execution. During composition, the developer creates operators and connects compatible ports. Compilation checks the graph declarations and creates an execution plan. The runtime executes the accepted plan through a blocking run or a run session for asynchronous control.
This separation helps identify where a problem occurs. Graph and contract errors can be detected before execution, while device availability, model loading, and processing failures must be checked at runtime. Together, these decisions define how a graph moves from its initial description to a running application.
HSDK Life-cycle And Architecture
The main life-cycle can be summarized as follows:
The architecture has two related views. The first view describes the runtime life-cycle. The second view describes the data path inside the graph:
The important point is that the graph is the application architecture. The operators are the reusable processing units, and the runtime plan determines how the connected work is scheduled and executed on the target system. The life-cycle defines how the application is prepared and run. The operators define the work performed at each stage.
HSDK Operators and Integration Options
HSDK organizes its functionality into modules. A module is a collection of related operators and sample applications for a specific workflow.
The Core module provides general-purpose processing operators. Holoscan Camera is the camera module, with operators for camera sources and camera data processing. Holoscan RidgeRun is a RidgeRun-developed Holoscan module that provides operators and sample applications for camera capture, raw Bayer processing, TensorRT inference, display, and benchmarking.
These modules can be combined in the same Holoscan application graph. The Core operators provide the general-purpose processing blocks used together with the camera and RidgeRun-specific components.
Core Operators
Core operators are the general-purpose building blocks provided by the SDK. They cover common operations such as tensor and image conversion, resizing, normalization, inference, visualization, and other processing steps.
Examples include ReformatOp, ResizeOp, InferenceOp, and HolovizOp. They can reformat data, resize images or tensors, run inference, and visualize results.
Holoscan Camera Operators
Holoscan Camera is a HSDK module that provides camera-oriented sources and processing paths. It connects supported camera interfaces to a HSDK graph and produces data for visualization or later processing.
The module separates camera integration from the rest of the application. The graph can then connect the camera output to SDK operators, custom operators, or an inference stage.
Holoscan RidgeRun Operators
Holoscan RidgeRun is a HSDK module that contains a set of operators and sample applications developed by RidgeRun. It includes operators for camera capture, raw Bayer processing, TensorRT inference, display, and benchmark pipelines.
Representative operators include:
V4L2UserPtrSourceOp for CUDA-mapped V4L2 USERPTR RG10 capture.
RawBayerToBgraOp for converting raw Bayer data into displayable BGRA data.
RawBayerToYoloTensorOp for fused Bayer-to-YOLO CUDA preprocessing.
TensorRtInferenceOp for static single-input, single-output TensorRT models.
DrmBgraDisplayOp for sending BGRA frames to a DRM display.
FakesinkOp for consuming results without adding display work.
These operators can be connected to build a camera-to-inference pipeline such as:
V4L2UserPtrSourceOp publishes unpacked RG10 values as a CUDA-visible U16 tensor, avoiding an additional memory copy to the GPU before CUDA processing.
RawBayerToYoloTensorOp performs Bayer demosaicing, RGB conversion, aspect-ratio-preserving resize, letterbox padding, optical-black correction, gain adjustment, and YOLO normalization. It emits the CUDA FP32 tensor expected by the model.
TensorRtInferenceOp executes the TensorRT network directly on CUDA tensor memory. It can load a serialized TensorRT engine or build an engine from an ONNX model during startup. The output is the raw network tensor; detection decoding, box filtering, overlays, and display remain separate stages.
For a display pipeline, the graph can use this alternative:
This distinction is important when interpreting performance. A display graph and a headless inference graph do not perform the same work and should not be compared as if they had identical measurement boundaries. These operators form the camera, preprocessing, inference, and display paths evaluated in the performance comparison.
Performance Compared With GStreamer
The RidgeRun benchmark compares GStreamer/DeepStream and HSDK on an NVIDIA Jetson AGX Orin with an IMX274 camera. The benchmark separates two workloads:
Capture and display, measured using Glass-to-Glass latency, process CPU, GPU utilization, and process memory.
Capture and inference without display, measured using software interlatency, inference processing time, CPU, GPU, and process memory.
The benchmark configuration was NVIDIA Jetson AGX Orin with Jetson Linux R39.2.1, MAXN power mode, Jetson clocks enabled, 1920 × 1080 input, and a 60 FPS target. The tested software versions were Holoscan SDK 5.0, GStreamer 1.24.2, and DeepStream 9.1.0. The inference workload used YOLO11n with TensorRT 10.16.2. Each configuration used five runs with 600 frames per run.}
Capture And Display
This test evaluates a complete camera-to-display pipeline. It measures Glass-to-Glass (G2G) latency, process CPU usage, GPU utilization, and process memory. The comparison includes three pipeline groups: GStreamer with Argus capture, HSDK with Argus capture, and HSDK with direct V4L2 capture.
The following graphic summarizes the six tested capture-and-display configurations. Each bar represents a complete configuration, including its capture interface and display path.
These results reflect three different capture paths. GStreamer and HSDK Argus use the Jetson ISP to process the camera stream, leaving more GPU capacity for other processing stages. HSDK V4L2 receives the raw Bayer data and performs additional Bayer processing on the GPU. This gives developers more control over the processing pipeline, but it can increase GPU utilization.
Argus is useful when ISP processing and lower GPU usage are priorities. Direct V4L2 is useful when the application needs more control over raw Bayer processing. The display method also affects latency and resource usage, so X11 and DRM should be treated as separate configurations. The capture results describe the complete camera-to-display path. Removing the display stage makes it possible to examine the processing and inference path separately.
Capture And Inference Without Display
To evaluate the processing path independently of display output, this test runs a headless YOLO11n inference pipeline. It compares DeepStream with Argus capture, and HSDK with V4L2 USERPTR capture on an NVIDIA Jetson AGX Orin with the IMX274 camera.
The graph compares software interlatency, inference processing time, process CPU utilization, GPU utilization, and process RSS. Since no display is included, the results focus on capture, preprocessing, and inference rather than display latency.
HSDK reported the lowest software interlatency in this group, and also the lowest measured inference processing time. This is a useful technical distinction: a lower value for one instrumented stage does not automatically mean a lower end-to-end software path.
G2G latency and software interlatency also answer different questions. G2G is an optical capture-to-display measurement. Software interlatency is measured between software stages. They are not interchangeable and should not be combined into one performance score.
The benchmark describes the tested configurations and measurement boundaries. It should not be presented as a universal ranking of HSDK against GStreamer. A fair comparison must identify the capture API, pixel format, preprocessing stages, inference model, display or sink, and measurement method.
From Benchmark Results to Real-World Applications
For engineers developing Edge AI applications, selecting the right processing framework involves more than comparing latency or GPU utilization. Camera interfaces, memory management, preprocessing requirements, and hardware integration all influence system performance.
HSDK provides a modular architecture for building sensor-processing and AI pipelines, while RidgeRun's custom operators demonstrate how the framework can be adapted to specific camera interfaces, CUDA processing requirements, and TensorRT inference workflows.
These capabilities are particularly relevant for engineering teams developing embedded vision and real-time AI applications on NVIDIA Jetson platforms.
How RidgeRun Can Help
Holoscan SDK provides the graph and runtime structure, but a real product still has to match a specific camera, sensor, model, board, display path, and performance target.
RidgeRun can help with camera and sensor integration, custom CUDA and HSDK operator development, TensorRT and model-pipeline integration, and optimization of memory movement, latency, and GPU usage. RidgeRun can also integrate HSDK with existing C++, CUDA, GStreamer, and device-specific code, then support the build, deployment, monitoring, and productization stages.
The practical value is adapting the graph to the customer’s hardware and requirements while keeping the data flow measurable, maintainable, and ready for deployment.
HSDK Developer Wiki
HSDK provides a structured way to build sensor-processing and edge-AI applications on NVIDIA platforms. Its graph model makes the data path visible, its operators divide the application into reusable processing stages, and its execution model separates composition, compilation, runtime, and validation.
The RidgeRun benchmark shows that performance depends on the complete pipeline: capture method, memory path, preprocessing, inference, display, and the measurement boundary. Understanding those details is essential before drawing conclusions from a framework comparison.
Our developer wiki is the complete technical guide for building sensor-processing and edge-AI applications with HSDK. It covers the SDK architecture, application lifecycle, build and validation process, tensor and memory handling, camera modules, operators, examples, and performance benchmarks.
It follows a practical development path, from understanding the SDK to building, running, and optimizing HSDK applications. For the complete guide, see the Holoscan SDK - Wiki.
Ready to optimize your Edge AI applications?
Contact RidgeRun to discuss your project.
