Un noyau CUDA rapide ne suffit pas : NVIDIA passe un nœud ROS 2 en zéro-copie avec un agent IA

GPU acceleration alone does not guarantee a fast ROS 2 graph. You can write a blazing CUDA kernel , yet if messages still get serialized and copied through CPU memory as they flow between nodes, you erode the benefit of keeping perception and AI on the GPU. NVIDIA's September 22, 2026 Technical Blog makes this the headline: the bottleneck is not the kernel, it is the transport layer of the ROS 2 graph — Figure 1 shows the classic path (kernel finishes, payload drops to CPU, serialize, copy) while Figure 2 shows how flat the new path becomes, without breaking the standard ROS message.

What rosidl::Buffer is and what changed in ROS 2 Lyrical

In ROS 2 Lyrical , variable-length primitive array fields such as `uint8[]` are now represented in generated C++ as `rosidl::Buffer` . The default CPU-backed `rosidl::Buffer` behaves like the `std::vector` interface existing code expects, so source compatibility is preserved and the abstraction becomes pluggable : vendors can back the same field with externally managed storage without defining a new message type. NVIDIA's CUDA buffer backend puts GPU memory behind the same field using CUDA Virtual Memory Management (VMM) — when publisher and subscriber meet the runtime requirements ( same host, same CUDA device, same Linux user , supported RMW like `rmw_fastrtps_cpp` or `rmw_zenoh_cpp`) the payload moves between co-located nodes without serialization or host copies ; otherwise ROS 2 falls back to the CPU path automatically.

Why Depth Anything 3 is the ideal example node

The tutorial anchors on the Depth Anything 3 (DA3) TensorRT ROS 2 node — the ByteDance Seed model that predicts spatially consistent geometry from arbitrary views. The current callback is straightforward: convert the incoming ROS image to an OpenCV view, run monocular metric-depth inference with TensorRT , convert the `cv::Mat` back to a ROS image and publish as 32FC1 floating-point depth (Video 1 shows RGB-to-depth). The algorithm is already GPU-accelerated, which is exactly why it is useful: a CPU-backed ROS boundary surrounds a GPU-native core , and those two payload-sized host transfers + host allocation + serialization are the optimization at the interface.

The goal is not to redesign the model or replace the message — keep the existing ROS contract and let the output `Image.data` field carry storage from the appropriate backend. Same `sensor_msgs/msg/Image`, same topic, same API — just the backing can now be CUDA-backed when both ends allow it; on an edge platform like Jetson AGX Thor , that is how you win milliseconds without rewriting the perception pipeline.

The agentic audit: the migrate-node-to-rosidl-buffer skill

The harder task is finding the right boundaries to update — allocations, serialization, stream ownership, fallback. That audit is investigative work , well-suited to an AI coding agent . The purpose-built migrate-node-to-rosidl-buffer skill turns it into a repeatable workflow: instead of replacing the node with a template, it directs the agent to follow payloads through callbacks and helpers, preserve the node contract, and coordinate source, dependency, launch, and test changes.

The skill walks the agent through: record the starting revision, target ROS environment and local changes , confirm the field type is compatible and add CUDA backend packages as dependencies , trace each field from receipt to publication including transitive CUDA calls, strides, streams, optional outputs and ownership, run the read-only copy-boundary audit and inspect each result in context, produce a per-field migration plan identifying removed copies and required promotions/materializations, implement the smallest interface-preserving patch , and independently verify semantics, backend negotiation, separate-process transport, buffer lifetime and actual memory-copy behavior.

Anatomy of the refactor: one option, one allocation, two handles

The resulting patch is deliberately small. Most changes adapt the TensorRT wrapper to accept CUDA buffer handles; the ROS transport change is just one subscription option, one CUDA allocation, two stream-aware handle extractions and one publish — no custom message, no duplicate CUDA topic, no CPU/CUDA publisher branch is required and the graph stays standard. The message stays `sensor_msgs/msg/Image`; the package adds `cuda_buffer` and `cuda_buffer_backend` as deps, and the subscription is updated with `rclcpp::SubscriptionOptions options; options.acceptable_buffer_backends = "cuda";` so the subscriber can accept CUDA-backed buffers while CPU remains an acceptable fallback and the existing `image_transport` + `message_filters` topology is preserved.

In the callback the output message gets CUDA-backed storage via ` cuda_buffer_backend::allocate_buffer() ` on `Image.data`; the TensorRT stream from `tensorrt_depth_anything_->getCudaStream()` feeds ` from_input_buffer() ` for read-only input and ` from_output_buffer() ` for write output — both stream-safe CUDA handles . The existing CUDA postprocess writes its final 32FC1 result directly into the outgoing message's buffer via the write handle, avoiding a device-to-host copy and an intermediate device-to-device buffer ; an inner scope releases the handle before publish to record a write CUDA event and order the operations, then `pub_depth_image_->publish(std::move(depth_msg))` runs as usual while sharing is handled automatically by middleware + backends .

The skill deliberately keeps point-cloud construction and debug visualization as separate, optional host consumers. When enabled they may still need a device-to-host copy and synchronization , but they do not define the representation published on the depth topic, so leaving them as explicit optional boundaries keeps the optimized publication path clean — a small architectural judgment that avoids complicating the fast path for rarely used features.

Build, run and ship to Jetson AGX Thor

The feature landed in ROS 2 Lyrical , so the migrated node works on Lyrical and above with supported RMWs (`rmw_fastrtps_cpp`, `rmw_zenoh_cpp`); message type and core functions stay the same, only `cuda_buffer` and `cuda_buffer_backend` are added as deps. To enable the backend, build the plugins from source : `git clone https://github.com/ros2/rosidl_buffer_backends.git`, `colcon build --symlink-install --packages-up-to cuda_buffer_backend`, `source install/setup.bash`, then `colcon build --symlink-install --packages-up-to depth_anything_v3` and `export RMW_IMPLEMENTATION=rmw_fastrtps_cpp` before running the same launch file — core `rosidl::Buffer` is already in Lyrical, no core rebuild needed, the backend is a plugin shipped in Isaac ROS 5.0 targeting Jetson AGX Thor .

Verify: did the host copies really disappear?

Verification targets the transport, not the compute: use NVIDIA Nsight Systems to inspect GPU activity — on an eligible CUDA path there should be no payload-sized host-to-device or device-to-host transfers at the ROS boundary, and you should record before/after latency (Figure 3 shows the migrated DA3 node preserving the RGB-to-depth result while using CUDA-backed storage). For backend negotiation, `msg->data.get_backend_type()` should report "cuda" when both ends meet requirements — the example subscriber logs `received backend=%s` and throws if not `cuda`, while production code typically accepts CPU fallback and lets `from_input_buffer()` handle the CPU-to-GPU promotion internally. The skill can also generate source and sink nodes to test both pipelines without code changes: a CPU-publishing source arrives as CPU-backed and is promoted to CUDA in the callback, while a CUDA-publishing source arrives and is consumed via CUDA handles without extra copies — proving the single code path works for both.