The JetsonHacks Github repository offers kernel modules which allow you to install and run a RealSense camera in around 5 minutes. Looky here:
Background
The Intel RealSense cameras were some of the pioneers in affordable RGBD cameras. RGBD meaning Color + Depth. In addition to a pair of infrared cameras, RealSense Cameras like the D435i use an infrared laser projector to help illuminate indoor scenes. This allows for better performance in low light situations. The RealSense cameras have on board compute to help build the depth map and overlaying the RGB camera image for a point cloud.
For several different computers, the RealSense cameras offer a near plug and play experience using the RealSense SDK. For the Jetsons, it’s always been a little bit more trouble to run natively. There is a work around which allows you to use the RealSense SDK from user space on the Jetson. But there is a tradeoff in speed and accuracy which Intel warns against in a production environment.
Prior to JetPack 6, in order to run RealSense natively you had to patch and recompile the kernel and enable some kernel modules on the Jetson. Let’s say this was unfun, and the user had to know a little more than they should to get things running smoothly.
A nicer solution
With the advent of JetPack 6, the game changed. The important RealSense changes all happen in loadable modules, which means the user doesn’t need to build and install a new kernel. So as long as we have the new pre-built modules, the install is straight forward.
What you need
To get started, you’ll need:
Your Jetson; Jetson Orin Nano (Super) Developer Kit or Jetson AGX Orin Developer Kit
RealSense Camera, such as a RealSense D435i (Amazon Link)
An Internet connection
If you really want to build the kernel modules …
The JetsonHacks Github repository has a build directory which contains instructions on how to patch and build the modules. Follow the directions there, we also do a walk through in the video above.
5 Minute Install
The key to the 5 minute install is the pre-built kernel modules which enables you to install and use the RealSense SDK from a Debian package.
Read through the README file in JetsonHacks Github repository for step by step directions. The outline is:
Untar the modules
Make sure the checksums match
Switch to the install_modules directory
Run install-realsense-modules.sh
Register the Intel Librealsense public key
Add the server to your Ubuntu repositories
Install librealsense2-utils
Install librealsense2-dev
Run realsense-viewer to verify everything works
Update the camera firmware if needed
Camera Format Changes
The patches for the kernel modules add depth camera frame format information, and frame accurate metadata for a variety of functions, such as IMUs. The camera formats added are:
Greyscale 16 L/R (Y16I)
FourCC: V4L2_PIX_FMT_Y16I
Description: Interleaved 16-bit greyscale format, typically for left/right stereo data.
GUID: UVC_GUID_FORMAT_Y16I
Depth data 16-bit (D16)
FourCC: V4L2_PIX_FMT_Z16
Description: 16-bit depth data (mapped to the same pixel format as Z16 but with a distinct GUID).
GUID: UVC_GUID_FORMAT_D16
Packed raw data 10-bit (W10)
FourCC: V4L2_PIX_FMT_W10
Description: 10-bit packed raw data, formatted as 8888[2222] (likely 8-bit groups with 2-bit padding).
GUID: UVC_GUID_FORMAT_W10
Confidence data (C )
FourCC: V4L2_PIX_FMT_CONFIDENCE_MAP
Description: Packed confidence data (two pixels per byte, [44] format), often paired with depth data.
GUID: UVC_GUID_FORMAT_CONFIDENCE_MAP
Raw data 8-bit (RAW8)
FourCC: V4L2_PIX_FMT_GREY
Description: 8-bit monochrome raw data, often used in fisheye or IR cameras.
GUID: UVC_GUID_FORMAT_RAW8
Raw data 16-bit (RW16)
FourCC: V4L2_PIX_FMT_RW16
Description: 16-bit raw data (legacy format for backward compatibility).
GUID: UVC_GUID_FORMAT_RW16
16-bit Bayer BGBG/GRGR
FourCC: V4L2_PIX_FMT_SBGGR16
Description: 16-bit Bayer pattern (BGGR ordering), typically for raw sensor data.
GUID: UVC_GUID_FORMAT_BAYER16
Z16 Huffman Compression (Z16H)
FourCC: V4L2_PIX_FMT_Z16H
Description: 16-bit depth data with custom Huffman compression.
GUID: UVC_GUID_FORMAT_Z16H
Frame Grabber (FG )
FourCC: V4L2_PIX_FMT_FG
Description: Format for frame grabber data (specific use case unclear, possibly diagnostic).
GUID: UVC_GUID_FORMAT_FG
SR300 Depth/Confidence (INZC)
FourCC: V4L2_PIX_FMT_INZC
Description: Planar format combining depth and confidence data, specific to SR300 cameras.
GUID: UVC_GUID_FORMAT_INZC
Relative IR (PAIR)
FourCC: V4L2_PIX_FMT_PAIR
Description: Relative infrared data, likely for stereo IR processing.
GUID: UVC_GUID_FORMAT_PAIR
These formats are added to support advanced imaging devices, such as Intel RealSense cameras (e.g., D435, SR300), which use VCSEL-based projectors for depth sensing.
Notes
In the video, the installation is being shown on a NVIDIA Jetson Orin Nano Super Developer Kit. The Intel RealSense D435i camera is attached to the Jetson.
The post Jetson Orin + RealSense in 5 minutes appeared first on JetsonHacks.
