Jetson AGX Orin validation and support tracking (JetPack 6, sm_87)

[xiao-yang25]

Motivation

Embodied.cpp targets edge deployment and lists Jetson-class devices as a
supported deployment platform. I have access to a Jetson AGX Orin 64GB device
and would like to validate the current runtime on a real JetPack 6 environment.

The goal is to produce a reproducible Orin build and inference report, fix
platform-specific compatibility issues, and document the verified setup for
other Jetson users.

Test environment

• NVIDIA Jetson AGX Orin Developer Kit 64GB
• Architecture: aarch64 / CUDA compute capability 8.7
• JetPack 6 / L4T r36.4.4
• Ubuntu 22.04
• CUDA 12.6
• cuDNN 9.3
• CMake 3.22.1
• GCC 11.4
• MAXN power mode

Proposed validation scope

1. Build validation

• Configure a CUDA build with CMAKE_CUDA_ARCHITECTURES=87
• Build the VLA server on aarch64
• Build the WAM CUDA kernels where model artifacts are available
• Record build time, peak memory usage and generated artifact size
• Document all required JetPack/Ubuntu dependencies

2. Runtime validation

• Start the server and load a GGUF model successfully
• Run a deterministic smoke-test request
• Verify output shape, finite values and repeatability
• Check CPU/GPU memory usage during model loading and inference
• Check for CUDA errors, unsupported kernels or architecture-specific fallbacks

3. Performance characterization

• Measure model-loading time
• Measure warm-up and steady-state inference latency
• Report peak unified-memory usage
• Report power mode, clocks and active background workload
• Clearly separate functional validation from controlled performance results

Performance measurements will only be reported from a sufficiently idle system;
initial testing may be limited to build and functional validation.

Initial findings from repository inspection

Before the full validation, I found several Jetson-related areas that may need
attention:

1. The README's explicit CUDA architecture examples omit 87, although the WAM
   CMake defaults already include it.
2. The README defaults to CUDA_ARCH=native; the documented fallback for CMake
   versions older than 3.24 is relevant because JetPack 6/Ubuntu 22.04 provides
   CMake 3.22.1 by default.
3. The optional LingBot-VA and Cosmos3 cuDNN discovery is restricted to CUDA
   toolkit and x86_64-specific paths with NO_DEFAULT_PATH. JetPack normally
   installs cuDNN under aarch64 multiarch paths.
4. The Cosmos3 source currently includes cudnn.h using an absolute
   /usr/local/cuda-11.6/include/cudnn.h path.

These are preliminary findings. I plan to confirm each one with actual build
logs before proposing code changes.

Proposed deliverables

I would like to submit a PR containing:

• Necessary aarch64/JetPack build fixes confirmed by testing
• A documented AGX Orin build command using sm_87
• A minimal inference smoke-test procedure
• An Orin validation report with software versions and resource measurements
• No changes to public APIs or model behavior

Questions

1. Which model would maintainers prefer as the first Jetson validation target?
   pi0.5 appears suitable for initial VLA build and inference validation.
2. Should platform-specific fixes be submitted as one Orin support PR or as
   smaller independent PRs?

I am happy to carry out the testing and submit the corresponding fixes and
documentation.

[feather1023]

Sorry for the delayed response(We got a long holiday since V1.0 released. : > ), and thank you for preparing such a detailed Jetson validation proposal.

We reviewed the current repository and confirmed that the issues you identified are relevant:

1. The README does not currently list CUDA architecture 87, which is required for Jetson AGX Orin.
2. The documented CUDA_ARCH=native default is not suitable as the primary path for the CMake 3.22.1 environment commonly provided by JetPack 6; an explicit 87 architecture should be used.
3. The optional LingBot-VA and Cosmos3 cuDNN discovery logic is currently tied to CUDA/x86_64-specific paths and may not locate the aarch64 cuDNN installation provided by JetPack.
4. The Cosmos3 WAM VAE CUDA source still contains a hard-coded CUDA 11.6 cuDNN header path.

We agree that Jetson AGX Orin should be validated on a real JetPack 6 environment. We suggest the following staged plan:

Phase 1: VLA build and functional validation

• Use pi0.5 as the first validation target.
• Configure an aarch64 CUDA build with CMAKE_CUDA_ARCHITECTURES=87.
• Build the VLA server on JetPack 6 / CUDA 12.6.
• Load a compatible GGUF checkpoint and run a deterministic smoke test.
• Verify output shape, finite values, repeatability, CUDA error behavior, and model-loading success.
• Record build time, binary size, model-loading time, inference latency, and peak unified-memory usage.

Phase 2: Portable Jetson compatibility fixes

• Document an explicit Jetson/Orin build command using sm_87.
• Avoid relying on native as the only architecture-selection path for older CMake versions.
• Replace hard-coded cuDNN include paths with portable CMake target/include discovery.
• Allow JetPack aarch64 multiarch locations to be found without assuming x86_64 paths.
• Remove the hard-coded Cosmos3 CUDA 11.6 cuDNN header include.
• Keep these changes limited to build configuration and platform compatibility; no public API or model behavior changes are intended.

Phase 3: WAM validation

• Build the LingBot-VA and Cosmos3 CUDA kernels where compatible model artifacts are available.
• Run the corresponding smoke tests and check for unsupported kernels or architecture-specific fallbacks.
• Separate functional validation from controlled performance measurements.

Phase 4: Documentation and report

• Add a Jetson AGX Orin setup section to the README or evaluation documentation.
• Include the exact JetPack, CUDA, cuDNN, compiler, CMake, power-mode, and architecture settings.
• Add a reproducible smoke-test command and summarize the measured resource usage.
• Clearly distinguish functional validation results from performance results collected on an idle system.

For the contribution structure, we recommend two smaller PRs:

1. A platform-compatibility PR containing the CMake, cuDNN discovery, sm_87, and documentation fixes.
2. A validation-report PR containing the Orin test procedure, logs, measurements, and model-specific results.

If the Cosmos3 changes require substantial model-specific implementation work, they can be submitted separately from the general Jetson compatibility fixes.

We agree that pi0.5 is the preferred first target. Thank you again for carrying out this validation work, and apologies again for the delayed response.