Skip to content

[android] Re-benchmark wav2vec2 bench app after onnxruntime-android 1.22.0 -> 1.28.0 (PR #205) to capture KleidiAI ARM64 gains #213

Description

@grzanka

Context

Dependabot PR #205 bumps com.microsoft.onnxruntime:onnxruntime-android 1.22.0 → 1.28.0 in
bench/android/wav2vec2/app/build.gradle. That app (BenchActivity.java:3-5,95) uses only the
stable Java bindings (OrtEnvironment, OrtSession, default empty SessionOptions) with
abiFilters "arm64-v8a" — plain CPU EP, no NNAPI/XNNPACK configured.

Across the 1.23.0–1.28.0 release notes, ARM64 CPU inference got several KleidiAI kernel upgrades
that apply automatically with no app code changes:

  • 1.23.0: enhanced SGEMM/IGEMM/dynamic-quantized-MatMul performance, esp. Conv2D on SME2 hardware
  • 1.25.0: BF16 SME2 kernel integration, asymmetric 4-bit MatMulNBits on ARM64
  • 1.28.0: new ARM64 2-bit weight CPU kernels

docs/android-asr-runtime-bench.md already has committed baseline numbers for this exact app on
onnxruntime-android 1.22.0. Since the model it loads (model_quantized.onnx, wav2vec2-base-960h) is
quantized, these are exactly the kernels that would benefit.

Ask

  1. Merge PR build(deps): bump com.microsoft.onnxruntime:onnxruntime-android from 1.22.0 to 1.28.0 in /bench/android/wav2vec2 #205 (onnxruntime-android 1.22.0 → 1.28.0).
  2. Rebuild bench/android/wav2vec2 and re-run the existing benchmark harness on the same
    device/test set used for the baseline numbers already in docs/android-asr-runtime-bench.md.
  3. Update docs/android-asr-runtime-bench.md with the new numbers, noting whether the KleidiAI
    kernel upgrades above produced a measurable speedup (or regression) versus the 1.22.0 baseline.

This is a "capture the free win" task, not a new feature — the version bump alone may already
improve on-device latency for this benchmark without any code change.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions