Skip to content

perf(jit): consolidate feedback and tier-state lookups - #20

Merged
huacnlee merged 3 commits into
mainfrom
perf/m3-feedback-lookups
Sep 6, 2026
Merged

perf(jit): consolidate feedback and tier-state lookups#20
huacnlee merged 3 commits into
mainfrom
perf/m3-feedback-lookups

Conversation

@huacnlee

@huacnlee huacnlee commented Sep 5, 2026

Copy link
Copy Markdown
Member

Issue # (if available)

Follow-up to merged #19; review this diff against main.

Description of changes

Consolidate observe_type and observe_call into one B-tree entry lookup, preserving occupied-at-capacity updates, rejected-entry accounting, exact feedback versions, and independent call-slot growth. Cache baseline tier state once after maintenance in record_hot; successful direct refresh returns before that cached value is reused.

Add three characterization tests covering the capacity, generation, version and megamorphic edges. Deliberate capacity and arity mutations fail the corresponding tests. Validation: 518 release runtime tests, 82 targeted ASAN/LSAN tests, workspace Clippy and formatting pass; independent code review found no issues.

Measured performance

Compared with a38bd01 (merged #18), the cumulative #19 + #20 changes improve generic-call Tier 1 speed by 19.3%, generic-call automatic by 18.1%, and recursive Tier 1 by 18.7%. The measured candidate is 99a1e78; the conflict-resolution merge retains that exact content tree. These are cumulative results, not the incremental gain of #20 over the current main containing #19.

The values below are steady-state latency converted to speed relative to the named baseline: 1.00x is parity; higher is faster. Intervals are paired 95% bootstrap confidence intervals.

Workload Requested mode / actual execution Speed vs. a38bd01 Speed improvement 95% interval
Generic calls Tier 1 1.193x 19.3% faster 1.190x–1.197x
Generic calls Tier 2 / actually Tier 1 1.163x 16.3% faster 1.160x–1.165x
Generic calls Automatic / actually Tier 1 1.181x 18.1% faster 1.175x–1.185x
Recursive Fibonacci Tier 1 1.187x 18.7% faster 1.183x–1.189x
Scalar loop Tier 2 1.035x 3.5% faster 1.028x–1.038x
Scalar loop Automatic 1.041x 4.1% faster 1.037x–1.047x

Controls and limitations:

  • All direct-call modes, property-access modes, and interpreter comparisons are statistically tied. For example, direct-call Tier 2 ranges from 1.9% slower to 0.2% faster; property automatic ranges from 0.24% slower to 0.15% faster.
  • Scalar Tier 1 is tied, between 2.3% slower and 0.6% faster. Recursive requested Tier 2 and automatic are also tied; automatic recursion has zero native entries and remains an interpreter-path control.
  • The scalar-loop improvements have no isolated causal attribution. These measurements neither establish an overall no-regression result nor replace the complete clean-source acceptance matrix.
  • Later rooted-property-guard changes and their measurements are outside this PR and are not included in this table.

Method: five discarded warmup processes, 30 fresh-process latency pairs per mode, ten throughput windows of at least one second per engine/mode, and 10,000 paired bootstrap resamples, pinned to CPU 0. Both runtimes use the identical corrected benchmark harness. Engine order is balanced independently of mode order (15/15 first positions for latency; 5/5 for throughput). Checksums, native entry/exit balance, and executable/source hashes were audited. The recorded runs use dirty-source diagnostic provenance; throughput windows count fresh workers including setup and warmup, not JavaScript operations.

Raw samples, exact driver/harness source, hashes, and intervals:
generic calls,
direct calls,
recursive Fibonacci,
scalar loop,
property access.

Archive raw evidence and a property profile: helper-frame validation remains the main sampled property cost. Document the next bounded ownership optimization and its safety constraints; property lowering itself is unchanged in this PR.

Checklist

  • Added change to the changelog
  • Created unit tests for my feature if needed

Base automatically changed from perf/m3-native-timing-stack to main September 6, 2026 02:48
@huacnlee
huacnlee merged commit 47aeb11 into main Sep 6, 2026
42 checks passed
@huacnlee
huacnlee deleted the perf/m3-feedback-lookups branch September 6, 2026 03:17
huacnlee added a commit that referenced this pull request Sep 6, 2026
### Issue # (if available)

N/A

### Description of changes

Archive a fresh comparison of repository-pinned QuickJS-ng, quickjs-jit
automatic mode, and Bun 1.4.0 default configuration after #19 and #20
merged into clean main `47aeb11`. Include forced Tier 1/2 diagnostics,
raw samples, methodology/provenance, CSV, confidence intervals, charts,
and an evidence-based next-stage optimization roadmap. No runtime
behavior changes.

Automatic JIT is faster than the interpreter in 8 workloads,
statistically tied in 1, and slower in 13, based on per-workload paired
95% intervals. Selected results below express **speed relative to the
QuickJS-ng interpreter** (1x = parity):

| Workload | JIT automatic speed [95% CI] | Bun default speed |
|---|---:|---:|
| scalar-loop | 24.024x [23.382, 24.316] | 47.26x |
| float64-dense | 7.819x [7.708, 7.895] | 6.27x |
| property-heavy | 0.268x [0.267, 0.271] | 3.87x |
| generic-call-entry | 0.135x [0.134, 0.136] | 57.85x |
| arrays-typed | 0.656x [0.654, 0.658] | 10.41x |

These are current-protocol results, not equally warmed peak-engine
measurements: QuickJS and Bun differ in process-internal warmup and host
timing boundaries. Native counters include warmup. Bun uses a recorded
launcher that removes the runner's `--smol` flag. The unmerged property
candidate `9ccda01` is excluded.

The roadmap prioritizes aligned measurement and steady-state counters,
property and generic-call costs, automatic tiering/fallback overhead,
then cold start and library hotspots. Property and generic-call stages
propose a paired speed lower bound of 1.5x versus the same-harness main
baseline, with eventual interpreter parity or better.

Evidence: [full comparison](benchmarks/results/main-47aeb11-engines.md),
[raw samples](benchmarks/results/main-47aeb11-engines.json),
[methodology](benchmarks/results/main-47aeb11-methodology.json), [next
targets](docs/PERFORMANCE_NEXT.md).

Validation:
- Completed 22 workloads × 5 modes × 30 latency samples, with 5
discarded warmup processes and 10 one-second throughput windows per
workload/mode.
- Audited all 3,300 latency samples, checksums, entry/exit counters,
source/binary hashes, derived results, links, and chart rendering;
independent report review found no substantive issue.
- Staged whitespace check passes; CSV/SVG formatting normalization
preserves data/path tokens. No runtime tests rerun for this
documentation/data-only commit.
- **Performance acceptance remains failing** (runner exit 2 after
complete output): compute speed lower bound is below 5x, automatic
decision evidence is missing, and startup/definition-eval proxy budgets
fail. The gpui-shell gate is inconclusive because external host evidence
is absent. Raw gate output is preserved.

### Checklist

- [x] Changelog assessed: not applicable to benchmark evidence and
planning documents.
- [x] Unit tests assessed: no runtime changes; artifact and data
validation completed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant